Menu

Announcement

2016 ICH CAHPS Spring Survey Issues from Site Visits
folder_openData Submissionscalendar_todayPosted July 15, 2016

Over the past few months, the ICH CAHPS Coordination Team conducted site visits with some of the ICH CAHPS Survey vendors. Overall, the site visits went well; however, we identified some issues in the 2016 ICH CAHPS Spring Survey operations and in the data submitted for the 2015 Fall Survey.

A list of the most common issues observed during the site visits is provided below. Please review the information below and make sure that your organization is applying the required protocols described in the ICH CAHPS Survey Administration and Specifications Manual, Version 4.0 to all aspects of the survey, including data collection, data processing, and construction of the XML files in the current and subsequent survey periods.

As a reminder, the XML files containing data from the 2016 Spring Survey must be submitted to the ICH CAHPS Data Center by 11:59 PM on July 27, 2016. Please contact the ICH CAHPS Coordination Team via email at ichcahps@rti.org or call (866) 245-8083 if you have any questions about these data quality reminders or need further clarification.

  1. Implement a password policy. It was noted that some vendors do not have an official password policy in place to ensure the security of ICH CAHPS data. Vendors are strongly urged to implement a password policy that requires their employees to create and use strong passwords that must be changed on a regular and frequent basis.
  2. Implement a policy for returning important non-survey items to sample patients. All vendors should implement a policy to return important documents to sample patients that were inadvertently included in the questionnaire package that the sample patient returned to the vendor. This might include things like a check or other type of payment to the facility, medical supplies, or other items that need to be sent to the appropriate place. Vendors are not permitted to send such items onto a facility on behalf of a patient. Instead, vendors must send these items back to the sample patient with a note indicating that the item was inadvertently included in the ICH CAHPS Survey package, and it is being returned to the sample patient so that he or she can send it to the intended recipient.
  3. Implement required quality check on outgoing mail survey materials. As a reminder, vendors must check a minimum of 10 percent of all outgoing questionnaire packages to ensure that all package contents are included and that the same unique SID number appears on both the cover letter and the questionnaire. Vendors must also check a sample of the printed materials to make sure that there are no bleed-through text, missing pages in the survey questionnaire, or other problems with the printed materials.
  4. Administer the correct version of the mail survey and the telephone survey. Survey vendors must make sure that they are using the correct version of the ICH CAHPS mail survey and telephone interview script. We observed that some vendors had not implemented the changes that were included in the version of the mail survey and telephone scripted posted on the ICH CAHPS website.
  5. Retrain telephone interviewers on observed pronunciation issues. ICH CAHPS Survey vendors should review the correct pronunciation of all words and phrases in the ICH CAHPS Survey questionnaire with their telephone interviewers when interviewers are trained, and then check for correct pronunciations of words and phrases when monitoring telephone interviews conducted by each interviewer. If telephone supervisory staff observes an interviewer mispronouncing words in the survey during when monitoring telephone interviews, the supervisor should retrain the interviewer following the monitoring session, and before additional interviews are conducted by the interviewer.
  6. Make sure interviewers are trained on correct probing techniques. Survey vendors must train telephone interviewers on how and when to probe a respondent for more information or to clarify a response. Make sure that telephone interviewers do not probe if the respondent has already given a response to the question and the response given is clear. The Coordination Team observed some telephone interviewers who seemed unsure of when to employ probing techniques, and some interviewers probed when it was not necessary to do so. Telephone interview training should include a section on the use of effective neutral or nondirective probing techniques. Such techniques will assist the interviewer with obtaining a more complete or more specific answer from a respondent, when needed, without the unintended consequence of suggesting answers or leading the respondent.
  7. Make attempts to contact a sample patient on different days of the week and at different times of the day. During the site visits we noted that some vendors are not making telephone call attempts during the evening hours while other vendors are not making attempts during the morning hours. Survey vendors must make a maximum of 10 telephone contact attempts for each sample patient, unless the sample patient refuses or the survey vendor learns that the sample patient is ineligible to participate in the survey. The 10 contact attempts must be made on different days of the week and at different times of the day, including nights and mornings, and spread over the data collection period.
  8. Implement all required quality checks for both scanned and keyed data. The Coordination Team observed that some vendors are not implementing the required ICH CAHPS quality control measures on scanned and keyed survey data.
    • If using an optical scanning program, vendors must re-scan a minimum of 10 percent of the questionnaires received and compare the responses scanned from the first scanning to the re-scanned responses.
    • In addition, survey vendors must compare the scanned data for a minimum of 10 percent of the completed questionnaires received against the responses marked in the actual hardcopy questionnaire to make sure the scanning program is scanning the responses that are marked. c
    • For data that are keyed into a data entry program, all questionnaires must be 100 percent rekeyed to ensure that all entries are accurate. Vendors must compare the responses keyed in the first and second keying. This review should be done by a third party – that is, someone other than the data entry operators who keyed the data. That person must resolve any discrepancies identified.
  9. Check all systems, programs and equipment used in data collection and processing to make sure they are operating correctly. Some vendors’ optical scanning equipment was not correctly scanning the responses marked in completed surveys. Vendors are strongly urged to check all of their systems, computer programs, and equipment (including optical scanners) used to administer the ICH CAHPS Survey on a regular basis to ensure that all are working properly and as intended. Vendors should also check to make sure that the scanning parameter or settings are large enough to scan response options that are not directly inside the circle or box for the response option, and that the scanner is sensitive enough to pick up marked responses that might be lighter than some others.
  10. Verify that the correct final disposition code has been assigned to each sample case. We observed that some vendors had discrepancies between the disposition code listed in their data system and the code entered in the XML file submitted to the Data Center. Survey vendors are required to check to make sure that the correct final disposition code has been assigned to each sample case. Vendors should ensure that the assigned final disposition code is correct and that the data included in the XML file are accurate based on the final code.
  11. The correct use of Code X vs. Code M. Prior to submission of their XML files, vendors are asked to review data on each XML file to make sure that Code M and Code X are appropriately assigned. As a reminder, when follow-up questions are appropriately skipped by the respondent, the follow-up question response should be coded as “Not Applicable,” which is Code X. When follow-up questions should have been skipped (based on the response to the screening question) but are answered, scan or key the response that the respondent provides. If a question should have been answered but was not, assign Code M for “Missing” to the survey item.
  12. Guidelines for coding both multiple responses and ambiguous responses. If the respondent marked two or more response options for a single-response option question, when coding the responses select the one that appears darkest. If it is not possible to make that determination, leave the response blank and code it as “Missing” (Code M) rather than guess. In addition, all responses or response boxes that are not circled, checked, underlined, or in some other way clearly designated by the respondent as his or her intended response must be coded M for “Missing”.