Menu
General Information  /  Announcements
Back
breadcrumb shadow

Announcement

2018 ICH CAHPS Fall Survey Site Visit Issues & Data Quality Reminders
folder_openOther Announcementscalendar_todayPosted January 9, 2019

Over the past few months, the ICH CAHPS Coordination Team conducted site visits with some of the ICH CAHPS Survey vendors. Overall, the site visits went very well; however, we identified some issues in the 2018 ICH CAHPS Fall Survey operations.

A list of the most common issues observed during the site visits is provided below. Please review the information below and make sure that your organization is applying the required protocols described in the ICH CAHPS Survey Administration and Specifications Manual, Version 7.0 (to be posted on the ICH CAHPS website in early February 2019) to all aspects of the survey, including data collection, data processing, and construction of the XML files in the current and subsequent survey periods.

As a reminder, the XML files containing data from the 2018 Fall Survey must be submitted to the ICH CAHPS Data Center by 5:00 PM on January 30, 2019. Please contact the ICH CAHPS Coordination Team via email at ichcahps@rti.org or call (866) 245-8083 if you have any questions about these data quality reminders or need further clarification.

Observations during Site Visits

  • Make sure updated version of Pre-Notification Letter is sent to all sample patients.During site visits we observed that some vendors were still using a prior version of the letter instead of the one revised prior to the 2018 Spring Survey. Recall that this letter was specifically revised to read at a lower reading level, making it more accessible to our ICH CAHPS sample patients.
  • Make sure all telephone interviewers are trained on probing techniques and general telephone interviewing procedures. Some vendors’ telephone interviewers were not implementing probing techniques when the respondent provided an unclear response.  In addition, some telephone interviewers were not reading the response options in their entirety (such as when the respondent interrupted the interviewer).  Vendors should be providing immediate feedback to interviewers when issues such as these are determined during live monitoring.
  • Administer the correct version of the mail survey, the telephone survey, and other survey materials. Survey vendors must make sure that they are using the correct version of the ICH CAHPS mail survey and telephone interview script. During site visits we observed that some vendors had not implemented the changes that were included in the most recent versions of the mail survey and telephone script posted on the ICH CAHPS website.
  • Spread phone contact attempts with sample patients across the data collection period. During the site visits we noted that some vendors were ending telephone call attempts earlier in the data collection period for cases coded a 250 (no response after maximum attempts). As a reminder, the 10 contact attempts must be made on different days of the week and at different times of the day, including nights, afternoons, and mornings, and spread over the 12-week data collection period.
  • Vendors must be in possession of a signed confidentiality agreement from all staff, including subcontractor staff. During the site visits we noted that some vendors did not have in their possession copies of all signed confidentiality agreements from subcontractor staff. If a vendor’s subcontractor is restricted from providing the vendor with copies of the signed agreements, the vendor should obtain from the subcontractor a list with the names of the individuals who have signed the agreement.
  • Recontact sample patients for callbacks and scheduled appointments. If a sample patient is reached but is unable to speak with the telephone interviewer at that time, and he/she requests that a telephone interviewer call back at a different date/time (for either a callback or scheduled appointment), vendors must make a concerted effort to recontact the respondent on that requested date/time.
  • Vendors must verify each mailing address/phone number that is included in the sample file provided by the Coordination Team using a commercial address/phone update service. Some vendors indicated they are not verifying the contact information included in the sample files with a commercial address/phone update service.
  • Vendors must restart the 10 call attempts if the vendor receives a new telephone number for the sample patient. A total of 10 call attempts must be made on the updated telephone number, if there is enough time left in the data collection period after the new number is identified.  If the new number is identified later in the data collection period, survey vendors should use their best judgment in implementing the number of attempts, keeping in mind the rule that they may make more than one attempt in one 7-day period, but cannot make all 10 attempts in one 7-day period.
  • Log receipt of completed questionnaires so that respondents do not receive a second questionnaire/telephone follow-up. Some vendors had a lag in the logging of received questionnaires, such that sampled patients received a follow-up survey.  Completed questionnaires received must be logged into the tracking system in a timely manner to ensure that they are taken out of the cases being rolled over to receive a second mail survey or telephone follow-up activity.
  • Implement all required quality checks for both scanned and keyed data. The Coordination Team observed that some vendors are not implementing the required ICH CAHPS quality control measures on scanned and keyed survey data.
    • If using an optical scanning program, vendors must re-scan a minimum of 10 percent of the questionnaires received and compare the responses scanned from the first scanning to the re-scanned responses.
    • In addition, survey vendors must compare the scanned data for a minimum of 10 percent of the completed questionnaires received against the responses marked in the actual hardcopy questionnaire to make sure the scanning program is scanning the responses that are marked.
    • For data that are keyed into a data entry program, all questionnaires must be 100 percent rekeyed to ensure that all entries are accurate. Vendors must compare the responses keyed in the first and second keying. This review should be done by a third party – that is, someone other than the data entry operators who keyed the data. That person must resolve any discrepancies identified.