2019 ICH CAHPS Spring Survey Site Visit Issues & Data Quality Reminders
folder_openOther Announcementscalendar_todayPosted August 14, 2019

The ICH CAHPS Coordination Team (CT) conducted site visits with some of the ICH CAHPS Survey vendors during the 2019 Spring Survey period. Overall, this was the most successful site visit period yet! However, we did identify some issues in the 2019 ICH CAHPS Spring Survey operations that we want to generally share.

A list of the most common issues observed during the site visits is provided below. Please review the information below and make sure that your organization is applying the required protocols described in the ICH CAHPS Survey Administration and Specifications Manual, Version 7.0 (posted on the ICH CAHPS website on February 6, 2019) to all aspects of the survey, including data collection, data processing, and construction of the XML files in the current and subsequent survey periods.

Please contact the ICH CAHPS Coordination Team via email at ichcahps@rti.org or call (866) 245-8083 if you have any questions about these data quality reminders or need further clarification.

Observations during Site Visits

• Make sure checks are being performed on the sample file received from the Coordination Team.During site visits we observed that some vendors are not conducting adequate checks on the sample file received to ensure they are receiving data for the facilities they expect (and only from facilities they are expecting). This includes utilizing the facility non-participation form (NPF) file and the sample summary file the CT provides.
• Make sure non-completed mail and phone surveys are dispositioned correctly. Some non-complete cases are being coded with the incorrect final disposition code. Please take time to review section 9.2 of the ICH CAHPS Survey Administration & Specifications Manual, Version 7.0 to make sure all systems and individuals are assigning the correct final disposition code based on the rules set out in this section.
• Implement quality checks for both scanned and keyed data. The CT observed that some vendors had recorded “M” for missing for questions when in fact there were marked responses on returned mail surveys. One example involved missed light marks due to the scanning process being done in black and white. Upon discovery, the vendor changed it to grayscale to better capture these lighter marks. Another example included an instance where a sample patient had clearly underlined their chosen response; however, the keyer missed it. Both examples highlight the importance of implementing the required QC measures on scanned and keyed survey data:
  • If using an optical scanning program, vendors must re-scan a minimum of 10 percent of the questionnaires received and compare the responses scanned from the first scanning to the re-scanned responses.
  • In addition, survey vendors must compare the scanned data for a minimum of 10 percent of the completed questionnaires received against the responses marked in the actual hardcopy questionnaire to make sure the scanning program is scanning the responses that are marked.
  • For data that are keyed into a data entry program, all questionnaires must be 100 percent rekeyed to ensure that all entries are accurate. Vendors must compare the responses keyed in the first and second keying. This review should be done by a third party – that is, someone other than the data entry operators who keyed the data. That person must resolve any discrepancies identified.
• Compare raw CATI data against the XML file data. Per protocol, vendors must compare a sample of cases on the XML file to the original CATI data file, to ensure the data on the XML file are accurate.
• Spread phone contact attempts with sample patients across the data collection period. During the site visits we noted that some vendors were starting telephone call attempts later in the data collection period and/or ending telephone call attempts earlier in the data collection period for cases coded a 250 (no response after maximum attempts). As a reminder, the 10 contact attempts must be made on different days of the week and at different times of the day, including nights, afternoons, and mornings, and spread over the 12-week data collection period. Additionally, we strongly encourage all vendors with Saturday/Sunday operating hours to incorporate ICH CAHPS calls into those days. Given our patient population, we know that it may be harder to reach them during the week when they are receiving their dialysis and instead may be more prone to answer the phone and have time to complete the survey on the weekends.
• Vendors must have a distressed respondent protocol in place for all telephone staff. In addition to telephone interviewers, toll-free customer support staff for both telephone and mail administration must be trained on and provided a distressed respondent protocol for use when needed.
• Recontact sample patients for callbacks and scheduled appointments. If a sample patient is reached but is unable to speak with the telephone interviewer at that time, and he/she requests that a telephone interviewer call back at a different date/time (for either a callback or scheduled appointment), vendors must make a concerted effort to recontact the respondent on that requested date/time.