ICH CAHPS Semiannual Data Quality Reminders
folder_openOther Announcementscalendar_todayPosted January 11, 2021

As we approach the deadline for submitting data from the 2020 ICH CAHPS Fall Survey and the beginning of the 2021 Spring Survey, we would like to remind vendors of the importance of implementing quality control measures on all aspects of the survey.

This announcement includes a list of some of the data quality issues that the Coordination Team detected in the limited data from the 2020 Spring Survey and the few issues we observed during virtual and offsite site visits with survey vendors during the 2020 Fall Survey. As you conduct final data cleaning and processing of the 2020 Fall Survey data and prepare the XML file that will be submitted, please apply appropriate quality control measures so that you can detect and avoid these errors and data problems in all upcoming survey periods.

As a reminder, the XML files containing data from the 2020 Fall Survey must be submitted to the Data Center by 5:00 PM EST on Wednesday, January 27, 2021. Please contact the Coordination Team via email at ichcahps@rti.org or call (866) 245-8083 if you have any questions about any of the information provided below.

Data Quality Reminders (based on 2020 Fall Site Visits and 2020 Spring Data Submission)

1. Check Assigned Disposition Codes. Survey vendors should make sure that the correct final disposition codes are assigned to all sample cases. Within 2 to 3 weeks after the data collection period begins and then after data collection ends (at a minimum), survey vendors should generate and review the interim and final disposition codes assigned to a sample of cases. Once final disposition codes are assigned, vendors should have a staff member (other than the one who assigned the disposition codes) conduct quality control on the disposition codes assigned.

   The Coordination Team conducts quality control on each vendor’s data files after they are submitted. The following are some of the issues/errors that were found:

   1. Some cases were coded a 130 (Completed Mail Survey – Eligibility Unknown) but the respondent reported in Q2 that they no longer receive care at the facility and skipped to Q45 as instructed. The cases should have been coded as 190 (Ineligible: No Longer Receiving Care at Sampled Facility).
   2. Some cases were coded a 130 (Completed Mail Survey – Eligibility Unknown) but the responses to Q1 and Q2 indicated the cases should have been coded a 160 (Ineligible – Does Not Meet Eligibility Criteria).
   3. Some telephone cases were coded as 240 (Wrong, Disconnected, or No Telephone Number) but these cases had data; 240s should have no data and another code should apply.
   4. Some mail cases were coded as 110 (Completed Mail Survey) but the response to Q62 indicated they were completed by a proxy. These cases should have been coded a 199 (Survey Completed by Proxy Respondent) as proxy respondents are not permitted on ICH CAHPS.
   5. Some telephone cases were coded as 210 (Breakoff) but the survey responses indicated that the cases should have been coded a 190 (Ineligible: No Longer Receiving Care at Sampled Facility). If the case is finalized by telephone and the response to Q1 or Q2 is DK/RE, a code 190 should be assigned to the case.
2. Vendors must update their QAP and/or Vendor Application to reflect any major changes with personnel or to survey implementation protocols and procedures. In addition to the annual QAP update due May 31^st each calendar year, vendors must update their QAP (and in some instances their Vendor Application) to reflect these changes as they occur throughout the year. If you are unsure whether a particular change warrants updating your QAP, please reach out to the Coordination Team.
3. Vendors must ensure that they are implementing adequate QC on data prior to submission of their XML file to the Data Center. Vendors should consistently review their QC procedures throughout the data collection period to make sure they are still functioning as intended. Vendors that conduct a more manual QC of data may consider automating some of their processes to avoid human error. Vendors that conduct QC using mostly automated processes should also consider conducting a small manual QC of data on top of the automated QC, to ensure the formulas and/or programs are written correctly. For more information on Quality Control, please refer to the ICH CAHPS Administration and Specifications Manual.
4. Vendors must continually review and update, to the extent possible, the precision of their scanning software. Vendors who utilize optical scanning software to capture survey responses should review the requirements in the ICH CAHPS Survey Administration and Specifications Manual (section 5.5.2 or 7.4.2) to ensure their software is capturing quality data. There were instances where scanning software did not pick up survey responses or inaccurately coded stray marks or smudges as a response.

5. Telephone interviewers should be reading the telephone script verbatim. The site visit teams observed several instances of telephone interviewing staff not reading the script verbatim. Although the majority of deviations were minor (i.e., did not alter the meaning of the question), it is important that each sample patient be administered the exact same survey to ensure the highest quality data.