2019 ICH CAHPS Fall Survey Site Visit Issues & Data Quality Reminders
folder_openOther Announcementscalendar_todayPosted January 17, 2020

As we approach the deadline for submitting data from the 2019 ICH CAHPS Fall Survey and the beginning of the 2020 Spring Survey, it is a good time to remind vendors of the importance of implementing quality control measures on all aspects of the survey.

This announcement includes a list of some of the data quality issues that the Coordination Team (CT) detected in the data from the 2019 Spring Survey and issues we observed during site visits to survey vendors during the 2019 Fall Survey. As you conduct final data cleaning and processing of the 2019 Fall Survey data and prepare the XML file that will be submitted, please apply appropriate quality control measures so that you can detect and avoid these errors and data problems in all upcoming survey periods.

As a reminder, the XML files containing data from the 2019 Fall Survey must be submitted to the ICH CAHPS Data Center by 5:00 PM EST on Wednesday, January 29, 2020. Please contact the ICH CAHPS Coordination Team via email at ichcahps@rti.org or call (866) 245-8083 if you have any questions about any of the information provided below.

Data Quality Reminders

1. Check Assigned Disposition Codes. Survey vendors should make sure that the correct final disposition codes are assigned to all sample cases. Within 2 to 3 weeks after the data collection period begins and then after data collection ends, survey vendors should generate and review the interim and final disposition codes assigned to a sample of cases. Once final disposition codes are assigned, vendors should have a staff member (other than the one who assigned the disposition codes) conduct quality control on the disposition codes assigned. Note that the CT conducts quality control on each vendor’s data files after they are submitted. The following are some of the issues/errors that we found in data that were submitted for the 2019 ICH CAHPS Spring Survey period:
   1. Code 210 (breakoff) was incorrectly assigned to telephone cases in which the data showed that Q1 and/or Q2 = Missing (DK or Ref). If a phone interview, and Q1 and/or Q2 = Missing, then CATI should skip to Q45 and the case should be coded to 190 (Ineligible: No Longer Receiving Care at Sampled Facility).
   2. Multiple cases were coded a 190 (Ineligible: No Longer Receiving Care at Sampled Facility) based on the answer to Q2, but the responses to Q1 indicate the cases should be coded a 140 or 160. The Coordination Team assigns the first possible ineligible code, which would be either 140 or 160 indicated in Q1, instead of 190 indicated in Q2.

Other Observations during Site Visits

2. Vendors must have a procedure in place to deal with a HIPAA or PHI breach of security with ICH CAHPS data. To protect PII and PHI, vendors must develop and implement procedures for identifying and responding to data or a security breach with ICH CAHPS data, including when an unauthorized individual has gained access to confidential information and when an authorized individual has distributed confidential data in an unauthorized manner. The survey vendor’s plans must include a system to notify the ICH CAHPS CT in a timely manner of a security breach, a means to detect the level of risk represented by the breach in security, a means to take corrective action against the individual who created the breach, and a means of notifying any persons affected by the breach, including sample patients, if necessary.
3. Make sure interviewers are trained on correct telephone interviewing techniques. While conducting silent monitoring of live phone calls/reviewing previously recorded telephone interviews, the CT noted a few issues that we recommend all vendors retrain their interviewers on.
   1. First, if a respondent indicates that they are currently at their dialysis facility at the time of the call, the interviewer should stop the interview and reschedule to complete it at a time when the sample patient is not at the facility.
   2. Second, interviewers should be reading the survey introduction, questions, and response options verbatim and with good pace. There were multiple instances in which interviewers changed or dropped words completely or read response options too quickly, leaving the sample patient confused.
   3. Third, interviewers should follow correct neutral probing techniques when there is a need to obtain a more complete or more specific answer from a respondent. Please see Appendix K of the ICH CAHPS Survey Administration & Specifications Manual for more information on probing.
4. Make attempts to contact a sample patient on different days of the week and at different times of the day. During the site visits we noted that some vendors are not making telephone call attempts during the evening hours while other vendors are not making attempts during the morning hours. This is an issue we see every site visit period. In addition, some vendors were not contacting sample patients on weekend days or were ending all call attempts well before the data collection period ended. Survey vendors must make a maximum of 10 telephone contact attempts for each sample patient, unless the sample patient refuses, the survey vendor learns that the sample patient is ineligible to participate in the survey, or if the vendor obtains a new telephone number for the sample patient in which the 10 call attempts should start over. The 10 contact attempts must be made on different days of the week and at different times of the day, including nights and mornings, and spread over the data collection period. The CT understands that the vendor may use historical data on response rates to drive their calling algorithms (e.g. they purposefully place more ICH calls during the hours of 6-8pm EST due to a high instance of completes for their clients in the past); however, we recommended re-evaluating their algorithms prior to, during, and after each survey period to ensure the call spread still makes sense for their ICH clients.