The following is a list of some of the data quality issues that the Coordination Team detected in data submitted to the ICH CAHPS Data Center, and some of those detected during site visits to survey vendors during the 2016 Spring Survey. We hope that each survey vendor will apply appropriate quality control measures to detect and avoid these errors and data problems in upcoming survey periods.

Data Quality Reminders

1. Check Data Collection and Data Processing Systems and Computer Programs. Survey vendors should check computer programs and systems used on the ICH CAHPS Survey prior to and periodically throughout each data collection period, to ensure that all programs are working correctly. Computer-assisted telephone interview (CATI) and data capture programs (data entry and scanning) should especially be checked on a periodic basis to ensure that programmed skip logic is working accurately. The following are just a few examples of situations in which the quality of the data might be affected because computer system/programs are not working properly.
• Example A. There was an error in the CATI program, which resulted in the answers to some questions being miscoded or overwritten.
• Example B. The data scanning program contained an error that resulted in the respondents’ answers to some questions not being scanned or being scanned incorrectly.
• Example C. There was an error in the CATI skip logic, which resulted in the interviewer not administering some questions to the sample patient that should have been asked, or administering questions that should not have been asked.
• Example D. The algorithm or program that the vendor used to export data from the mail or CATI file to the XML file contained an error, which resulted in survey responses for some questions not being included on the XML file. Note that in some instances described in the examples above, some responses that were miscoded, overwritten or not scanned correctly were to survey items that are used in patient-mix adjustment. These errors can potentially affect the ICH CAHPS scores that are publicly reported, so it is important that your computer programs and systems accurately capture sample patients’ responses, and those responses are included on the XML files submitted to the Data Center.


2. Check Assigned Disposition Codes. Survey vendors should make sure that the correct final disposition codes are assigned to all sample cases. Within 2 to 3 weeks after the data collection period begins and then after data collection ends, survey vendors should generate and review the interim and final disposition codes assigned to a sample of cases. Once final disposition codes are assigned, vendors should have a staff member (other than the one who assigned the disposition codes) conduct quality control on the disposition codes assigned. Note that the Coordination Team conducts quality control on each vendor’s data files after they are submitted. The following are some of the issues/errors that we found in data that were recently submitted.
1. Code 110 (survey completed by mail) was assigned to cases that were completed by telephone. Code 110 is only applicable to mail-only and mixed-mode cases that are completed by mail.
2. Code 120 (survey completed via telephone) was assigned to a completed mail survey. Code 120 cannot be assigned to a case that was completed by mail.
3. Code 130 (mail survey case that is ineligible) was assigned to phone-only cases. Code 130 can only be assigned to cases that were returned via mail.
4. Code 230 (bad/wrong addresses) was assigned to cases from a phone-only CCN. Code 230 is only applicable to cases in the mail-only mode.
5. Code 240 (bad/wrong phone numbers) was assigned to cases in the mail-only mode. Code 240 can only be assigned to cases in phone-only or mixed-mode CCNs.
6. Code 120 was incorrectly assigned to sample members who are ineligible for the survey. Code 120 should not be assigned if the answer to Q1 = 1 (at home) or 3 (am not currently receiving dialysis), or if the answer to Q2 = 1 (less than 3 months) or 5 (no longer receives dialysis at this facility).
7. Code 130 should be assigned to a mail case if the respondent answered one or more of questions in Questions 2 through 44 AND one or more of the following applies:
   • Q1—The answer to Q1 is “Receive dialysis care at home.”
   • Q1—The answer to Q1 is “I am not currently receiving dialysis.”
   • Q2—The answer to Q2 is “Less than 3 months.”
   • Q2—The answer to Q2 is “No longer receives dialysis at this facility.”
   • Q1 is blank.
   • Q2 is blank.
   • Q1and Q2 are both blank.
   • Q1 and Q2 both indicate ineligibility.
8. Code 110 was assigned to some mail survey cases that were completed by a proxy respondent. Remember that for all completed mail survey cases, vendors must first check the answer to Q62 (How did that person help you?) before assigning Code 110 to the case. If “Answered the questions for me” (response option 3) is marked in Q62, the case was completed by a proxy respondent. Proxy respondents are not allowed on the ICH CAHPS Survey. All cases completed by a proxy respondent must be assigned Code 199.
9. Codes 140 (ineligible, no longer receiving dialysis), 160 (ineligible, receive dialysis at home or at facility for less than then three months or on hospice or institutionalized/nursing home), and 190 (ineligible, no longer receiving dialysis at sampled facility) should be assigned based on a respondent’s answers to Q1 and Q2, or via a note or phone call to the vendor.
3. Check to make sure that all completed survey cases meet the completeness criteria before assigning Code 110 (for completed mail survey cases) or Code 120 (for completed telephone interview cases.)
4. Make sure that all cases assigned Code 110 and 120 have survey data on the XML file. There were some cases on some of the data files submitted that were assigned Code 110 or 120 (completed survey) that did not have any survey response data. During the data collection period, survey vendors should check their raw data files to make sure that survey response data were captured via scanning or data entry. In addition, they should check telephone survey data files to make sure that survey response data is captured correctly. When preparing the XML files, vendors must compare the survey response data for a sample of cases assigned Code 110 with the survey responses on the hardcopy mail survey questionnaire to make sure the survey response data was exported correctly from the raw data files. Vendors must ensure that all cases assigned final code 110, 120, 130, or 210 have data (at least partial data for a 130 or 210). If not, vendors must investigate to determine whether an incorrect disposition code was assigned or whether data exist and did not get processed correctly.
5. Make sure that all variables are included on the XML file, as required. All completed survey cases should have a survey mode, a language, and a final disposition code assigned to them.
6. Use of Not Applicable (X) and Missing Codes (M) on XML files. When preparing XML files, survey vendors should make sure that they are assigning the not applicable code (X) and the code for missing response (M) correctly. Assign X to a follow-up question that was correctly left blank based on the response to the preceding screening or gate question. For example, if the respondent’s answer to Q20 is response option 3 (Catheter) and the respondent correctly skipped Q21, assign Code X to Q21. Code M for missing should be assigned if the respondent should have answered a question but did not.
7. Review Data Frequencies/Distributions Regularly. To determine whether there is a potential data problem or to identify a problem with computer programs, vendors are strongly encouraged to generate response distributions (also referred to as frequencies) and compare the survey response record with the data on the hardcopy mail questionnaire (if the survey was completed by mail) or the CATI file (for interviews completed by phone). Once the frequencies are generated, look for anomalies or outliers and for unusual patterns of missing data.

Please contact the Coordination Team via e-mail at ichcahps@rti.org or by calling 1-866-245-8083 if you have any questions about data quality or appropriate quality control measures that should be implemented on ICH CAHPS Survey data.