2018 ICH CAHPS Spring Survey Site Visit Issues & Data Quality Reminders
folder_openOther Announcementscalendar_todayPosted September 6, 2018

This announcement includes a list of the data quality issues that the Coordination Team (CT) detected in the data from the 2018 Spring Survey and issues we observed during site visits to survey vendors during the 2018 Spring Survey. As you prepare for the 2018 Fall Survey data collection period, please apply appropriate quality control measures so that you can detect and avoid these errors and data problems in all upcoming survey periods.

Please contact the ICH CAHPS Coordination Team via email at ichcahps@rti.org or call (866) 245-8083 if you have any questions about any of the information provided below.

Data Quality Reminders

2018 Spring data submitted by vendors looked great! The one general issue found involved the assignment of final disposition codes.

• Check Assigned Disposition Codes. Survey vendors should make sure that the correct final disposition codes are assigned to all sample cases. Within 2 to 3 weeks after the data collection period begins and then after data collection ends, survey vendors should generate and review the interim and final disposition codes assigned to a sample of cases. Once final disposition codes are assigned, vendors should have a staff member (other than the one who assigned the disposition codes) conduct quality control on the disposition codes assigned. Note that the CT conducts quality control on each vendor’s data files after they are submitted. The following are some of the issues/errors that we found in data that were submitted for the 2018 ICH CAHPS Spring Survey period:
  1. Code 120 was incorrectly assigned to sample members who are ineligible for the survey. Code 120 should not be assigned if the answer to Q1 = 1 (at home) or 3 (not currently receiving dialysis), or if the answer to Q2 = 1 (less than 3 months) or 5 (no longer receives dialysis at this facility).
  2. Code 130 should be assigned if the respondent answered one or more of questions in Questions 2 through 44 in a mail survey AND one or more of the following applies:
     • Q1—The answer to Q1 is “Receive dialysis care at home.”
     • Q1—The answer to Q1 is “I am not currently receiving dialysis.”
     • Q2—The answer to Q2 is “Less than 3 months.”
     • Q2—The answer to Q2 is “No longer receives dialysis at this facility.”
     • Q1 is blank.
     • Q2 is blank.
     • Q1and Q2 are both blank.
     • Q1 and Q2 both indicate ineligibility
     • Also assign Code 130 if both Q1 and Q2 are blank but then skipped to Q45.
  3. Codes 140 (ineligible, no longer receiving dialysis), 160 (ineligible, receive dialysis at home or at facility for less than then three months or on hospice or institutionalized/nursing home), and 190 (ineligible, no longer receiving dialysis at sampled facility) should be assigned based on a respondent’s answers to Q1 and Q2, or via a note or phone call to the vendor.
  4. Code 210 (breakoff) should be assigned if the sample patient responds to some questions but not enough to meet the completeness criteria.

Observations during Site Visits

The 2018 Spring site visits were successful! There are a few general reminders that we would like to share with all vendors.

• Make sure updated version of Pre-Notification Letter is sent to all sample patients.During site visits we observed that some vendors were still using a prior version of the letter instead of the one revised for the 2018 Spring Survey. Recall that this letter was specifically revised to read at a lower reading level, making it more accessible to our ICH CAHPS sample patients.
• Make sure all telephone interviewers are trained on the appropriate pace, annunciation, pronunciation and tone of voice to use during administration of the telephone survey.Some vendor’s telephone interviewers were skipping small words like “how,” and “a,” and sounded bored. Interviewers should also be trained on basic pronunciation of common English and Spanish sounds, as well as on how to listen to the respondent and correct name mispronunciations, if applicable.
• Administer the correct version of the mail survey, the telephone survey, and other survey materials. Survey vendors must make sure that they are using the correct version of the ICH CAHPS mail survey and telephone interview script. During site visits we observed that some vendors had not implemented the changes that were included in the most recent versions of the mail survey and telephone script posted on the ICH CAHPS website.
• Make sure all envelopes are marked with “address service requested, change service requested, etc.” Some vendor’s envelopes did not contain this required message; however, it was noted that if vendors are using USPS First Class, all letters will always be returned if undeliverable. Vendors who use USPS First Class mailing are permitted to not include any of the above required messages on their envelopes.
• Make attempts to contact a sample patient on different days of the week and at different times of the day. During the site visits we noted that some vendors are not making telephone call attempts during the evening hours while other vendors are not making attempts during the morning hours. In addition, some vendors were not contacting sample patients on weekend days or were ending all call attempts well before the data collection period ended. Survey vendors must make a maximum of 10 telephone contact attempts for each sample patient, unless the sample patient refuses or the survey vendor learns that the sample patient is ineligible to participate in the survey. The 10 contact attempts must be made on different days of the week and at different times of the day, including nights and mornings, and spread over the data collection period.
• Vendors must be in possession of a signed confidentiality agreement from all staff, including subcontractor staff. During the site visits we noted that some vendors did not have in their possession copies of all signed confidentiality agreements from subcontractor staff. If a vendor’s subcontractor is restricted from providing the vendor with copies of the signed agreements, the vendor should obtain from the subcontractor a list with the names of the individuals who have signed the agreement.
• Vendors are strongly encouraged to conduct frequency checks on all XML data prior to submission. During the site visits we found anomalies in data frequencies for some survey items and discovered programming errors were leading to these anomalies. Quick frequency checks on all survey items are an easy way to spot a potentially larger problem.