The HCAHPS Survey – Frequently Asked Questions Page 1
The HCAHPS Survey – Frequently Asked Questions
What is the purpose of the HCAHPS Survey? The HCAHPS (Hospital Consumer Assessment of Healthcare Providers and Systems) Survey, also known as the CAHPS® Hospital Survey or Hospital CAHPS®, is a standardized survey instrument and data collection methodology that has been in use since 2006 to measure patients’ perspectives of hospital care. While many hospitals collect information on patient satisfaction, HCAHPS (pronounced “H-caps”) created a national standard for collecting and public reporting information that enables valid comparisons to be made across all hospitals to support consumer choice. The HCAHPS sampling protocol is designed to capture uniform information on hospital care from the patient’s perspective.
Three broad goals shape the HCAHPS Survey. First, the survey is designed to produce comparable data on patients’ perspectives of care that allows objective and meaningful comparisons among hospitals on topics that are important to consumers. Second, public reporting of the survey results is designed to create incentives for hospitals to improve quality of care. Third, public reporting serves to enhance public accountability in health care by increasing transparency. With these goals in mind, the HCAHPS project has taken substantial steps to assure that the survey is credible, useful, and practical. This methodology and the information it generates are available to the public. More information about the HCAHPS Survey can be found at http://www.hcahpsonline.org/home.aspx.
Note: CAHPS® (Consumer Assessment of Healthcare Providers and Systems) is a registered trademark of the Agency for Healthcare Research and Quality, a U.S. Government agency.
What items are on the HCAHPS Survey? The HCAHPS Survey is composed of 27 items: 18 substantive items that encompass critical aspects of the hospital experience (communication with doctors, communication with nurses, responsiveness of hospital staff, cleanliness of the hospital environment, quietness of the hospital environment, pain management, communication about medicines, discharge information, overall rating of hospital, and recommendation of hospital); four items to skip patients to appropriate questions; three items to adjust for the mix of patients across hospitals; and two items to support congressionally-mandated reports. The HCAHPS Survey is available in English, Spanish, Chinese, Russian and Vietnamese in the mail format, and in English and Spanish in the telephone and Interactive Voice Response formats. On average, it takes respondents about seven minutes to complete the HCAHPS survey items. The core set of HCAHPS questions can be combined with customized, hospital-specific items to complement the data hospitals collect to support internal customer service and quality-related activities.
The actual wording of the HCAHPS questions and response categories, as well as the scripts for conducting the survey in the Telephone and Active Interactive Voice Response (IVR) modes, can be found under “Survey Instruments” on the HCAHPS On-line website, http://www.hcahpsonline.org/home.aspx. Complete information about how to implement the HCAHPS survey can be found in the HCAHPS Quality Assurance Guidelines, also available on this Web site.
How was the HCAHPS Survey developed? The Centers for Medicare & Medicaid Services (CMS) partnered with the Agency for Healthcare Research and Quality (AHRQ), another agency in the federal Department of Health and Human Services, to develop HCAHPS. AHRQ carried out a rigorous, scientific process to develop and test the HCAHPS instrument. This process entailed multiple steps, including a public call for measures; literature review; cognitive interviews; consumer testing and focus groups; stakeholder input; a large-scale pilot test and a number of small-scale field tests. In addition, CMS responded to hundreds of public comments generated by several Federal Register notices.
In May 2005, the National Quality Forum (NQF)—which represents the consensus of many healthcare providers, consumer groups, professional associations, purchasers, Federal agencies, and research and quality organizations—endorsed the HCAHPS. In December 2005, the federal Office of Management and Budget gave its final approval for the national implementation of HCAHPS. HCAHPS was also endorsed by the Hospital Quality Alliance. CMS commissioned an independent research firm, Abt Associates Inc., to conduct an analysis of the benefits and costs of HCAHPS. The Abt report, which includes detailed cost estimates for hospitals, can be found at: http://www.cms.gov/HospitalQualityInits/downloads/HCAHPSCostsBenefits200512.pdf.
The HCAHPS Survey – Frequently Asked Questions Page 2
When did hospitals begin to implement the HCAHPS Survey? Voluntary collection of HCAHPS data for public reporting began in 2006, and public reporting of HCAHPS scores began in 2008. Since July 2007, hospitals subject to IPPS payment provisions (“subsection (d) hospitals”) must collect, submit and publicly report HCAHPS data in order to receive their full IPPS annual payment update (APU). IPPS hospitals that fail to report the required quality measures, which include the HCAHPS survey, may receive an APU that is reduced by 2.0 percentage points. Non-IPPS hospitals, such as Critical Access Hospitals, can voluntarily participate in HCAHPS. HCAHPS Survey results also form the basis for the Patient Experience of Care domain in the Hospital Value-Based Purchasing program.
Which modes of survey administration can be used for HCAHPS? Because hospitals and survey vendors survey patients a number of ways, HCAHPS is available in four different modes: Mail Only, Telephone Only, Mail with Telephone follow-up (also known as Mixed mode), and Active Interactive Voice Response (IVR). Detailed information on the proper use of each mode of survey administration can be found in the HCAHPS Quality Assurance Guidelines manual, which is located at “Quality Assurance” at www.hcahpsonline.org.
CMS recognizes that patients’ responses to the survey may be affected by the mode of survey administration. For instance, respondents typically give somewhat more positive responses when surveyed by telephone, as compared to mail. Thus, choice of mode of survey administration could potentially affect comparisons of hospitals. CMS conducted a large-scale experiment to test for mode effects, and based on this research an adjustment has been built into the calculation of HCAHPS scores to remove the effect of survey mode on how patients respond to HCAHPS survey items.
The Mode Experiment was based on a nationwide random sample of short-term acute care hospitals. Participating hospitals contributed patient discharges from a four-month period in 2006. Within each hospital, equal numbers of patients were randomly assigned to each of the four modes of survey administration. In total, 27,229 discharges from 45 hospitals were surveyed.
In general, patients randomized to the Telephone Only and active IVR provided more positive evaluations than those randomized to the Mail Only and Mixed modes. Mode effects varied little by hospital. More information, as well as an overview of the results of the mode experiment, can be found under “Mode Adjustment” at http://www.hcahpsonline.org/home.aspx.
The HCAHPS Survey – Frequently Asked Questions Page 3
What must hospitals do in order to participate in HCAHPS? CMS has developed detailed Rules of Participation and Minimum Survey Requirements for hospitals that either self-administer the survey or administer the survey for multiple hospital sites, and for survey vendors that conduct HCAHPS for client hospitals. The HCAHPS Rules of Participation include the following activities:
■ Attend HCAHPS Introduction and Update Training
■ Follow the Quality Assurance Guidelines and Policy Updates
■ Attest to the accuracy of the organization’s data collection process
■ Develop a HCAHPS Quality Assurance Plan
■ Become a QualityNet Exchange Registered User for data submission
■ Participate in oversight activities conducted by the HCAHPS Project Team.
Hospitals and survey vendors administering the survey must also meet HCAHPS Minimum Survey Requirements with respect to survey experience, survey capacity, and quality control procedures. Details about these activities and requirements can be found in the Quality Assurance Guidelines under “Quality Assurance” at www.hcahpsonline.org.
Note: If a hospital, or its survey vendor, is found to be non-compliant with these rules or requirements, the hospital’s HCAHPS data may not be publicly reported and the hospital may be at risk for an annual payment update (APU) reduction.
Which patients are eligible to participate in HCAHPS? The HCAHPS survey is broadly intended for patients of all payer types that meet the following criteria:
■ 18 years or older at the time of admission
■ At least one overnight stay in the hospital as an inpatient
■ Non-psychiatric MS-DRG/principal diagnosis at discharge
■ Alive at the time of discharge
Patients who meet these criteria (except those that fall into an exclusion category, described below) should be included in the sample frame from which the survey sample is drawn.
A patient’s principal diagnosis at discharge is used to determine whether he or she falls into one of the three service line categories (medical, surgical or maternity care) for HCAHPS eligibility. The Medicare Severity-Diagnosis Related Group (MS-DRG) is the preferred method for determining whether the service line is Medical, Surgical or Maternity Care.
Pediatric patients (under 18 years old at admission) and psychiatric patients are ineligible because the current HCAHPS instrument is not designed to address the unique situation of pediatric patients and their families, or the behavioral health issues pertinent to psychiatric patients. Patients whose MS- DRG/principal diagnosis is Medical, Surgical or Maternity Care but who also have psychiatric comorbidities are eligible for the survey. Patients who did not have an overnight stay are ineligible because their experiences and interactions with the staff during the hospital visit may be limited.
There are a few categories of otherwise eligible patients who, because of logistical difficulties in collecting data, are excluded from the sample frame before the random sample is selected. These are:
■ Patients discharged to hospice care
■ Patients discharged to nursing homes and skilled nursing facilities
■ Court/Law enforcement patients (i.e., prisoners)
■ Patients with a foreign home address (excluding U.S. territories—Virgin Islands, Puerto Rico,
and Northern Mariana Islands)
· “No-Publicity” patients (see below) · Patients who are excluded because of rules or regulations of the state in which the hospital is
Complete information about patient eligibility and exclusions for the HCAHPS survey can be found in the Quality Assurance Guidelines under “Quality Assurance” at www.hcahpsonline.org.
The HCAHPS Survey – Frequently Asked Questions Page 4
Note: A “No publicity patient” is a patient who requests at admission that the hospital: 1) not reveal that he or she is a patient; and/or 2) not survey him or her.
Note: Hospitals must document their use of all patient exclusions.
How are patients sampled for the HCAHPS survey? The basic sampling procedure for HCAHPS is the drawing of a random sample of eligible discharges on a monthly basis. Smaller hospitals should survey all HCAHPS-eligible discharges. Data are collected from patients throughout each month of the 12-month reporting period. Data are then aggregated on a quarterly basis to create a rolling 4-quarter data file for each hospital. The most recent four quarters of data are used in public reporting. To ensure comparability, hospitals may not switch type of sampling, mode of survey administration, or survey vendor within a calendar quarter. More information about the HCAHPS sampling protocol can be found in the Quality Assurance Guidelines under “Quality Assurance” at www.hcahpsonline.org.
How is the sample drawn for the HCAHPS Survey? The basic sampling procedure for HCAHPS entails drawing a random sample of all eligible discharges from a hospital on a monthly basis. Sampling may be conducted either continuously throughout the month, or at the end of the month, as long as a random sample is generated from the entire month.
The target for the statistical precision of the publicly reported hospital scores is based on a reliability criterion. In brief, higher reliability means a higher ratio of “signal to noise” in the data. The reliability target for the HCAHPS global items and most composites is 0.8 or higher. Based on this target, hospitals must obtain at least 300 completed HCAHPS surveys over the 12-month reporting period.
The HCAHPS sample must be drawn according to this uninterrupted random sampling protocol. Hospitals/Survey vendors must sample from every month throughout the entire reporting period and not stop sampling or curtail ongoing interview activities once a certain number of completed surveys has been attained. All completed surveys should be submitted to the HCAHPS data warehouse. More information about the HCAHPS sampling protocol can be found in the Quality Assurance Guidelines under “Quality Assurance” at www.hcahpsonline.org.
Note: Smaller hospitals that are unable to reach the target of 300 completes in a 12-month reporting period must survey ALL eligible discharges and attempt to obtain as many completes as possible.
When are patients surveyed? Sampled patients are surveyed between 48 hours and six weeks after discharge, regardless of the mode of survey administration. Interviewing or distributing surveys to patients while they are still in the hospital is not permitted.
Data collection for sampled patients must end no later than six weeks following the date the first survey is mailed (Mail Only and Mixed Modes) or the first telephone attempt (Telephone Only and IVR Modes) is made. More information about the HCAHPS sampling protocol can be found in the Quality Assurance Guidelines under “Quality Assurance” at www.hcahpsonline.org.
How is the HCAHPS Survey data analyzed? Data submitted to the HCAHPS data warehouse is cleaned and analyzed by CMS, which then calculates hospitals’ HCAHPS scores and publicly reports them on the Hospital Compare website.
Which results from the HCAHPS Survey are publicly reported? Hospital-level HCAHPS results are publicly reported on the Hospital Compare website at http://www.hospitalcompare.hhs.gov. Results are reported for four quarters on a rolling basis, which
The HCAHPS Survey – Frequently Asked Questions Page 5
means that the oldest quarter of survey data is rolled off as the newest quarter is rolled on. Ten HCAHPS measures are publicly reported on Hospital Compare:
■ Nurse Communication (Question 1, Q2, Q3)
■ Doctor Communication (Q5, Q6, Q7)
■ Responsiveness of Hospital Staff (Q4, Q11)
■ Pain Management (Q13, Q14)
■ Communication About Medicines (Q16, Q17)
■ Discharge Information (Q19, Q20)
■ Cleanliness of Hospital Environment (Q8)
■ Quietness of Hospital Environment (Q9)
■ Overall Rating of Hospital (Q21)
■ Willingness to Recommend Hospital (Q22)
All ten HCAHPS measures are publicly reported for each participating hospital, as well as the national and state averages for each measure. The survey response rate and the number of completed surveys (in broad categories) are also publicly reported on Hospital Compare. CMS publicly reports HCAHPS results for hospitals that obtain fewer than 100 completed surveys. However, a footnote is added when public reporting these results to denote the lower level of precision. Additional information about hospital performance on HCAHPS is available under “Summary Analyses” on the HCAHPS On-Line Web site, http://www.hcahpsonline.org/home.aspx.
The HCAHPS Survey – Frequently Asked Questions Page 6
How are HCAHPS results adjusted prior to public reporting? To ensure that differences in HCAHPS results reflect differences in hospital quality only, HCAHPS survey results are adjusted for patient-mix and mode of data collection. Only the adjusted results are publicly reported and considered the official results. Several questions on the survey, as well as items drawn from hospital administrative data, are used for the patient-mix adjustment. Neither patient race nor ethnicity is used to adjust HCAHPS results; these items are included on the survey to support congressionally-mandated reports. The adjustment model also addresses the effects of non-response bias.
More information about the mode experiment, as well as patient-mix adjustment coefficients for publicly reported HCAHPS results, can be found under “Mode and Patient-Mix Adjustment” at http://www.hcahpsonline.org/home.aspx.
- What is the purpose of the HCAHPS Survey?
- What items are on the HCAHPS Survey?
- How was the HCAHPS Survey developed?
- When did hospitals begin to implement the HCAHPS Survey?
- Which modes of survey administration can be used for HCAHPS?
- What must hospitals do in order to participate in HCAHPS?
- Which patients are eligible to participate in HCAHPS?
- How are patients sampled for the HCAHPS survey?
- How is the sample drawn for the HCAHPS Survey?
- When are patients surveyed?
- How is the HCAHPS Survey data analyzed?
- Which results from the HCAHPS Survey are publicly reported?
- How are HCAHPS results adjusted prior to public reporting?