HHS Risk Adjustment Data Validation (HHS-RADV) White Paper

1 Centers for Medicare & Medicaid Services Department of Health and Human Services Center for Consumer Information and Insurance Oversight HHS Risk Adjustment data Validation (HHS-RADV) White Paper December 6, 2019 HHS-RADV Error Estimation Methodology Discussion Paper 2 TABLE OF CONTENTS Glossary of Terms, Acronyms, and Definitions .. 4 Executive Summary .. 8 1. HHS Risk Adjustment data Validation (HHS-RADV) Overview .. 11 PURPOSE AND STRUCTURE OF THIS White Paper .. 11 STATUTORY AND REGULATORY BACKGROUND OF HHS-RADV .. 11 HHS-RADV Process Overview .. 14 Original HHS-RADV Error Estimation Methodology.

2 16 Current HHS-RADV Error Estimation Methodology .. 16 HHS-RADV EXPERIENCE .. 19 Overview of HHS-RADV Pilot Years Results (2015 and 2016 benefit year HHS-RADV) .. 19 Overview of First Non-Pilot Year of HHS-RADV Results (2017 Benefit Year HHS-RADV) .. 21 CONSIDERATION OF HHS-RADV CHANGES .. 22 2. HHS-RADV Initial Validation Audit (IVA) Sampling .. 24 BACKGROUND AND PURPOSE OF HHS-RADV IVA SAMPLING .. 24 FUTURE OF HHS-RADV IVA SAMPLING .. 24 CURRENT HHS-RADV IVA SAMPLING METHODOLOGY .. 25 Proxy Issuer Populations .. 25 Stratification .. 25 Target Precision and Confidence Interval.

3 26 Sample Size Calculation .. 27 Neyman Allocation .. 28 Precision of Current Sample Sizes .. 29 Accuracy/Representativeness of Current Sample Sizes .. 30 HHS-RADV IVA SAMPLE SIZE REFINEMENT .. 32 Goals for HHS-RADV IVA Sample Size Refinement .. 32 Options for Sample Size Refinement .. 33 HHS S PERSPECTIVE .. 42 3. Modifications to Outlier Determination .. 44 OVERVIEW OF FAILURE RATE OUTLIER DETERMINATION .. 44 The Current Methodology .. 44 ADDRESSING THE INFLUENCE OF HCC COUNT ON OUTLIER DETERMINATION . 46 Basic Modifications to Current Methodology Considered.

4 49 Alternative Methodologies Based on Classical Statistics Considered .. 52 Alternative Methodologies Using Advanced Techniques Considered .. 61 ADDRESSING THE INFLUENCE OF HCC HIERARCHIES ON FAILURE RATE OUTLIER DETERMINATIONS .. 63 Ordinal-by-ordinal relationships as Applied to HHS-RADV .. 67 Assessing Outlier Status based on Risk Score Directly .. 68 HHS-RADV Error Estimation Methodology Discussion Paper 3 SUMMARY OF APPROACHES DETAILED IN THIS CHAPTER .. 72 4. Error Rate Calculation .. 73 KEY FACTORS USED IN THE ERROR RATE CALCULATION .. 73 DIFFERENCES IN TYPES OF OUTLIERS.

5 73 Outlier Observations .. 74 APPLICATION OF THRESHOLDS UNDER THE CURRENT METHODOLOGY .. 76 ALTERNATIVE OPTIONS TO CALCULATE THE ERROR RATE AND THEIR IMPACT 77 Original Error Estimation Methodology .. 77 Only Adjusting to Confidence Intervals .. 80 Only Make Adjustments for Positive Error Rate Outliers .. 81 Sliding Scale Adjustment Options .. 83 Evaluating the Sliding Scale Adjustment Options .. 87 NEGATIVE ERROR RATE ISSUERS WITH NEGATIVE FAILURE RATES .. 91 ALTERNATIVE OPTIONS .. 93 5. Application of HHS-RADV results .. 94 OVERVIEW OF THE APPLICATION OF HHS-RADV RESULTS.

6 94 TRANSITION YEAR OPTIONS .. 95 6. Conclusion .. 97 Appendix A: Overview of HHS-RADV Regulations .. 98 Appendix B: Comparing the 2017 Benefit Year HHS-RADV Results using the Current Error Rate Methodology, Original Error Rate Methodology, Confidence Intervals Methodology, and Only Positive Methodology in Chapter 99 Appendix C: Comparing the 2017 Benefit Year HHS-RADV Results using Sliding Scale Options in Chapter 4 .. 103 Appendix D: Diagrams and Tables of Current HCC Hierarchy Structure .. 107 Appendix E: Table of HCC Failure Rate Groupings for 2017 Benefit Year HHS-RADV.

7 117 HHS-RADV Error Estimation Methodology Discussion Paper 4 GLOSSARY OF TERMS, ACRONYMS, AND DEFINITIONS Term Acronym (if applicable) Definition Accuracy The property of being close to a target or true value. It measures how well the sample measurements match the true population value, but does not measure how close sample measurements are to each other (refer to definition of precision below). Bootstrap Resampling Technique A method of testing precision, wherein one large sample is taken from the parent distribution and then multiple samples with replacement from that initial sample are drawn.

8 It is used to determine standard errors and confidence intervals when the underlying distribution is unknown, when sample sizes may be too small, and/or when no formula may exist for a complex calculation. Center for Consumer Information and Insurance Oversight CCIIO The component within CMS charged with helping implement many reforms of the Patient Protection and Affordable Care Act (PPACA). CCIIO oversees the implementation of the provisions related to private health insurance. In particular, CCIIO works with states on the operation of the Health Insurance Exchanges. CCIIO works closely with state regulators, consumers, and other stakeholders to ensure the PPACA best serves the American people.

9 Centers for Medicare & Medicaid Services CMS A federal agency within the United States Department of Health and Human Services that administers the Medicare program and works in partnership with state governments to administer Medicaid, the State Children s Health Insurance Program, and the PPACA private market reforms. Confidence Level In terms of HHS-RADV error estimation, the theoretical probability that an issuer whose population-level failure rate for an HCC group is very similar to the national mean will not be found to be an outlier, given that all statistical assumptions about the underlying distribution are upheld.

10 Default data Validation Charge DDVC Charge imposed under 45 CFR (b)(10) if an issuer fails to engage an initial Validation auditor or submit initial Validation audit (IVA) results. The DDVC is calculated similarly to the Risk Adjustment Default Charge (RADC), but is an independently calculated and assessed penalty. Demographic and Enrollment D&E Describes an enrollee s demographics and enrollment status. External data Gathering Environment EDGE Issuer-distributed data collection services (also known as an EDGE server) that issuers in states where HHS operates a risk Adjustment program are required to establish and to compile enrollment, pharmaceutical claims and medical claims information on enrollees in risk Adjustment covered plans from issuers proprietary systems.