摘要：The Oregon Health Study was a groundbreaking experiment in which uninsured participants were randomized to either apply for Medicaid or stay with their current care. The study showed that Medicaid produced numerous important socioeconomic and health benefits but had no statistically significant impact on hypertension, hypercholesterolemia, or diabetes. Medicaid opponents interpreted the findings to mean that Medicaid is not a worthwhile investment. Medicaid proponents viewed the experiment as statistically underpowered and, irrespective of the laboratory values, suggestive that Medicaid is a good investment. We tested these competing claims and, using a sensitive joint test and statistical power analysis, confirmed that the Oregon Health Study did not improve laboratory values. However, we also found that Medicaid is a good value, with a cost of just $62 000 per quality-adjusted life-years gained. Mark Twain is believed to have penned the saying “There are lies, damn lies, and statistics.” 1 Even when working with gold standard data—a well-conducted randomized trial of a social policy—there is both an art and a science to the analysis of the data at one’s fingertips. Although social science experiments are the most rigorous means of evaluating a social policy, they tend to be logistically messy, requiring post hoc analytic adaptations. It is often the case that less than perfect policy experiments come under fire in the media, particularly when the findings do not align with a particular group’s beliefs. One recent example of this mix of science and media politics can be found in the case of the Oregon Health Study (OHS). In 2008, the state of Oregon provided Medicaid coverage to approximately 10 000 individuals randomly selected from 30 000 names drawn from the 90 000 who were eligible for Medicaid. 2 An expert interdisciplinary research team collected comprehensive survey responses, administrative information, and biomarker data on this subsample of winners and losers of the lottery. They found that Medicaid provided substantive financial protections, increased rates of preventive testing, reduced depression, and improved self-rated health. 2 They also found that those randomized to receive Medicaid did not achieve a statistically significant reduction in blood pressure, serum cholesterol levels, or blood glucose levels. However, relatively few of those who won the right to enroll in Medicaid actually did, and those who did turned out to be quite healthy to begin with. This greatly reduced the effective sample size of the “treated” group—those participants who were assigned to receive Medicaid. Further complicating matters, Oregon rapidly accelerated enrollment when more funds became available. This greatly shortened the time that the researchers had for data collection. 2 These complications led to a less than perfect experiment. As a result of these issues, some researchers pointed out that the study was statistically “underpowered,” 3 meaning that the number of participants should have been larger. These researchers based their claims on post hoc statistical analyses of each individual laboratory measure. For example, they showed that there were only 80 participants who might qualify as diabetic, and many more would have been needed to detect a meaningful reduction in diabetes. 4 Proponents of Medicaid point to these flaws to suggest that conclusions cannot be drawn about the effectiveness of Medicaid in improving these laboratory measures of health. They further note that, even if one sets the laboratory results aside, the other benefits are important, meaningful, and worth the investment. 5 Various opponents of Medicaid, conversely, tended to focus on the null results in the laboratory tests and declared that the study “proved” that Medicaid is a poor policy investment. 6–8 Supporting these claims, the OHS authors objectively note that the joint effect of all the tests combined was also not statistically significant. 2 This suggests that even if one considers the impacts of all the tests together, the OHS still fails to show statistically robust improvements in laboratory measures of health. The arguments of both Medicaid proponents and opponents are plausible. They rest on concerns about whether (1) the improvements in laboratory values would have been statistically significant had the OHS sample been larger and (2) the nonlaboratory benefits that were realized are meaningful enough to justify further expansion of Medicaid. We have addressed the first concern by conducting a highly sensitive joint test coupled with a post hoc power analysis of this test. If this test, which is more sensitive than is the one the authors originally used, is powerful enough to detect combined differences in laboratory values, it should validate or refute the critique that the sample size was too small. To address the second concern, we performed a cost-effectiveness analysis and a cost–benefit analysis to test the concern that Medicaid is not worth the investment.
