摘要：We used the recent lowering of the alcohol purchasing age in New Zealand to examine the proposition that routinely collected data are often insufficient in evaluating important policy changes. We estimated prechange and postchange incidence rate ratios for actual and hypothetical population sizes and hospital admissions related to alcohol poisoning and assaults. Even with a hypothetical youth population 10 times larger than New Zealand's actual youth population, comparisons were underpowered because there were too few observations. Governments should use the enactment of health legislation as an opportunity to build the research evidence base by ensuring that evaluations are initiated in advance. Evaluations of public health policies and interventions often rely on routinely collected data, such as coroners' reports, cancer registries, and traffic crash records, from which incidence rates of key health outcomes are estimated. For example, in New Zealand, routinely collected data have been used to inform numerous studies on important health outcomes, serving New Zealand's citizens as well as the international scientific community. 1 Nevertheless, our experience in using routinely collected data to evaluate public health policies 2 – 6 is that these data are often ill suited to such evaluations, requiring compromises in the questions that can be addressed or exclusion of studies that could provide an empirical basis for policy decisions. In our efforts to seek funding for studies designed to evaluate policy changes, we have often been informed that data “are already being collected” by a certain government agency or in a particular survey, such that funding our study would represent duplication and unnecessary spending of New Zealand's meager health research dollars. Another common refrain is that the government should devote its resources to monitoring important health risk indicators rather than investing in “one-off” studies. There is a long-standing tradition of evaluating changes in social policies through the use of quasi-experimental study designs. 7 In such studies, key population groups are sampled and studied more comprehensively than routine data allow, and an appropriate control series is incorporated so that comparisons can be made. Well-controlled quasi-experiments typically allow stronger inferences about causality than do studies relying on routine monitoring. 7 In 1999, New Zealand lowered the country's minimum purchasing age for alcohol from 20 to 18 years. However, the government did not put into place, prior to the law change, baseline measures to evaluate its effects. For example, prechange surveys of alcohol consumption and related risk behaviors should have been conducted among affected youths and appropriate age comparison groups. Ideally, objective data (e.g., data gathered from roadside random breath testing) would also have been collected before the law change. The Ministry of Justice concluded that, as a result of limitations associated with routinely collected data, it was impossible to determine whether the law change had produced detrimental effects. 8 Notwithstanding the government's conclusions, 3 academic studies used routinely collected data and a variety of study designs to measure the effects of the law change on traffic crash injuries. 4 , 9 , 10 These studies showed increases in traffic crash injuries involving drivers aged 15 to 19 years that were attributable to the law change. For this study, we sought to critically examine the proposition that routinely collected data and routine monitoring cannot be relied on in evaluations of important policy changes or interventions and that predictive studies are therefore required. We examined this proposition with reference to the recent lowering of the minimum alcohol purchasing age in New Zealand. Although traffic crash injuries are clearly an important health outcome in relation to increases in the availability of alcohol, other health outcomes merit examination as well. We first sought to estimate the effects of the change in the alcohol purchasing age on incidence rates for 2 related outcomes: alcohol poisoning and assaults. Next, to determine whether difficulties associated with routine data were attributable to the small population of New Zealand or small effect sizes, we examined the influence of larger hypothetical populations and larger effect sizes on our ability to detect effects of public health significance (assuming that such effects occurred).
