Multi criteria assessment of county public health capability disparities.
Zuniga, Miguel A. ; Zuniga, Genny Carrillo ; Seol, Yoon Ho 等
BACKGROUND
Capacity evaluation and planning of public health resources is
increasingly more important for state and local health departments and
policy makers (Jia & Borawski, 2004). At present, there are few
models that reflect both the public health system and, those aspects
beyond its control that influence health status.
Decision analytic techniques are popular for use in a wide range of
decision problems (Chatburn & Primiano, 2001). This paper presents
an analytical approach to assessing relative population health
disparities between counties in the state of Mississippi. The level of
public health services within each county was identified through the
consolidation of public health related indicators into geographic
specific scores. This method allows counties to be ranked according to
their relative capability scores as a representation of direct and
indirect public health capability. This is important due to the fact
that counties do not have the same health profiles, measured as
available resources, disease epidemiology, public attitudes, etc. It is
assumed that the differential in disparities across counties can be
identified and its variation highlighted. This method may serve as a
complement to planning, allocating, and implementing primary health care
resources in an efficient and cost-effective manner.
The public health delivery system varies by locale and is an
important concern for health care planning and policy formulation. Given
this inherent variability, the question arises whether a county has the
requisite capability to meet the demand for public health services. The
concern is whether a local or regional public health system can provide
public health services in areas where demand is not under its control.
For example, not all of the indicators used in the model can be
addressed by a public health system. Unemployment rates, education, and
other social issues that have a significant influence on health are
examples of concerns that can not be affected by the public health
system.
In Mississippi, the State Health Department (MSDH) operates
approximately 200 clinics in 82 counties. These clinics are organized
into nine administrative health districts. The MSDH provides basic
public health services throughout the state, such as immunizations,
family planning, health promotion, health education, disease screening,
disease surveillance, license and certification of health facilities and
health professionals, and environmental health. The MSDH also provides
limited medical services in some counties where there is an identified
lack of other primary care providers. The array of services provided by
the MSDH is dependent on the availability of services from private and
other public sector providers.
The goal of health care planning is to improve the effectiveness of
established and future systems of health care and in doing so, the
identification of health problems and the population at risk is
essential (Stevens & Gillam, 1998). Before any plan can be
formulated, population characteristics, including age, sex, race,
socio-economic status, and health insurance coverage, must be identified
(Reid, Barnette & Mahan, 1998). Health resources are an essential
factor in health planning and thus, an inventory of human, physical, and
financial resources must be established, including a survey of the
geographic distribution of these resources over the target area. It is
also important to examine the breakdown of the specializations of
providers during the inventory process.
The capability differentials across counties must be evaluated and
understood in order to establish priorities for disparity minimization
and equalization intervention targets. Establishment of priorities is
intuitive and subjective, requiring insight, experience, and wisdom.
This task is best accomplished through a very structured and
well-defined procedure.
METHODS
Public health capability refers to the degree of occurrence of
factors associated with access to health promotion and disease
prevention services, health care processes and outcomes, and
socio-environmental factors that influence the relative well-being and
quality of life of populations.
The methodology for geographically determining and ranking relative
public health capability involves the development of a scoring system
that assigns a numerical score to each county (Raifaa, 1996). This score
represents the county's relative capability of basic public health
services and allows each county to be ranked relative to all others in
the state. This decision analytic methodology, known as multiple
attribute utility evaluation, has been used in previous health services
research (Fos & Zuniga, 1999; Fos, Miller, Amy, Zuniga, 2004).
The public health capability scoring system is composed of 20
weighted indicators divided into four major categories. Indicator
selection was based on the Center for Disease Control and
Prevention's Consensus Set of Health Status Indicators for the
General Assessment of Community Health Status(CDC, 2004), United Health
Group State Health Ranking-Selection of Components (United Health Group,
2007), and the Institute of Medicine Priority Areas for National Action:
Transforming Health Care Quality (Adams and Corrigan, 2003). Population
halth inputs and outputs are represented by indicators that depict the
healthcare delivery system and characterize: access to, and quality of
care; health outcomes; and, populations at- risk for disease and/or
death (Alciaiti & Glanz, 1996; Last & Logan, 1999).
Model indicators and their impact on county public health
capability are presented in Figure 1. There are four major categories of
model indicators: (1) access to health care, (2) quality of care, (3)
populations at higher risk of disease and/or death, (4) health outcomes.
All 20 explanatory indicators used in the scoring model are collected,
updated, and available from state and federal agencies.
Indicators that describe access to healthcare characteristics
include percent of pregnant women receiving prenatal care in the first
trimester, percent of county population covered by designated Health
Professional Shortage Area (HPSA), physician to population ratio,
percent CHIP enrollment, and percent of population with Medicare
enrollment in each county.
Quality is an important issue today in the provision of public
health care services. Five quality of care indicators are included in
the public health capability scoring model. These include: (1) children
immunization coverage; (2) adult immunization coverage; (3) percentage
of nursing home residents with infections; (3) mean nursing home nursing
staff hours per resident per day; and, (5) cesarean section rates per
1,000 live births.
The socio-economic condition in each county was evaluated using
indicators that describe populations at high health risks. Five
indicators were included to describe high health risk populations: (1)
childhood poverty, measured by the proportion of children under 15 years
of age living in families at or below the poverty level, (2) births to
adolescents (ages 10-17 years) as a percentage of total live births, (3)
prevalence of low birth weight, measured by the percentage of live born
infants weighing under 2,500 grams at birth, (4) unemployed population,
and (5) high school graduation rate.
The final category of the public health capability scoring model
contains mortality and morbidity indicators, essential for evaluating
public health care delivery systems. These include cardiovascular
disease deaths per 100,000 population, potential years of life lost to
age 75, female breast cancer incidence per 100,000 women,
race/ethnicity-specific infant mortality rate, and motor vehicle crash
deaths per 100,000 population.
SOURCES OF WEIGHTS AND MODEL DATA
The weighting of indicators was based on expert input, a well
accepted technique whose procedure has been reported in studies of
health care issues (Hankins & Fos, 1989; Fos & McLin, 1990;
Edwards & Newman, 1982). Since the assigned relative weights in this
study represent value judgments of policy makers and administrators,
this is a valid and reasonable approach for planning and policy
formulation. Determining value trade-offs of stakeholders is essential
in evaluation and policy formulation. The group of experts selected for
this procedure was composed of eighteen physicians, health care
planners, health care policy makers and public health personnel who are
very knowledgeable of public health in Mississippi.
Experts were interviewed in groups and individually by the authors.
Interviews ranged from 45 minutes to an hour. Experts completed a
structured questionnaire that rank ordered and rated indicators
according to their relative importance to (or impact on) the public
health capability of a county. The experts individually ranked and rated
indicators following the method discussed earlier.
The model does not seek expert consensus to develop the scoring
procedures (Morgan, DeKay, Fischbeck, et. al., 2001). The concern for
agreement can be evaluated by calculating the construct known as
Kendall's coefficient of concordance. Kendall's statistic
tests the hypothesis that there is no agreement across and among
experts. Table 1 presents the results of these tests.
After the expert group ranked and rated model indicators,
individual indicator-specific weights were calculated based on each
expert's input across model indicators. These individual weights
represent normalized weights of relative priority of each indicator in
relation to public health capability. After the individual normalized
weights were calculated, the mean values of all the experts' input
were computed in order to be used as the model indicator weights. Figure
1 includes indicator weights identified by the expert group.
Model indicator data were obtained from secondary sources and
compiled by the authors. Demographic information originated from 2000
United States Census data. Mortality and morbidity data were obtained
from existing, routinely collected databases of the MSDH. Additional
indicator data were obtained from the Medicaid and Medicare Office in
Mississippi. SPSS for Windows version 15 was used to compute public
health capability model variables and to test the hypothesis of
agreement among experts (SPSS, Chicago, IL, 2006).
DEVELOPMENT OF THE PUBLIC HEALTH CAPABILITY SCORING MODEL
The methodology presented below follows the techniques presented
previously by the authors.6 Following identification, each indicator was
assigned a specific public health capability score. This capability
score indicates the comparison of county-wide indicators to a standard,
the indicator value for the state. The indicator-specific score was
determined as follows:
IndicatorSpecific _Score = (County _Value/State _Value) x 10
The county value indicates the specific indicator value for the
county and the state value indicates the specific indicator value for
the state. Ratios of county and state values are then multiplied by 10
to avoid small numbers. This process was repeated for all 20 explanatory
indicators for each county.
The technique used in the scoring model to assign weights to
indicators is classified as numerical estimation (von Winterfeldt &
Edwards, 1986). After all indicators are rated, the ratings are summed,
and each individual indicator's rating is divided by the sum of the
ratings. This is a computational step which converts indicator ratings
into a mathematical representation of the influence of each indicator on
public health capability. This summation step assumes an additive
function, so the sum of all weights equals 1 . Even though the
indicators are not independent, as is the case in all complex problems,
an additive function was used because of the documented robustness of
additive models, as well as, the relative difficulty in building more
comprehensive multiplicative models (Dyer & Sarin, 1979; Keeney
& Raiffa, 1996).
After the indicator-specific scores and indicator weights have been
determined, the scoring system, which will calculate each county's
public health capability score, can be constructed. The scoring system
proposed is an additive model, and it is represented as follows:
County _Capability _Score = [n.summation over (i=1)] WiVi
Where, [w.sub.i] = each individual indicator weight, and [v.sub.i]
= each indicator-specific score.
Since we have assumed an additive model, the individual weighted
indicator scores can be added together to determine the county score. It
is important to note that weights of indicators which have a negative
effect on public health capability are assigned a negative sign, and
weights of indicators which have a beneficial effect on public health
capability are assigned a positive sign. This is analogous to positive
and negative coefficients in multiple regression statistical techniques.
RESULTS
Figure 1 represents the weighted value judgments of all experts. In
this study, model indicators are issues that may influence the health
status of a population and may determine the state of public health
capability within each county. The preference given to a model indicator
by the stakeholder shows the impact (positive or negative) of such
attributes on the health status of a community. To distinguish the
relative importance of the model indicators, the scores are calculated
and normalized to sum 1 (Fos & Zuniga, 1999; Edwards, 1977).
Following the calculation of individual normalized weights, the
computation of mean values of all experts' input was possible;
these mean values are the model variable weights. The weighting of the
four major categories reveals the order of importance given to these
categories as they impact public health capability. Health
access/resources have a weight of 29%; healthcare quality has a weight
of 27%, high risk populations have a weight of 26%; and health outcomes
have a weight of 18%. Weighting of individual variables shows the impact
of the top five variables on the model. These are: child immunization
coverage (15%); child poverty (10%); access to adequate prenatal care
(9%); cardiovascular death rate (8%); and percent of population covered
by (Note: Acronym needs to be written out--could not find a previos
reference to it--Health Public Service Agencies???) HPSAs (7%).
Toghether, these variables have an aggregate weight of 49% of the
overall model.
Table 2 presents the distribution of overall public health
capability scores, depicting the presence and development of public
health activities across the state. DeSoto County has the highest public
health capability score of 1.358 in relation to all others in the state.
This indicates that DeSoto County is the best suited to meet the demand
for basic public health services when compared with the rest of the
state. Pearl River County follows, with a public health capability score
of 1.164. Marshall County has the lowest public health capability score
in the state, -5.789. Others with low capability scores include:
Humphreys County (-3.687), Holmes County (-3.023), Issaquena County
(-2.979), and Tallahatchie County (-2.789).
Geographical Information Systems is a viable method for public
health disparity evaluation and planning (Gordon & Womersley, 1997).
Figure 2 presents the State of Mississippi according to the statistical
distribution of public health capability scores.
[FIGURE 2 OMITTED]
DISCUSSION AND CONCLUSIONS
As in any analytical model which attempts to quantify priorities
and individual trade-offs, the results should be viewed as
recommendations to health system planners and policy makers. It is
important to reiterate that several determinants of public health
capability are not under the influence of the public health services
delivery system, but are controlled by the private sector and other
public entities. However, there are many disparity determinants that the
public health delivery system has influence over, to affect some level
of improvement. Through the policy making arm of most public health
delivery systems, highlighting areas of improvement may have a profound
influence on improvements in education, employment status, and other
socioeconomics.
Given these limitations of the influence of the public health
delivery system, the discussion below will address the uses of the
capability assessment model. The strength of the model is that it allows
for an aggregate numerical value which represents several aspects of
public health systems. In this study, these aspects include 20
indicators, each measured in different units. Traditional
decision-making methodologies did not have the ability to combine the
effects of these indicators on public health capability. Multi-criteria
decision-making approaches allow for one unique numerical value which
reflects disparities, alternatives, choices, and preferences associated
with the decision situation.
Information derived from this model can be used to (a) identify
areas within a state which need targeting for some form of intervention,
(b) discriminate between specific areas which can be directly affected
by the public health care system and areas in which other agencies have
control, (c) identify specific needs of targeted areas, (d) guide budget
and resource allocation, and (e) establish expectations for disparity
reduction or equalization in a targeted area.
The public health capability assessment model can be used to make
absolute or relative comparisons. Absolute comparison uses the
county's individual rank as a benchmark. Improvement in model
variables can be evaluated with respect to whether a county's score
undergoes change. If the county's score moves higher, then this is
a reasonable improvement. If a county's score is unchanged, then
more thought and discussion should take place before targeting the areas
described by the model variables.
Relative comparisons are made by evaluating the public health
capability for each individual county compared to all others. Evaluation
of an individual county's rank in the State gives a relative view
of the county's public health capability. This benchmark can be
used to direct improvement efforts in targeted areas indicated by the
model that will change the county's rank and thus, equalize
disparities. For example, a county may plan to expend resources to
increase the percent of children who are immunized. This will change the
county's public health capability score, but may or may not improve
the county's rank. It is important to mention that the mean
performance value is a "moving average" and interpretation of
the public health capability score must be carefully considered,
examining each of the 20 indicators that comprise the score. For a
dynamic online demonstration of the model visit
http://www.msdh.state.ms.us/county/CountyPlanningModu le/CPM.html
With only minor exceptions, counties with the highest of public
health capability scores and those with the lowest of scores are not
randomly distributed throughout the state. In these two categories
appear geographic clusters, which may be used to guide policy and
planning. Since the Mississippi model employs several parameters which
are health status indices, it is possible that some in the upper
quartile of scores may be less capable of accessing public health
services than is apparent. Neighboring may positively influence each
other with respect to public health capability. One county may have more
than adequate resources for some services, while the neighboring county
may have more than adequate resources for other services. A
complementary effect may exist, where residents from neighboring
counties may benefit from each other's public health services.
Given the identified need for improvement, and the limited supply
of resources and funding, the model can be used to focus attention on
areas of least capability. One improvement strategy is to individually
review each model parameter for counties. The counties in the 25th
percentile scores which are also in the 25th percentile of
indicator-specific scores are the counties of concern. After specific
counties are targeted, the model has the ability to identify specific
areas for improvement in a county. The indicator-specific scores of
targeted counties can be reviewed relative to other un-targeted counties
and the state as a whole. Indicator specific scores for each parameter
in the model are ordered numerically across targeted counties. Those
parameters in the targeted counties which ranked low are selected for
improvement.
Finally, the model is ideal for establishing expectations and
benchmarks for reduction or equalization of disparities. Specific values
of selected parameters can be calculated in advance, serving as
disparity intervention goals. These "improvement" values are
important in deciding whether it is practical or feasible to plan and
implement specific interventions. "Improvement" values must be
attainable before planning, funding, and implementation of interventions
can occur.
REFERENCES
Adams K, Corrigan JM, eds. (2003). Priority Areas for National
Action: Transforming Health Care Quality. Washington, DC: The National
Academies Press.
Alciati M.H. and Glanz K. (1996). Using Data to Plan Public Health
Programs: Experience From State Cancer Prevention Programs. Public
Health Reports, 111(2):165-72.
Centers for Disease Control and Prevention (1991). Consensus Set Of
Health Status Indicators For The General Assessment Of Community Health
Status: United States. Morbidity and Mortality Weekly Report. 40(27):
E-1-E-3.
Chatburn R.L. and Primiano, Jr., F.P. (2001). Decision Analysis for
Large Capital Purchases: How to Buy a Ventilator. Respiratory Care,
46(10):1038-53.
Dyer J.S. and Sarin R.A.(1979). Measurable Multiattribute Value
Functions. Operations Research, 22:810-22.
Edwards W. (1977). How to Use Multiattribute Utility Measurement
for Social Decision Making. IEEE Transactions on Systems, Man and
Cybernetics, SMC-7, 326-346.
Edwards W. and Newman J.R. (1982). Multiattribute Evaluation, Sage
University Paper Series in Quantitative Applications in the Social
Sciences, Series No. 07-026, Beverly Hills, CA.
Fos P.J. and McLin C.L. (1990). The Risk of Falling In the Elderly:
A Subjective Approach. Medical Decision Making, 10(3): 195-200.
Fos P.J. and Zuniga M.A. (1999). Assessment of Primary Care Health
Care Access Status: An Analytic Technique for Decision Making. Health
Care Management Science, 2(4):229-38.
Fos PJ, Miller DL, Amy BW, Zuniga MA. (2004). Combining the
Benefits of Decision Science and Financial Analysis in Public Health
Management: A County-specific Budgeting and Planning Model. Journal of
Public Health Management Practice. 10(5): 406-412.
Gordon A. and Womersley J. (1997). The Use of Mapping in Public
Health and Planning Health Services. Journal of Public Health Medicine,
19(2):139-47.
Hankins R.W. and Fos P.J. (1989). Objectives for a System of Health
Care Delivery for HIV-Infected People. Socio-economic Planning Sciences,
23(4):181-193.
Jia H, Muennig P, and Borawski E. (2004). Comparison of Small-Area
Analysis Techniques for Estimating County-Level Outcomes. American
Journal of Preventative Medicine, 26(5): 453-460.
Keeney R.L. and Raiffa H.(1996). Decisions with Multiple
Objectives: Preferences and Value Tradeoffs. New York: John Wiley and
Sons.
Last J. and Logan H. (1999). Monitoring, Surveillance And Research
Needs. Public Health Planning Priorities and Policy Options. Canadian
Journal of Public Health. Revue Canadienne de Sante Publique, 90(6):SU
1-16.
Morgan K.M., DeKay M.L., Fischbeck P.S., Morgan M.G., Fischhoff B.,
and Florig H.K. (2001). A Deliberative Method For Ranking Risks (II):
Evaluation Of Validity And Agreement Among Risk Managers. Risk Analysis.
21(5):923-37.
Stevens A., and Gillam S. (1998). Health Needs Assessment: Needs
Assessment: From Theory to Practice. British Medical Journal.
316(7142):1448-1452.
Raiffa H. (1996). The Art and Science of Negotiation. Cambridge,
MA: Belknap Press.
Reid W.M., Barnette D.M., and Mahan C.S. (1998). Local Health
Departments: Planning For a Changed Role in the New Health Care
Environment. Journal of Public Health Management & Practice, 4(5):
1-12.
United Health Group (UHG) (2007). America's Health: State
Health Ranking 2007 Edition. St. Paul, MN: UHG.
von Winterfeldt D. and Edwards W. (1986). Decision Analysis and
Behavioral Research, Cambridge University Press, Cambridge.
MIGUEL A. ZUNIGA
South Texas Center
Texas A&M University System
Health Science Center
GENNY CARRILLO-ZUNIGA
School of Rural Public Health
Texas A&M University System
Health Science Center
YOON HO SEOL
Medical College of Georgia
PETER J. FOS
The University of Texas at Tyler
Table 1 Kendal Coefficient of Concordance
Expert Group k df [chi square] p-value W
All 18 23 115.10 <0.00000 0.312
Board of Health 7 23 38.18 0.02348 0.331
Members
District Health 6 23 62.97 <0.00001 0.456
Administrators
Central Office Staff 5 23 43.17 0.00662 0.375
Table 2
Public Health Scores Across All Counties
Rank Upper Score
Quartile
1 DeSoto County 1.35872
2 Pearl River County 1.16459
3 Hancock County 1.08799
4 Itawamba County 1.03010
5 Lafayette County 0.94927
6 Prentiss County 0.91636
7 Lamar County 0.86815
8 Harrison County 0.57041
9 Newton County 0.50921
10 Rankin County 0.50778
11 Forrest County 0.41914
12 Wayne County 0.40843
13 Amite County 0.33519
14 Madison County 0.33190
15 Smith County 0.29946
16 George County 0.29631
17 Yalobusha County 0.28516
18 Tippah County 0.28121
19 Oktibbeha County 0.23892
20 Lawrence County 0.07274
21 Pontotoc County 0.06704
Rank Mid-Upper Score
Quartile
22 Simpson County 0.01561
23 Hinds County -0.00882
24 Calhoun County -0.04571
25 Tate County -0.11152
26 Lee County -0.11702
27 Jones County -0.16877
28 Scott County -0.20517
29 Alcorn County -0.20575
30 Union County -0.21343
31 Neshoba County -0.37476
32 Warren County -0.48666
33 Stone County -0.49765
34 Attala County -0.49929
35 Lincoln County -0.50961
36 Perry County -0.52258
37 Jackson County -0.53995
38 Jasper County -0.54593
39 Lauderdale County -0.70650
40 Panola County -0.72961
41 Walthall County -0.78904
42 Sharkey County -0.82263
Rank Mid-Lower Score
Quartile
43 Monroe -0.82702
44 Tishomingo County -0.82924
45 Marion County -0.84707
46 Copiah County -0.87878
47 Sunflower County -1.00571
48 Clarke County -1.04767
49 Chickasaw County -1.06705
50 Bolivar County -1.11142
51 Lowndes County -1.12705
52 Montgomery County -1.12865
53 Leake County -1.30313
54 Covington County -1.35419
55 Adams County -1.35535
56 Pike County -1.37140
57 Benton County -1.50750
58 Claiborne County -1.53612
59 Quitman County -1.54115
60 Grenada County -1.59706
61 Tunica County -1.65324
62 Yazoo County -1.65376
Rank Lower Score
Quartile
63 Carroll County -1.67668
64 Winston County -1.71860
65 Greene County -1.74070
66 Wilkinson County -1.76331
67 Jefferson Davis County -1.87106
68 Franklin County -1.93648
69 Jefferson County -2.02680
70 Kemper County -2.28934
71 Coahoma County -2.42653
72 Choctaw County -2.43448
73 Leflore County -2.51697
74 Clay County -2.55039
75 Webster County -2.56901
76 Noxubee County -2.66457
77 Washington County -2.78341
78 Tallahatchie County -2.78910
79 Issaquena County -2.97996
80 Holmes County -3.02318
81 Humphreys County -3.68726
82 Marshall County -5.78989
The county with the highest level of capability is
ranked 1, and the county with the lowest level of
capability is ranked 82.
Figure 1
Value Tree
Health Access/ Resources IMPACT
0.2936 Percent Prenatal care|0.086 Positive
Percent Population covered by HPSA|0.017 Positive
Population to Physician Ratio|0.059 Positive
Percent CHIP Enrollment|0.059 Positive
Percent Medicare Population|0.020 Positive
Healthcare Quality
0.2677 Child Immunization Coverage|0.151 Positive
Adult Immunization Coverage|0.048 Positive
Percent Nursing Home Residents Negative
with Infections|0.030
Mean Nursing Home Staffing per Resident|0.022 Positive
Cesarean Section Rates|0.016 Negative
High Risk Populations
0.2584 Childhood Poverty|0.095 Negative
Births to Adolescents|0.051 Negative
Prevalence of Low Birth Weight|0.049 Negative
Unemployed Population|0.037 Negative
High School Graduation Rate|0.027 Positive
Health Outcomes
0.1804 Cardiovascular Deaths|0.080 Negative
Potential Years of Life Lost|0.033 Negative
Female Breast Cancer Mortality|0.029 Negative
Infant Mortality|0.026 Negative
Motor Vehicle Crash Death|0.012 Negative
