Professional policy work in federal states: institutional autonomy and Canadian policy analysis.
Howlett, Michael ; Wellstead, Adam M.
Introduction: Gaps in knowledge of policy work in multi-level
states
Despite having been a matter of academic attention for over half a
century, the working world of policy analysts is largely unknown, as
Colebatch and Radin argued in their 2006 survey of comparative policy
analysis. Many studies rely on anecdotal case studies and interview
research (Radin 2000; Hoppe and Jeliazkova 2006; Colebatch et al. 2011;
Noordegraaf 2011) which, while informative and useful for theory
building, raise questions about the robustness of findings. (1)
Large-scale, comprehensive, empirical studies of policy work are
not only very rare but are also dated and cover few jurisdictions, most
commonly the US. Some analyses of these studies employ only partial
survey evidence (see, for example, Page and Jenkins' 2005 study of
the UK "policy bureaucracy") and in many cases observers
seeking large-n empirical data rely on one or two dated American
studies, such as Meltsner's (1976) work from three decades ago or
Durning and Osuna's 1994 study, to support their conclusions about
the tasks, duties and roles played by policy analysts in contemporary
policy processes. These gaps in knowledge about the characteristics and
activities of professional policy workers have led many observers to
decry the lack in many countries of such basic data as how many policy
analysts there are in government, what subjects they work on, and with
what effect (New Zealand, State Services Commission 1991, 1999; Uhr and
Mackay 1996; Bakvis 1997 and 2000; Weller and Stevens 1998) and to
suggest that the conduct of newer, large-n surveys of professional
policy workers in government is required to advance thinking on this
topic. (2)
This situation is especially acute in multi-level countries, since
what little work exists focuses almost exclusively on the national or
central level (for rare exceptions see Hollander and Prince 1993; Rieper
and Toulemonde 1997; Hird 2005b; Dobuzinskis, Howlett and Laycock 2007;
Liu et al. 2010), despite the fact that sub-national governments have
extensive policymaking authority in many important areas of social and
economic life, such as healthcare, social welfare programs,
transportation and urban policy (Howlett 1999). Even high-profile
federal states, such as Australia, Germany, the US and Canada, whose
central government decision-making processes have been studied
intensively have had little attention paid to their sub-national
jurisdictions (Wollmann 1989) until recently and empirical data have
been lacking on the policy analytical techniques and practices found in
these sub-national governments (Hird 2005b; Voyer 2007; Howlett 2009b,
2009c; Howlett and Newman 2010).
Understanding what sub-national policy workers do and with what
effect, however, is an essential pre-requisite to understanding how
policy processes and policy advice work in federal states and, more
generally, in many states where policy processes involve complex,
multi-level governance arrangements. This article uses data gleaned from
the first large-scale survey of federal, provincial and territorial
policy analysts in Canada to shed some light on these questions.
Policy work in multi-level systems
The limited autonomy hypothesis
A key question in the study of sub-national policy work is whether,
and to what extent, analytical activities differ from those identified
at the national or central level. Limited case study and interview-based
research examining state- or provincial-level policy making (see for
example Halligan 1995; Segsworth and Poel 1997; Rasmussen 1999; Maley
2000; McArthur 2007) is far from conclusive about how sub-national
analytical work differs from that conducted by central governments, but
it does provide a basic set of working hypotheses which can be tested
against larger-scale and more comprehensive empirical survey results.
McArthur's 2007 work on federal and provincial policy making
in Canada is especially useful in this regard. This work highlights the
impact upon policy work and workers of organizational differences
between sub-national and national levels of government, and, in
particular, the consequences for policy work flowing from the smaller
size of government and the tighter lines of political control found at
the sub-national level. Because of these structural characteristics,
McArthur argued, provincial government agencies and the analysts they
employ suffer from a lack of autonomy from demands placed upon them by
political masters and by prominent social actors such as trade unions,
business associations, think tanks and interest groups. As a result,
sub-national policy workers are much more constrained and short-term in
their activities and orientations than their national counterparts, and
are much more likely to follow political dictates and fashions in
government, and much less able to resist pressure-group politics. Liu et
al. (2010) found a similar pattern and effect at the local government
level.
This study will evaluate this limited autonomy hypothesis. By
comparing policy work and workers at different levels in a multi-level
state such as Canada, we would expect to see large similarities across
levels of governments, with professional analysts using similar
techniques and approaches, but with some major differences between their
analytical practices and policy work. This hypothesis will be tested
using empirical data gathered from a unique set of surveys of
professional policy analysts working in the federal, ten provincial and
three territorial governments of the Canadian federation. As a
high-profile federal state with significant policy-making
responsibilities at the sub-national level, Canada provides a strong
case from which to generate insights into the differences and
similarities of policy analytical work in multi-level governance systems
(Howlett and Lindquist 2004).
Data
In order to assess this institutional autonomy thesis, data from
two sets of surveys of Canadian policy workers conducted in 2007 and
2008 by the authors were combined so that the attitudes, practices and
situations of respondents in Canada's National Capital Region (NCR)
(located in the adjoining communities of Ottawa, Ontario, and Gatineau,
Quebec) could be compared with those of analysts in the country's
thirteen provincial and territorial jurisdictions. Data were divided
into five topic areas: demographic characteristics; job experience of
analysts; their education and training; the nature of their day-to-day
duties; and the techniques and data they employ in their analyses.
Overall survey results and demographic profiles for the federal level
are available in Wellstead et al. (2009) and for the provincial and
territorial level in Howlett (2009b, 2009c). Response rates by
provincial and territorial jurisdiction are set out in Appendix 1. (3)
The combined data collected from these surveys allowed profiles of
federal, provincial and territorial analysts to be compared and provided
the basis for the first large-n, comparative, empirical analysis of the
background and activities of sub-national and national-level government
policy analysts working in a multi-level governance system. The
comparison reveals interesting similarities and differences among policy
workers and policy work in such systems, and provides the empirical data
required to evaluate differences in the practices employed by analysts
working in a multi-level state.
Results
Federal-provincial-territorial commonalities
Table I illustrates many important similarities in the policy tasks
conducted at the federal, provincial and territorial levels of Canadian
government. Regardless of their location or level of operation, policy
workers are likely to be engaged in implementation work that requires
coordination with other levels of government and technical or
specialized knowledge, and to exhibit a similar division of short- and
long-term tasks. They also share many similar attitudes towards those in
power, and have similar views about such topics as their own and
non-governmental policy capacity.
Federal differences versus provincial-territorial similarities
There are also several statistically significant differences
between analysts working in national governments and those working at
the sub-national level. As Table2 shows, federal government analysts
display many unique characteristics as compared to their sub-national
counterparts. At the same time, there are many similarities between
provincial and territorial analysts in these same areas. Some results
are unsurprising, such as the likelihood of federal analysts to work on
national issues, but others confirm the McArthur expectation that
federal analysts tend to deal more often with more complex and
longer-term issues. National-level analysts are also more supportive of
head-office control and less supportive of the devolution of government
programs and services. More significantly for the McArthur hypothesis,
federal analysts have many fewer interactions with other governments
than do provincial and territorial analysts, and are less likely to be
involved with interest groups, NGOs or other provincial or municipal
governments than their provincial and territorial counterparts. Both
provincial and territorial analysts, in contrast, share similar
attitudes and practices in these and other areas of policy work.
Federal-provincial similarities versus provincial-territorial
differences
Prima facie, the findings in Tables 1 and 2 support the
expectations of the limited autonomy thesis. While there may be basic
commonalities in the analytical practices and behavior of analysts in
all three levels of government, they are different and these differences
reflect the institutional characteristics of government at each level,
especially the smaller size and operating characteristics of provincial
and territorial governments.
Further support for this hypothesis can be found by examining
similarities and differences between federal-provincial and territorial
analysts and between federal-territorial and provincial analysts. The
limited autonomy hypothesis suggests that there would be fewer
similarities between these sets of analysts than found between federal
and provincial-territorial analysts. As Tables 3 and 4 reveal, this is
indeed the case. Table 3 shows that there are only a few areas of policy
work in which the activities of federal and provincial analysts are more
similar than provincial-territorial ones. Territorial analysts are less
likely to be involved in regional issues despite their
"regional" status in the Canadian north, and are more likely
to be involved with head-office management and other head-office staff,
while being less involved with think tanks. These latter findings can be
linked to the small size and operating characteristics of territorial
administrations, again supporting the McArthur hypothesis.
Table 4 illustrates the areas in which provincial analysts differ
from their federal and territorial colleagues. Like Table 3, Table 4
underlines the relatively few areas in which federal and territorial
analysts are more similar than are provincial and territorial ones.
Significantly, with respect to the limited autonomy hypothesis,
provincial analysts are more likely to negotiate and consult with
stakeholders, especially industry and labour organizations, and to
perceive their work to increasingly involve networks of people both
within and outside of government. Territorial analysts do not share
these characteristics, given their isolated, sparsely populated
landbases in the far Canadian north.
Conclusion
Policy advice systems are complex entities (Boston 1994; Halligan
1995; Uhr and Mackay 1996; Maley 2000). Given their reliance on existing
institutional configurations, policy advice systems vary by
jurisdiction, especially by nation-state (Brinkerhoff and Crosby 2002;
Brinkerhoff and Morgan 2010) and, somewhat less so, by policy sector
(Hawke 1993) and department (Rochet 2004; Voyer 2007). An important and
oft-overlooked aspect of these institutional variations, however, is the
level of government involved in policy deliberations and policy making.
As Hooghe and Marks (2001, 2003) and others have noted, multi-level
governance systems place different demands on different levels of
government since specific responsibilities for policy making in major
areas like healthcare, education and the environment are distributed
unequally across levels and jurisdictions (Piattoni 2009). Despite the
impact that multi-level governance potentially has on the capacity of
analysts to engage in high-level, long-term, sophisticated policy
activities, the role and influence of such institutional characteristics
on policy analysts and their work have not been systematically evaluated
(Riddell 2007; Howlett 2009a, 2009b and 2009c). Knowing whether or not
it is possible (and accurate) to infer from national studies when
describing policy arrangements is essential to the better understanding
and improvement of the work carried out by professional analysts in
government.
Using a unique, multi-level data set derived from fifteen separate
but similar surveys of federal, provincial and territorial policy
analysts in Canada in 2007 and 2008, this article tested an hypothesis
derived from existing case study and anecdotal research, which posited
that the activities and work of sub-national level analysts would differ
in important ways from those undertaken at the national level due to
differences in the structural configuration of governments and the
interactions of analysts with interest group systems. The study examined
the behaviour and attitudes of federal, provincial and territorial
analysts in order to discern the similarities and differences between
them. It found many similarities in policy work areas related to overall
governance trends, such as moves towards increased consultation and
participation overtaking more traditional technical policy evaluations,
but it also found continuing significant differences between analysts
working at each level. Sub-national level analysts have more
interactions with societal policy actors and experience more direct
control by senior management than analysts employed by central
governments. These findings conform to the expectations of the limited
autonomy hypothesis, which argues that the level of autonomy from
political control and social actors affects policy work and workers.
This in turn has important implications for the ability of analysts to
undertake long-term independent research and analysis. Where lines of
control and social contacts are higher, as in sub-national governments,
analysis is expected to be more politically driven and short term in
nature.
While it remains up to future research in other countries with
multilevel governance systems to reveal how robust these conclusions
are, in the Canadian case it was found that policy work among federal,
provincial and territorial analysts, while broadly similar, varied
significantly in the direction suggested by the limited autonomy
hypothesis. Analysts and policy workers employed in smaller governments
were found to have significantly more interactions with central agencies
and pressure groups and to differ substantially from federal workers in
terms of the issues they dealt with and the nature of the input they
received in their work.
The findings suggest, moreover, that scholars cannot simply infer
from existing national studies about trends and activities in policy
analysis at sub-national levels of government, but must recognize each
level of government as a significant policy actor in its own right. In
other words, the policy styles and practices found at different levels
of government in multi-level states, while broadly similar, contain
different features which affect not only the nature of the processes
followed but also the content of decisions reached and policies adopted.
This finding helps to illuminate aspects of the behaviour of policy
analysts and decision makers in multi-level systems, and contributes to
our understanding of policy work, its institutional components, and its
ability or capacity to address important questions involving long-term,
data-oriented issues in an objective, technical way (Howlett 2009a).
While the specific sectoral nature of these impacts will depend upon the
division of jurisdictions between levels found in different countries,
in general, sub-national policy work can be expected to be more
participatory, partisan, pluralist and short term than at the central
level where policy makers enjoy higher levels of autonomy. As a result,
that work will display a corresponding difficulty in marshalling
expertise in the pursuit of longer-term solutions to ongoing policy
problems.
Appendix 1. Description of data set and response rates
Federal government analysts were identified through keyword
searches in the publicly accessible online Government Electronic
Directory Services (GEDS) system. In November 2007, an online survey
using the Zoomerang[R] software was conducted from a random sample of
725 policy-based, NCR federal government employees. The survey garnered
395 useable responses for an overall response rate of 56.4 per cent. The
provincial and territorial government surveys were carried out in
November and December 2008 also using the Zoomerang[R] software and an
appropriately amended version of the 2007 federal survey questionnaire.
The questionnaire was sent to over 4,000 provincial and territorial
civil servants covering each of Canada's thirteen provincial and
territorial jurisdictions. Mailing lists for the ten provinces and three
territories surveyed were compiled wherever possible from publicly
available sources such as online government telephone directories using
keyword searches for terms such as "policy analyst" in job
titles or descriptions. In some cases, additional names were added to
lists from hard-copy sources such as government organization manuals. In
other cases, lists or additional names were provided by provincial
public service commissions who also checked initial lists for
completeness and accuracy. (4) Over 1,600 provincial and territorial
completed survey were gathered from 3,856 valid e-mail addresses for a
total response rate of 43.3 per cent (see Table below). (5)
Provincial Survey Sample Sizes and Completion Rates
Initial Refusals
mail and Valid
list rejected partial Response
Province size e-mails completions Complete rate (%)
BC 513 51 30 194 48.5
Alberta 368 23 8 112 34.8
Saskatchewan 246 27 13 80 42.4
Manitoba 161 20 6 98 73.7
Ontario 1613 162 52 557 41.9
Quebec * 250 0 44 86 52.0
New Brunswick 162 15 4 62 44.9
Nova Scotia 181 20 15 83 44.1
PEI 27 6 1 4 23.8
Newfoundland 139 24 16 55 61.7
Yukon 75 8 6 58 95.5
NWT 80 2 2 41 55.1
Nunavut 41 8 2 13 45.4
TOTAL 3856 366 155 1357 43.3
(excluding
Quebec)
* Snowball sample methodology: data were excluded from totals and from
subsequent table. See endnote 5.
References
Abelson, Donald E. 2007. "Any ideas? Think tanks and policy
analysis in Canada." In Policy Analysis in Canada: The State of the
Art, edited by L. Dobuzinskis, M. Howlett and D. Laycock. Toronto:
University of Toronto Press, 298-310.
Bakvis, H. 1997. "Advising the executive: Think tanks,
consultants, political staff and kitchen cabinets." In The Hollow
Crown: Countervailing Trends in Core Executives, edited by P. Weller, H.
Bakvis and R.A.W. Rhodes. New York: St. Martin's Press, 84-125.
--. 2000. "Rebuilding policy capacity in the era of the fiscal
dividend: A report from Canada." Governance 13 (1): 71-103.
Bernier, Luc, and Michael Howlett. 2011. "La capacite
d'analyse des politiques au gouvernement du Quebec : Resultats du
sondage aupres de fonctionnaires quebecois." Canadian Public
Administration 54 (1): 143-52.
Boston, Jonathan. 1994. "Purchasing policy advice: The limits
of contracting out." Governance 7 (1):1-30.
Brinkerhoff, D.W., and B.L. Crosby. 2002. Managing Policy Reform:
Concepts and Tools for Decision-Makers in Developing and Transitional
Countries. Bloomfield: Kumarian Press.
Brinkerhoff, Derick W., and Peter J. Morgan. 2010. "Capacity
and capacity development: Coping with complexity." Public
Administration and Development 30(1): 2-10.
Bryman, Alan. 2004. Social Research Methods. 2nd ed. Oxford: Oxford
University Press.
Colebatch, H.K. 2005. "Policy analysis, policy practice and
political science." Australian Journal of Public Administration 64
(3): 14-23.
--. 2006a. "What work makes policy?" Policy Sciences 39
(4): 309-321.
--. 2006b. The Work of Policy: An International Survey. New York:
Rowman and Littlefield.
Colebatch, Hal K., Robert Hoppe, and Mirko Noordegraaf, eds. 2011.
Working for Policy. Amsterdam: Amsterdam University Press.
Colebatch, H.K., and B.A. Radin. 2006. "Mapping the Work of
Policy." In The Work of Policy: An International Survey. New York:
Rowman and Littlefield, 217-226.
Dobuzinskis, Laurent, Michael Howlett, and David Laycock. 2007.
Policy Analysis in Canada: The State of the Art. Toronto: University of
Toronto Press.
Durning, D., and W. Osuna. 1994. "Policy analysts' roles
and value orientations: An empirical investigation using Q
methodology." Journal of Policy Analysis and Management 13 (4):
629-57.
Gravetter, Frederick J. and Lori-Ann B. Forzano. 2010. Research
Methods for the Behavioral Sciences. 4th ed. Florence KY: Wadsworth
Publishing.
Halligan, John. 1995. "Policy Advice and the Public
Sector." In Governance in a Changing Environment, edited by B. Guy
Peters and Donald J. Savoie. Montreal: McGill-Queen's University
Press, 138-172.
Hawke, G.R. 1993. hnproving Policy Advice. Wellington: Victoria
University Institute of Policy Studies.
Hird, J.A. 2005a. "Policy analysis for what? The effectiveness
of nonpartisan policy research organizations." Policy Studies
Journal 33(1): 83-105.
--. 2005b. Power, Knowledge and Politics: Policy Analysis in the
States. Washington DC: Georgetown University Press.
Hollander, Marcus J., and Michael J. Prince. 1993. "Analytical
units in federal and provincial governments: Origins, functions and
suggestions for effectiveness." Canadian Public Administration 36
(2): 190-224.
Hooghe, L., and G. Marks. 2001. "Types of Multi-Level
Governance." European Integration Online Papers 5(11).
--. 2003. "Unraveling the central state, but how? Types of
multi-level governance." American Political Science Review 97(2):
233-243.
Hoppe, R., and M. Jeliazkova. 2006. "How policy workers define
their job: A Netherlands case study." In The Work of Policy: An
International Survey, edited by H.K. Colebatch. New York: Rowman and
Littlefield, 35-60.
Howlett, M. 1999. "Federalism and Public Policy." In
Canadian Politics--Third Edition, edited by J. Bickerton and A. Gagnon.
Peterborough: Broadview Press.
--. 2009a. "Policy analytical capacity and evidence-based
policy-making: Lessons from Canada." Canadian Public Administration
52 (2):153-175.
--. 2009b. "A Profile of B.C. provincial policy analysts:
Troubleshooters or planners?" Canadian Political Science Review 3
(3): 55-68.
--. 2009c. "Policy advice in multi-level governance systems:
Sub-national policy analysts and analysis." International Review of
Public Administration 13 (3).
Howlett, Michael, and Evert Lindquist. 2004. "Policy analysis
and governance: Analytical and policy styles in Canada." Journal of
Comparative Policy Analysis 6 (3): 225-249.
Howlett, Michael, and Joshua Newman. 2010. "Policy analysis
and policy work in federal systems: Policy advice and its contribution
to evidence-based policy-making in multi-level governance systems."
Policy and Society 29 (1): 123-136.
Liu, Xinsheng, Eric Lindquist, Arnold Vedlitz, and Kenneth Vincent.
2010. "Understanding local policymaking: Policy elites'
perceptions of local agenda setting and alternative policy
selection." Policy Studies Journal 38 (1): 69-91.
Maley, Maria. 2000. "Conceptualising advisers' policy
work: The distinctive policy roles of ministerial advisers in the
Keating government, 1991-96." Australian Journal of Political
Science 35(3): 449.
McArthur, Doug. 2007. "Policy analysis in provincial
governments in Canada: From PPBS to network management." In Policy
Analysis in Canada: The State of the Art, edited by L. Dobuzinskis, M.
Howlett and D. Laycock. Toronto: University of Toronto Press, 132-45.
Meltsner, A.J. 1976. Policy Analysts in the Bureaucracy. Berkeley:
University of California Press.
Murray, Catherine. 2007. "The Media." In Policy Analysis
in Canada: The State of the Art, edited by L. Dobuzinskis, M. Howlett
and D. Laycock. Toronto: University of Toronto Press, 286-97.
New Zealand. State Services Commission. 1991. Review of the
Purchase of Policy Advice from Government Departments. Wellington: State
Services Commission.
--. --. 1999. Essential Ingredients: Improving the Quality of
Policy Advice. Wellington: New Zealand State Services Commission.
Noordegraaf, Mirko. 2011. "Academic accounts of policy
experience." In Working for Policy, edited by Hal Colebatch, Robert
Hoppe, and Mirko Noordegraaf. Amsterdam: University of Amsterdam Press,
45-67.
Page, E.C., and B. Jenkins. 2005. Policy Bureaucracy: Governing
with a Cast of Thousands. Oxford: Oxford University Press.
Piattoni, Simona. 2009. "Multi-level governance: a historical
and conceptual analysis." Journal of European Integration 31 (2):
163.
Radin, B.A. 2000. Beyond Machiavelli: Policy Analysis Comes of Age.
Washington DC: Georgetown University Press.
Rasmussen, K. 1999. "Policy capacity in Saskatchewan:
Strengthening the equilibrium." Canadian Public Administration 42
(3): 331-48.
Riddell, Norman. 2007. Policy Research Capacity in the Federal
Government. Ottawa: Policy Research Initiative.
Rieper, Olaf, and Jacques Toulemonde. 1997. Politics and Practices
of Intergovernmental Evaluation. New Brunswick: Transaction Publishers.
Rochet, C. 2004. "Rethinking the management of information in
the strategic monitoring of public policies by agencies."
Industrial Management & Data Systems 104 (3): 201-208.
Saint-Martin, Denis. 1998a. "The new managerialism and the
policy influence of consultants in government: An
historical-institutionalist analysis of Britain, Canada and
France." Governance 11 (3): 319-356.
--. 1998b. "Management consultants, the state, and the
politics of administrative reform in Britain and Canada."
Administration Society 30 (5): 533-568.
--. 2004. Building the New Managerialist State: Consultants and the
Politics of Public Sector Reform in Comparative Perspective. Oxford:
Oxford University Press.
Segsworth, Robert V., and Dale H. Poel. 1997. "Two cases of
intergovernmental evaluation in Canada: 'Parallel play' and
cooperation without policy consequences." In Politics and Practices
of Intergovernmental Evaluation, edited by Olaf Rieper and Jacques
Toulemonde. New Brunswick: Transaction Publishers, 75-98.
Speers, Kimberly. 2007. "The invisible public service:
Consultants and public policy in Canada." In Policy Analysis in
Canada: The State of the Art, edited by L. Dobuzinskis, M. Howlett and
D. Laycock. Toronto: University of Toronto Press, 220-31.
Stritch, Andrew. 2007. "Business associations and policy
analysis in Canada." In Policy Analysis in Canada: The State of the
Art, edited by L. Dobuzinskis, M. Howlett and D. Laycock. Toronto:
University of Toronto Press, 242-59.
Uhr, John, and Keith Mackay, eds. 1996. Evaluating Policy Advice:
Learning from Commonwealth Experience. Canberra: Federalism Research
Centre--Australian National University.
Voyer, Jean-Pierre. 2007. "Policy analysis in the federal
government: Building the forward-looking policy research capacity."
In Policy Analysis in Canada: The State of the Art, edited by L.
Dobuzinskis, M. Howlett and D. Laycock. Toronto: University of Toronto
Press, 123-31.
Weller, Patrick, and Bronwyn Stevens. 1998. "Evaluating policy
advice: The Australian experience." Public Administration 76
(Autumn): 579-89.
Wellstead, Adam M., Richard C. Stedman, and Evert A. Lindquist.
2009. "The nature of regional policy work in Canada's federal
public service." Canadian Political Science Review 3 (1):1-23.
Wollmann, Hellmut. 1989. "Policy analysis in West
Germany's federal government: A Case of unfinished governmental and
administrative modernization?" Governance 2 (3): 233-266.
Notes
(1) On the merits and demerits of naturalistic and survey research
and the need to combine qualitative and quantitative analysis in social
research, see Bryman (2004) and Gravetter and Forzano (2010).
(2) Even less is known about both the "invisible civil
service" (consultants) and those analysts who work outside of
government in think tanks, business associations and labour unions, and
elsewhere in the NGO sector (Saint-Martin 1998a, 1998b, 2004; Hird
2005a; Abelson 2007; Murray 2007; Speers 2007; Stritch 2007).
(3) The 64-item survey instrument used in these studies asked the
respondents a series of questions about the tasks they undertake, the
nature of the issues they examine, the kinds of networks with which they
are engaged, and their attitudes towards policy making. The statistical
technique used to analyse the survey results was one-way analysis of
variance (ANOVA), which compares the means of more than two samples. In
this case the null hypothesis (H0) was
[mu]1 = [mu]2 = ... = [mu]k
where [mu]i is the mean of group i. The F statistic was constructed
for testing the hypothesis:
F = variation among sample means/variation within samples
If the means are far apart, especially relative to the variation
within each group, the F statistic is large and the null hypothesis is
rejected. Throughout this study, a test of homogeneity of variance (the
Levine test), ANOVA, and a post hoc pairwise multiple-comparison test
using the Dunnett's C method were employed. The Levine test was
used to test for equal variance. The ANOVA (F statistic) can indicate
differences among means but does not identify the means that differ from
each other. The Dunnett's c score identifies subsets of groups.
Thus, using this technique, it is possible to determine similarities and
differences between the three major groups of policy analysts. The
survey itself is available from the authors upon request.
(4) Provincial public service lists often included political
appointees who had been left off our lists. In most cases, however, our
lists and the internal lists were very close, with about an 80 per cent
or higher overlap rate. The lists revealed a roughly proportional
per-capita pattern of size of the policy analytical community in
Canadian provincial governments, with about 1,800-2,000 individuals in
Ontario, 500 in British Columbia and about 100 in the smallest
jurisdictions. The total number of policy analysts at the provincial and
territorial level, hence, is about 5,300 (3,000 in Quebec and Ontario;
1,000 in BC and Alberta; 500 in Saskatchewan and Manitoba; 400 in the
Atlantic provinces and 300 in the territories). It is expected that this
number would be matched by the federal government (Wellstead having
identified about 1,300 operating outside Ottawa, the remaining 4,000
being located in the National Capital Region) bringing the total number
of policy analysts actually employed in Canada to about 11,000.
(5) Because of problems with job classification systems and
terminology and privacy laws in Quebec, lists of analysts could not be
gathered from publicly available sources. A snowball sampling method was
instead used in which the questionnaire was sent to an initial seed of
42 potential respondents who were asked to pass the survey along to
colleagues working as policy analysts. After six weeks, approximately
250 respondents had looked at the survey, with 130 having fully or
partially completed it. Given this different data collection technique,
the Quebec results have been omitted from the tables presented below. A
separate analysis of the results from the Quebec survey, however, found
a similar pattern of responses to those found in the other 12 provinces
and territories (Bernier and Howlett 2011).
Michael Howlett is Burnaby Mountain Professor, Department of
Political Science, Simon Fraser University. Adam Wellstead is assistant
professor, Department of Social Sciences, Michigan Technological
University. The authors thank the Journal's anonymous reviewers for
their helpful comments on an earlier version of this manuscript.
Table 1. Similarities Across All Three Orders of Government (Mean
Scores)
Federal Provincial Territorial
Variable employees employees employees
I implement/deliver programs 2.83 2.90 2.87
I deal with issues that emerge 3.73 3.66 3.67
as a result of governmental
priorities in headquarters
I deal with issues that 3.04 2.90 3.02
require coordination with
other levels of government
I deal with issues that 3.65 3.64 3.50
require specialist or
technical knowledge
I appraise policy options 3.43 3.56 3.67
I consult with the public 1.96 1.97 2.17
I deal with regional issues 3.03 3.01 2.71
I identify policy issues 3.85 3.92 4.04
I deal with tasks which demand 3.94 3.81 3.79
immediate action (i.e., "fire
fighting")
I negotiate with central 2.40 2.62 2.62 **
agencies
I negotiate with program 2.85 3.06 3.07 **
managers
I deal with short-term tasks 3.69 3.69 3.71
which can be resolved in less
than a month
I deal with medium-term tasks 3.60 3.59 3.65
which are ongoing for between
1-6 months
I deal with long-term tasks 3.61 3.47 3.54
which are ongoing for between
6-12 months
I deal with urgent day-to-day 4.18 4.28 4.27
issues that seem to take
precedence over long-term
thinking
I negotiate with stakeholders 2.68 2.85 2.63 **
I consult with stakeholders 2.62 2.76 2.63 *
I think policy decisions seem 3.91 # 3.98 3.99
to increasingly be those that
are most politically
acceptable
I think there seems to be less 3.37 # 3.43 3.52
governmental capacity to
analyse policy options than
there used to be
I think much of the existing 2.76 # 2.91 2.82
policy capacity is outside the
formal structure of government
I think those who have more 3.77 # 3.80 3.74
authority in decision making
usually have less specialized
technical expertise
I think government is becoming 3.51 # 3.66 3.44 *
increasingly accountable for
its decisions
I prefer networking with 4.04 # 4.08 4.06
colleagues
I prefer more control from 2.48 # 2.69 2.62
central agencies
I prefer more control from the 2.71 # 2.80 2.76
regions
Based on a 1 to 5 scale where 1 = never and 5 = daily
# Based on a 1 to 5 scale where 1 = strongly disagree and 5 =
strongly agree
* significant at 0.05 level; ** significant at 0.01 level
Table 2. Provincial-Territorial Similarities (Federal Differences)
Federal Provincial Territorial
Variable employees employees employees
I collect policy-related data 3.01 3.98 4.05 ***
or information
I deal with provincial and 2.79 4.55 4.47 ***
territorial issues
I deal with national issues 4.27 2.61 2.71 ***
I identify policy options 3.55 3.72 3.87 **
I deal with issues that demand 2.30 2.64 2.85 ***
input from society-based
organizations
I deal with issues that emerge 3.08 3.31 3.35 ***
as a result of public pressure
on government
I deal with issues where it is 3.96 3.81 3.71 ***
difficult to identify a
single, clear, simple solution
I deal with tasks which are 4.2 3.27 3.33 ***
ongoing for more than a year
I interact frequently with 2.47 3.10 2.98 ***
senior regional management
I interact frequently with 3.72 3.83 4.00 *
other head office staff
I interact frequently with 2.5 3.09 3.07 ***
central agencies
I interact frequently with 1.4 2.14 2.02 ***
municipal government
departments
I interact frequently with 2.81 2.26 2.13 ***
federal departments in my
region
I interact frequently with 1.57 1.76 1.80 **
environment/conservation
groups
I am increasingly consulting 2.32 # 2.64 2.78 ***
with the public as I do my
policy-related work
I think policy problems 3.49 # 3.83 3.78 ***
increasingly require strong
technical expertise
I think an important role of 3.39 # 3.71 3.63 ***
the provincial government is
to foster involvement in the
policy process by other
non-governmental organizations
I think formal government 2.69 # 2.92 3.17 ***
institutions are becoming less
relevant to policy making
I think decisions about 2.64 # 2.86 3.01 ***
government programs and
operations are increasingly
made by those outside of
government
I network with 3.64 4.03 4.09 ***
non-governmental organizations
I network with other 3.77 4.36 4.39 ***
provincial government
departments or agencies
I network with municipal 3.07 3.80 3.66 ***
government departments or
agencies
I prefer more control from 3.14 # 2.83 2.93 ***
head office
I think policy capacity has 3.73 # 3.31 3.16 ***
increased
Based on a 1 to 5 scale where 1 = never and 5 = daily
# Based on a 1 to 5 scale where 1 = strongly disagree and 5 = strongly
agree
* significant at 0.05 level; ** significant at 0.01 level;
*** significant at the 0.001 level
Table 3. Federal-Provincial Similarities (Territorial Differences)
Federal Provincial Territorial
Variable employees employees employees
I deal with issues that 3.14 3.12 2.82 *
require coordination across
regions
I interact frequently with 3.67 3.85 *** 4.14 ***
senior head office-based
management
I interact frequently with 1.81 1.82 *** 1.44 ***
think tanks
Based on a 1 to 5 scale where 1 = never and 5 = daily
* significant at 0.05 level; *** significant at 0.001 level
Table 4. Federal-Territorial Similarities (Provincial Differences)
Federal Provincial Territorial
Variable employees employees employees
I deal with issues that 3.77 3.50 3.79 ***
require coordination with head
office
I deal with issues where it is 3.96 3.81 3.71
difficult to identify a
single, clear, simple solution
I interact frequently with 2.11 2.36 1.97 ***
industry organizations
I interact frequently with 1.47 1.74 1.56 ***
labour organizations
My policy-related work 3.56 3.74 3.44 **
increasingly involves networks
of people across regions or
levels of government or even
outside of government
Based on a 1 to 5 scale where 1 = never and 5 = daily
** significant at 0.01 level; *** significant at 0.001 level
