Academic research and engagement with the Canadian public service: Insights from three surveys.

Migone, Andrea R. ; Brock, Kathy L.

Introduction

Public policy and decision-making literatures have had a long engagement with the provision, distribution, and use of knowledge (Jennings and Hall 2012; Howlett and Newman 2010; Young 2013). Over the past decades the analysis of these patterns has increasingly become more sophisticated (Vesely 2017; Craft and Halligan 2017; Newman, Cherney and Head 2017; Howlett, Wellstead and Craft 2017) and we give a brief account of this trajectory below. Despite the richness of the literature, the specific role of academics in the provision of policy advice and research and the attitudes of the two communities to each other have been less studied. In this article, we use a survey of public sector executives and two surveys of academics to illuminate this relationship between the two communities in Canada. Do they exist in two worlds as the literature suggests? Are academics becoming less concerned with and less relevant to the production of policy as other communities become more integrated into the policy world? While our results are preliminary, the Canadian case begins to suggest that while senior public sector executives seem more receptive to academic knowledge than the literature suggests, the use of academic research is constrained by key factors identified in the literature such as the diminishing capacity within the public sector to discern and effectively use good qualitative research to address social problems and define policy objectives. More surprisingly, academics themselves may not be as concerned with helping to close this gap as the literature surmises.

This article is organized in three sections. In the first section, we briefly describe the recent evolution of the literature on policy advice and evidence-based decision-making. In the following section we outline our research questions and introduce the results from the initial round of our surveys. We conclude by answering our research questions and tying these findings back to the main trends in the literature.

Knowledge, advice and policy-making: three key debates

Public policy scholars have long studied the connection between decision-making, policy advice and evidence. At least since the 1970s, that focus shifted towards evidentiary advice, which the literature generally considers desirable and positively correlated with successful implementation. As a result, evidence-based decision-making (EBDM) and policy advice have become common concepts in the effort to analyze and explain the inputs, roles, and patterns of activity that exist in the public policy ecosystem. Meanwhile, they have been at the center of three major debates in the discipline. The first is a constructivist-rationalist debate on the nature of knowledge. A second involves the historical evolution of the use of knowledge and advice within policy-making environments, including the relationship between policy-makers and knowledge producers. Finally, various questions involve understanding why something like EBDM seems to have had so little measurable effect on actual policy implementation. Below we provide a general overview of these points.

The use of knowledge in policy-making has led to discussions about the sources of that knowledge, its connection with established power relations, and its potential applications. This has often been framed as a clash between constructivist (Neylan 2008) and rationalist arguments (Chalmers 2005), and around two main areas of contention. The first is about the way in which evidence is understood: to what extent can we have evidence that is not contested, socially constructed and subjectively defined. Second, can we use evidence and knowledge to solve social problems in a rational fashion (Davies 2012) or is this just a naive representation of an extremely complex multi-level system (Freiberg and Carson 2010), and what constitutes "good evidence" for each problem? Part of the literature (Newman 2016; Smith 2013) argues that the two approaches can be bridged and offer important common insights as to the limits of evidence-based decision-making but most of the field remains locked in this opposition.

Multiple communities are engaged in the policy-making ecosystem and in the production, distribution and use of knowledge within it. These communities differ in terms of values, internal logics and structures (Vesely 2017), and across jurisdictions (Craft and Halligan 2017). Historically, relationships among these actors were originally studied through locational models where influence depended on whether they produced, consumed or brokered advice and knowledge (Clark and Jones 1999; Howlett 2011; Cappe 2011). These models tended to be unidimensional, focusing on government knowledge uptake (Craft and Halligan 2017). However, over time increasing attention was paid to processes of externalization, which sometimes created intense competition in the supply of advice, and to the evolution of the advisory process (Savoie 2015; Weller 2015) where administrators increasingly became more engaged in "the politics of policy advice" (Halligan 1995: 160). If Wildavsky (1979) focused on "speaking truth to power," later authors found "weaving" (Parsons 2004) or "sharing truth with many actors of influence" (Prince 2007: 179) closer to the reality of policy advice. In short, we have paid much more attention to "the configuration, operation and dynamics of policy advisory systems" (Craft and Halligan 2017: 57) developing richer and stronger images for them.

One of the threads running through the literature is the relationship between academics and public bodies as consumers of knowledge. This relationship was never straightforward, with most of the analysis trying to explain why this seldom happens. During the 1970s, scholars focused on research utilization privileging the amount of academic research used by the public service as metrics (Caplan 1979; Booth 1990). These analyses yielded the image of "two communities," depicting policy makers and academics as separate and different in their goals, focuses and approaches (Weiss 1979, 1980). The literature also argued that policy analysts had become progressively less relevant in solving social problems by working with governments (Lasswell 1970; Howard 2005). Meanwhile, in the late 20th century, policy advisory systems and policy-making processes became increasingly complex and multi-layered (Gregory and Lonti 2008; Vesely 2017) and some of the internal logics shifted.

Notably, by the late 1990s the use of evidence-based advice, fashioned after the approach taken in health care (Boaz et al. 2008), was considered a necessary ingredient of policy-making by the United Kingdom's Cabinet Office (1999), and taking hold there (Boaz et al. 2008), in Canada (Howlett 2009; Young 2013), Australia (Head 2010a), the United States (Hall and Jennings 2010), the European Union (Bohme 2002), and the OECD. Alongside the increased focus on evidence (Sanderson 2006), the literature speaks of a progressive disengagement by academics from participating in discussions about solving policy issues outside of academic confines (Flinders 2013; Van der Arend 2014).

The scholarship itself evolved considerably in its description and analysis of the field. The two communities metaphor has been abandoned as a much more complex approach to the policy system has been introduced (Howlett and Wellstead 2011; Vesely, Wellstead and Evans 2014; Cherney et al. 2015), where the:

main focus is no longer to improve the uptake of academic research for the sake of research salience; it is now to improve the use of relevant research because it is believed that policy fully informed by research evidence will produce better outcomes for citizens (Newman and Head 2015: 384).

We also saw a perceptible shift by part of the scholarship and practice toward evidence-informed (as opposed to evidence-based) decisionmaking (Head 2015) correcting the original approach by recognizing that the limited ascertained impact of EBDM on the policy field can be explained by different information needs and practices of public organizations. Hence, while the instrumental use of knowledge (where knowledge is used to directly solve a social problem) may be theoretically desirable, it is likely that both conceptual (to generate ideas that affect an issue over longer periods) and political (to rationalize previously chosen policy) uses will be rather common within political environments (Daviter 2015) and especially so if these political environments underwent a process of "procedural politicization" (Eichbaum and Shaw 2008: 343). Furthermore, both the rigorous methodological and resource premises required by evidence-based policy advice (Head 2015; Howlett and Craft 2013) and the availability of practical elements such as the capacity of the public service to understand evidence (Newman, Cherney and Head 2017) and process it into policy (Howlett 2015; Lindquist 1988) remain critical components to the success of any evidence-based approach. In practice, the careful assessment of the connections between policy advice and policy outcomes that has been at the core of much of the recent literature has partially shifted the focus of inquiry from "what works" to the "best available evidence."

A final debate exists in the literature surrounding policy advice and knowledge. How can we explain what appears to be a minimal impact on the actual process of policy-making (Head 2010a, 2010b; Corbett and Bogenschneider 2011; Cartwright and Hardie 2012; John 2013; Watts 2014; Head 2014; Newman 2016) even as both the literature and folk wisdom underscore its relevance and there is a substantial body of literature arguing for its use (Head 2013; Howlett and Wellstead 2011). Attempts at explaining these underwhelming results rely on four arguments. The political use of evidence reduces the effectiveness of the process for example through biased selection of information and goals (Weiss 1979; Head 2013; Knaggard 2014). The existence of multiple potential interpretations of evidence (Watts 2014), especially in social policy fields and existing cultural differences between those who produce evidence and those who use it both affect the process (Mead 2015). The poorly and/or broadly defined policy objectives often associated with complex policy areas can reduce the effectiveness of evidence-based decision-making (Newman and Head 2015; Head 2015). Finally, the possibility that public service personnel and organizations may be unprepared to deal effectively with diverse forms of evidence is noted as a further potential dampening effect (Howlett 2015; Newman, Cherney and Head 2017).

Ultimately the literature acknowledges that policy-making is a complex process, where policy success is a multidimensional and at least in part political--and therefore subjective--exercise (Newman 2014). Policy makers are bounded rational actors who use both rational strategies to reduce empirical uncertainty and irrational ones to reduce political ambiguity. In this environment multiple actors compete as sources of knowledge and engage over the long-term with policy-makers (Cairney 2016; Cairney, Oliver and Wellstead 2016), and simply supplying policy advice does not ensure that it will be used (Head 2015). As a result, policy advice exists in a highly contextualized environment (Wesselink and Gouldson 2014) where there is no immediate, linear connection between it and policy decisions (Head 2013; Wesselink, Colebatch and Pearce 2014). In reality political and other considerations mediate between the two, multiple evidence sources add to the "messiness" of the process, and ideal decision-making conditions are about as common as perfect competition.

Here we begin to shed light on the role of academics within the Canadian policy-making ecosystem. The effect of academic policy advice on policy-making is not easily framed. Part of the literature and a variety of anecdotal analyses argue that within the public service academic research is seen as being generally only weakly relevant to political or administrative policy work because of a lack of practical connections to political and administrative realities and because of broadly diverse goals and epistemic frameworks between the two communities (Mead 2015). Multiple surveys found that public servants only weakly rely on academic research (Weiss 1980; Lester 1993; Landry, Lamari, and Amara 2003) and uncovered symptoms of weak policy capacity in various public organizations (Howlett and Newman 2010). That said, we should note that even within the scope of many of these analyses there are indications that a sub-group of these policy agents actually uses academic knowledge and is interested in embedding it in their work (Landry, Lamari, and Amara 2003; Jennings and Hall 2012; Head et al. 2014; Vesely, Wellstead and Evans 2014; Cherney et al. 2015).

Perhaps then, these disparate results may not necessarily indicate that there is no space for academic knowledge and research in the public sector, but rather that the uptake and use seems to depend on a variety of variables, which include the type of jurisdiction, the end-user of that research, the type of policy field, the policy actors' approach and so forth. The next section deals with the results of our surveys and attempts to map the role of a subset of academics within the Canadian policy advisory community.

Patterns of engagement of Canadian academics in the policy community

This article proposes two lines of inquiry. The first is the role of the academic community as a specific subset of the actors involved in Policy Advisory Systems (PAS) in particular trying to uncover the extent to which academics play a role in the provision of external policy advice. We divide the latter into "formal," which we chose to identify with contractual relations, and "informal," which is represented by participation in networks and informal conversations. With the second line of inquiry, we explore some of the more general patterns of engagement that academics have with the public service and look at academic perceptions of how these relationships evolved, but going beyond the provision of policy advice. We wished to answer empirically two research questions:

* RQ 1: What role are Canadian academics playing in providing advice and research to public servants?

* RQ 2: What are the salient features of the relationship between Canadian academics and the public service (that is, formalized through contracts, informal, nature, frequency)?

The literature that we have reviewed above indicates that academics do not play a key role in most policy-making activities and their role has declined despite the shift to EDBM, but most of that literature focuses on the perspective of policy analysts operating within the public service or on an analysis of government budgets. We wished to extend this analysis by focusing on the relationship of the academic community to the public sector. We were particularly interested in whether the experience in Canada would bear out the general findings in the literature.

Profiles of respondents

This section presents data from three surveys (see Table 1). The respondents to the November 2014 survey of Canadian Senior Executives (Deputy Ministers, Chief Administrative Officer of major cities, and Chairs, CEOs and Heads of agencies, boards and commissions) tended to be disproportionately from the provincial and territorial levels of government with few responses from local government or agencies and other bodies. As expected, over 70% were over 50 years of age. A May 2015 survey of all Canadian university professors specializing in public policy, public administration and political science mirrored the two questions asked of public executives about engagement with academic knowledge and inquired further about the terms of engagement and contracting and the state of the relationship. (1) The same survey was administered in early 2016 to all professors at a mid-sized comprehensive university and yielded answers from a broad array of disciplines as Table 2 shows. Both pools of academic respondents tended to be senior (full or associate) professors with at least one year of prior experience in government (73%). In sum, our respondents tended to be senior professionals with some knowledge of the other community.

The role of academics in the Canadian policy advisory community

In this section we examine the role of academics in the Canadian policy advisory community by comparing the responses of the senior Canadian public service executives and of the two surveys of academics. We asked respondents to rank on a 5-points frequency Likert Scale2 how often various types of engagement occurred. All of the responses are available in Appendix but in Table 3 below we highlight the five roles with the highest degree of participation in the knowledge exchange and policy advice area between academics and the public service. While the responses seem to indicate that many academics have relatively infrequent engagement with the public service, the frequencies of engagement (at least every month for "Often" and more than twice per month for "Very Often") can be considered relatively high in this context. For example, it is unlikely that an agency or department would frequently second personnel to an academic unit considering the complexity of the arrangement, so numbers on that area should not be considered damning.

Academics are relatively unlikely to sit on boards, panels or committees, nor do they attend many workshops or training sessions with public servants. It is also uncommon for them to receive requests for the production of research reports from public service officials, which is sometimes thought of as the "new normal" in the externalization of policy advice--especially where private actors are involved. The most common activities for these academics are of an informal kind: participating in networks and offering/ being requested informal advice. There is little difference between the two academic samples.

A marked difference, however, emerges between the responses of academics and of public service executives. Public sector executives indicate that they discuss academic research in meetings but few academics indicate that they participate in such meetings. Generally, academics report policy activity at least half as frequently as executives; at worst, they report just 20% of that activity. The next section delves deeper into those responses, but here the biggest differences exists for the "Never" answer where the gap between senior executives and academics ranges between 30 and 45 percentage points. Most academics report never providing advice or knowledge to the public service. These patterns seem to imply a polarization: a small percentage of academics in our sample provide knowledge and advice, especially in informal fashion. On the other hand, public sector senior executives seem to seek that input much more frequently but perhaps from this more select group.

Engagement with the non-profit and the private sectors

Here we map how frequently the academic and public sector communities engaged with the non-profit and private sectors (Table 4). In the academic field, the cross-discipline university survey shows a lower number of and less frequent contacts with social-enterprises and non-profits than the policy-oriented survey but more frequent engagement with the private sector. This difference may be a reflection of the tendency of PP & PA academics to engage more with the public sector than the private sector and the tendency of other disciplines, like business and the sciences, captured in the university survey to engage more with private and non-profit sectors.

Public service executives have a much higher frequency of contact with both communities than the academics do. The public sector responses are a fairly close inverse pattern of the academic ones. This is not surprising considering that in an era of New Public Management and "Deliverology," the public sector is increasingly engaging with actors from the non-profit and private sectors in the development and delivery of policy. In contrast, academics tend to be more focused on research and publications with the nonprofit and private sector as subjects of study rather than as partners. However, it raises the question of whether the public sector is turning more to the other sectors than to academics as its capacity for policy creativity is being hollowed out.

Finally, we can assess the relative frequency with which academics and public sector executives engage in research with (or about) one another. In Table 5, we can see how, during the previous year, a majority of academic respondents working in public policy, public administration and political science (about 53%) engaged in surveying or interviewing members of the public service. This appears to be a relatively high percentage of direct research emerging from this sample. However, we find a marked difference in the sample drawn from the University cross-discipline survey. Here only 21.3% engaged in surveying or interviewing members of the public service, which may again be a reflection of the different focus of the work by the PP & PA community versus other disciplines. However, it is not surprising that both groups of academics report broadly similar frequencies (around 30%) when partnering with public servants in research.

In contrast, 89% of senior public sector executives reported participating or engaging in academic research during the previous year. This disparity in reported engagements between the two communities may suggest that public sector executives are more regularly engaged with the same subsection of the academic community. Over 18% of executives were regularly engaged with academic research, and surprisingly, given that such research is often time-consuming to design and implement, we have strong rates of engagement from our policy-oriented academic sample, with at least 11% showing a constant engagement but much lower levels of consistent engagement when the academic community is broadened out to include other disciplines.

Research approaches

Our surveys of academics delved deeper into their perceptions of their relationship with the public sector, their preferred research tools and techniques and whether they were inclined to bid on contracts or provide policy advice to governments. The purpose was to ascertain the nature and possible formal and informal forms of engagement between academics and public sector officials, particularly in light of the previous public sector responses that they tended to engage with academics frequently or fairly often.

Three observations emerged when we examined how satisfied academics were with their relationship with the public service. Table 6 indicates that academics involved in political science, public policy and public administration are slightly more frustrated with their relationship with public servants when it comes to academic research than the University cross-disciplinary sample. This different level of satisfaction was even more pronounced when the satisfied and very satisfied responses were combined. Are the PP & PA academics harder to please or are relationships better in other fields? For both groups, the large portion (well over a third) of academics, who were blase or indifferent about the relationship was striking. Overall then, opinion among academics seemed divided on the prospects of engaging public servants in academic research in a satisfying way.

The comments of the respondents added nuances to these results. Academics tend to perceive federal public servants as being more difficult to engage in research than those from other jurisdictions. Some academics also perceived that their research was not interesting to public servants.

Given assumptions in the literature that the level of academic engagement with the public sector has been declining over the past few decades, we asked academics whether they perceived a change in the quality of that engagement over the previous five years (Table 7). Only 14.6% of respondents in the policy-oriented sample thought it had improved in any way, while 46.9% believed it had worsened. The University survey showed overall much better trends with a lower perception of a worse situation (33.6%) and more academics stating that engagement was better (17.9%). This might indicate that engagement has worsened at the policy level where PP & PA academics would focus while it has remained better at the project level where more science, medical and business scholars would interact. We noted the concentration of academics in both groups perceiving no change.

This question provoked many comments, allowing us to delve further into the perceptions of academics. From a general viewpoint, these answers tell us that academics see the public service as having become more inward-looking, and as having lost the capacity and/or the desire to participate in research undertaken by academics. In short, academics perceive a reticence in the public sector towards academia, the reasons for which are variously attributed. Most notably, academics stated that public servants operated in an environment that placed increased restrictions on their capacity to participate in research. Interestingly, this perception has been repeated to us in anecdotal conversations with retiring public servants.

The identified impediments to public sector engagement with academic research included budget pressure, increased media scrutiny, and strong political controls over communication. However, alongside this "loss of capacity" explanation, we also find a "cultural" one. The latter ascribes the reduced quality and scope of interactions between academics and public servants to an increased anti-academic turn in both public servants and politicians, especially at the federal level of government. While a number of commenters attributed this change to elected federal officials and their political staffers, a minority noted that this approach may actually have become embedded among senior administrators as well. Consistent with this view, we noted the academics' perception that avenues for collaboration that had existed a few years back at the federal level have disappeared.

While these statements would need increased testing and research, they seem to speak against two important trends that have emerged in the research field in recent years. The first is the increasing relevance of geographically and organizationally distributed networks and pipelines of research. The second is the focus on triple-helix models of research and innovation, which involve closer working relationships among university, industry and government as the means of creating new social and organizational formats for the production, transfer and application of knowledge and, ultimately, increased levels of innovation (Ranga and Etzkowitz 2013; Etzkowitz and Leydesdorff 1995). These findings also may be consistent with the trend noted in the literature towards the public sector eschewing general or high-level policy advice but not policy relevant research. This would also explain the difference in responses between the policy-oriented academics and university academics in the two surveys since the latter were more positive about the relationship. In either case, the results are worrying given the move away from the process of knowledge creation taking place within government to knowledge creation occurring outside government and being absorbed by it. Our findings may indicate that academics in general, but more particularly public administration academics, may not be seen as the preferred source of knowledge transfers, especially by federal public officials. If true, this finding is consistent with the literature.

To better understand the role of academics as creators of knowledge, we asked our respondents what tools they use to engage in research about and with the public service. While Table 8 reveals that interviews and informal discussions are the most used tools by policy academics as well as the university academics, it also captures a striking difference in approaches favoured by respondents to the two surveys. The PA & PP surveyed academics clearly preferred interviews and informal discussions but the academics in the university survey demonstrated only a slight preference towards those tools and a fairly strong preference for surveys, case studies, focus groups, lab and experimental work, and field work as well. Not only was this a reflection of the different instruments used by different disciplines, it also reflected the usage of more empirically verifiable research methods in non-PA fields. Although the use of interviews and informal discussions fits with the findings from the Senior Executives Survey regarding informal connections, the use of more empirically verifiable techniques fits with the trend towards policy relevant research and the more positive relationship of the university academics towards the public sector relationship than of the PP & PA academics. Again, this may indicate that policy-oriented academics may be a less coveted source of knowledge than academics whose research is directly relevant to current policies and projects and is empirically verifiable rather than being subject to multiple interpretations. It also speaks to the decreasing capacity in the public sectors to engage with and discern good qualitative research as a means of addressing social problems. On a more positive note, one glimmer of hope for all types of research lies in the relatively high success ratios with regard to all instruments. We can somewhat safely assume researchers have a good understanding of what approaches or research tools work best in their specific research environment.

Our next question was designed to assess the extent to which academics were responding to the dominant mode of research and partnership management by governments: contracting out. This provides insight into how formal the relationships are and whether they are mirroring relationships between government and the non-profit and private sectors. We asked how often academics engaged in bidding on competitively advertised government contracts, how often they had been selected for a sole-source contract and how often they had provided policy analysis advice. The next four tables (Tables 9a-9d) reveal the results broken down further by the order of government (Aboriginal, local, provincial and territorial, and federal).

A large majority of academics do not seem to engage in contractual research or provide policy analysis with government on a sustained or regular basis. Of the academics who do engage in these activities, bidding on federal contracts is most frequent, followed relatively closely by bidding on Provincial/Territorial contracts, and very limited engagement of this type with Local and Aboriginal governments. In terms of the two academic samples, patterns of contracting are not very different at the Federal and Provincial levels. At the Local and Aboriginal levels though, public policy academics are slightly more likely than the university academics to engage in contracted research. Perhaps not surprisingly, the PP & PA academics are more likely to provide policy advice at all levels of governments.

However, when the Never and Seldom answers are tabulated, then the university academics are more likely to rarely or not provide advice at Provincial/Territorial (98%), Local (100%), Aboriginal (99%) levels than at the Federal level (88%) whereas the policy academics are more likely to never or seldom advise at the Federal (97%), Local (99%) and Aboriginal levels (99%) than at the Provincial/Territorial (88%), although the seldom responses are higher for policy academics in all categories.

Four important initial conclusions can be drawn from the distribution of these answers about contracting and providing policy advice. First we notice that the role of academics as potential providers of formalized knowledge and policy advice through contracting is markedly different whether we are talking about the Federal, Provincial and Territorial jurisdictions on the one hand, or the Local and Aboriginal ones on the other. The former areas see anything between 15% and 20% of academics actively engaged every 2 year cycle in this formalized process. The latter jurisdictions have generally lower numbers of academics bidding on contracts and show an imbalance in favor of the policy-oriented academic community in this activity. Second, within this group of academics contracting with the Federal and Provincial/Territorial governments, we also can identify a subset of "super-providers" as 3% exhibits a very high frequency of formalized interaction. Third, the high rate of non-participators indicates that most academics are not utilizing the dominant mode of engagement favoured by governments. This may reflect a bias in the academic community towards independent research or a bias in the public sectors against using academic research unless it is directly relevant to a core policy initiative or a bit of both. Fourth, the results for contracting and, even more so, for providing policy advice are a bit hazy and need further study before firm conclusions may be drawn.

To shed light into these practices, we examined federal contracting data (Table 10). (3) This data is just a proxy but analyzing which contracts universities won from the federal government helps us to gauge the relevance of academics as knowledge providers. Overall, since 2009 universities secured federal contracts for almost $272M in various areas some of which were not knowledge related. Table 10 presents an overview of GSIN categories by highest amounts contracted to universities. In italics are those contracts that we considered falling within areas more likely to including policy advice, such as R&D in humanities and social sciences. We also include the total amount of contracts the federal government has allocated in the same period for the GSIN codes included in the table.

Universities do exceptionally well in terms of vaccine R&D, securing 97.6% of all federal contracts in that area. Dental services (37.1%), transportation R&D (26.0%) and medical R&D (23.8%) also represent rather relevant contractual services provided by universities although they are unlikely to provide much in the way of knowledge and policy capacity. It is also evident that the hard sciences take the lion's share of the contracts. What is surprising is that knowledge production areas where universities are often imagined as holding a comparative advantage in supporting policy-making (such as research and writing or humanities and social sciences) do much worse than the hard sciences. In all these areas, which appear italicized in the table, the Canadian government allocated almost $2B in contracts since 2009 and only 0.6% (just over $11M) was secured by university suppliers with the greatest share found in consulting on management and organizational development, which tends to be concentrated in the economic field. Financial data does not allow us to establish whether the contracts generated actual policy advice or knowledge transfer. However, it is a proxy for the "competitiveness" of universities in the bidding process and for their share of externally contracted knowledge.

Three observations about this data fit with our survey findings. First, academics generally are not choosing to play the contract game and governments do not tend to view academics as their primary partners or sources of research and policy advice. Second, governments are more likely to turn to the academic community for policy relevant research that is empirically verifiable; hence the higher percentage of contracts awarded to universities in the areas of vaccines, dental services, transportation, communications and medical research, accommodations and food services, and spacecraft instrumental development. Third, and with caution since more evidence is necessary, this data and our survey results regarding the underutilization of academic research and policy-oriented research in particular may indicate a lack of capacity in the public sector to identify good qualitative research and understand how to use it effectively to solve social problems. When multiple interpretations of data or policy solutions are available, then academic research seems to be less compelling to policy makers operating in a hyper-active and increasingly complex policy world.

Conclusions

Earlier we posed two research questions on which we hoped to shed some light.

* RQ 1: What role are Canadian academics playing in providing advice and research to public servants?

Our surveys suggest that the role of academics in the policy-making process is both active and passive. For example, in an evidence-based approach to policy-making we would expect a fair level of consumption of research on the part of the policy makers. This seems to be in place since over a third of Canadian public service executives respond that they participate Often or Very Often in meetings that discuss academic research, and only seven percent never did. However, we also noted that academics did not report engaging in such meetings on a regular and consistent basis. While more academics from the PA & PP community were likely to participate in these meetings or participate in a policy network or be invited to serve on committees, neither the policy academics nor the academics from a variety of fields had very high levels of participation. We also found that the greatest majority of academics--at least 80%--never provides advice or knowledge to the public service whereas senior executives in the public administration report that they seek such input much more frequently. This suggests that while public servants might not be averse to using academic research perhaps given its online availability, most academics were not directly and personally involved in providing that research. Their role may be more passive with publications speaking for them. However, a subset of academics did report that they were engaged in these forms of activities as well as in being commissioned to provide reports and research or bidding on contracts, suggesting that a smaller subset of academics, particularly from the policy community is playing a direct, active role in the policy process.

A second finding is that the role of policy-oriented academics may be substantively different from that of academics in other disciplines. We noted that the university academics tended to be slightly more satisfied with the ability to engage public servants in research and were more likely to view the relationship with public servants as having remained the same or improved over the past five years. We wondered if this may indicate that while the role of academics in the policy area may be declining, the role of academics in providing more policy and project relevant advice and research may not be. Given the higher tendency of the academics from a wider array of disciplines to use more quantitative and empirically verifiable research techniques, it may indicate a higher comfort level in the public service for hard data and policy specific or relevant research rather than research that is more subjective, based on qualitative analysis and open to various interpretations. The comments by academics that the loss of capacity and the cultural shift in the public sector to a more politicized, high pressure environment buttressed this interpretation. Further, the snapshot of federal contracts indicated that while the academic role was robust in the more empirically and measurable areas of policy (vaccines, dental, medical, transportation etcetera) served by the hard sciences, the academic role in "softer," high-level policy areas was weaker. However, these findings are tentative and more research is necessary to verify them.

* RQ 2: What are the salient features of the relationship between Canadian academics and the public service?

Our surveys indicate that academics provide both informal and formal inputs to the policy advisory system. Informal inputs are much more common than more formal ones (commissioned research or contract bids). Hence, providing informal advice and participating in a network of public servants and academics trump a variety of other formalized approaches. These patterns seem to imply a polarization: a small percentage of academics in our sample provide knowledge and advice, especially in an informal fashion. While these trends hold more or less across both surveys of academics, specialists in public policy, public administration and political science are markedly more likely to be involved in informal activities than the cross-disciplinary group of the University survey.

Our analysis allowed us to reflect on the more formalized subset of contracting. We found a strong differentiation between the level of activity of academics (bidding on contracts, being selected as sole-sourced providers, and providing policy analysis) for federal, provincial and territorial public services and for local and Aboriginal ones. For the first three jurisdictions, between 10% and 20% of respondents are involved. For the latter it is generally around 3%. Perhaps the more interesting finding here is the presence of a small but very active group of "super providers" of policy advice and research products who is frequently in contact with the public service. Finally, the large numbers of academics not pursuing formalized research contracts suggest that they may prefer engaging in independent research and resist the dominant mode of governments to foster or control the research being provided. On the side of the public sector, it may suggest a bias towards policy specific and relevant research. This impression was reinforced by the cursory look at federal contracting practices.

Both the federal contracting practices and our surveys indicated that academics are operating in a more competitive environment of knowledge creation and production. The relationships of the public sector to the private and non-profit communities remain strong. While the academic policy community is less likely than the broader academic community to turn to those sectors as partners, neither academic community seemed to report a high level of engagement. This data seems to indicate that just as the literature noted that policy analysts had become progressively less relevant in solving social and policy problems, academics in particular may be less relevant as partners in finding solutions, at least actively. Consistent with the literature, these findings raise questions about the quality of evidence being used by the public service and how to improve the use of academic research but the answers are not found here.

In terms of the quality and nature of the relationship between academia and the public service, our results captured some dissatisfaction with the relationship among academics, particularly those from the policy community. However, a majority were inclined to be either neutral or satisfied with its quality, with the cross-disciplinary group being both more likely to be satisfied and less likely to be dissatisfied than the policy specialists. The majority of academics were not inclined to engage directly with the public sector nor to be worried about improving the relationship or the use of their research by public servants. The two solitudes continue.

In sum, our research verified the general trends in the literature with some important departures and qualifications. Most notably, public servants remain open to the use of academic knowledge in the production of policy even as the policy environment has become more complex and multifaceted and politicized. A subset of academics is engaged and actively involved in ensuring that policy relevant research is used effectively in addressing a myriad of issues. However, the bias is still towards hard data and the capacity of the public sector to utilize effectively good qualitative date may be declining. It is not clear that the majority of academics are motivated to redress this problem or to ensure that their knowledge is used to its full potential.

Notes

(1) In this initial research we were unable to include all Canadian academics in our survey so we chose the subset of policy-oriented academics since these specializations often appear as dominant among policy advisors in the Canadian system (Howlett and Wellstead 2011; Howlett, Migone and Tan 2014; Howlett and Migone 2014) but we injected a broader, comparative perspective by replicating the survey across all disciplines at a selected university. The next phase of research will include multiple disciplines in universities across Canada. Detailed data on the respondents to the three surveys is available upon request.

(2) The scale was Never; Seldom (not more than twice per year); Occasionally (at least every three months); Often (at least every month); and Very Often (more than twice per month).

(3) The only budget open data structure that supported the detail required by our analysis was the federal one. Provincial and Territorial data were too highly aggregated for our intent.

References

Boaz, Annette, Lesley Grayson, Ruth Levitt and William Solesbury. 2008. "Does evidence-based policy work? Learning from the UK experience." Evidence and Policy 4 (2): 233-53.

Bohme, Kai. 2002. "Much ado about evidence: Reflections from policy making in the European Union." Planning Theory & Practice 3 (1): 98-101.

Booth, Tim. 1990. "Researching policy research: Issues of utilization in decision making." Science Communication 12 (1): 80-100.

Cabinet Office (United Kingdom). 1999. Modernising Government White Paper. CM4310. London: TSO.

Cairney, Paul. 2016. The Politics of Evidence-Based Policy Making. London: Palgrave.

Cairney, Paul, Katharine Oliver and Adam Wellstead. 2016. "To bridge the divide between evidence and policy: Reduce ambiguity as much as uncertainty." Public Administration Review 76 (3): 399-402.

Caplan, Nathan. 1979. "The two-communities theory and knowledge utilization." American Behavioural Scientist 22 (3): 459-70.

Cappe, Mel. 2011. Analysis and Evidence for Good Public Policy: The Demand and Supply Equation. The 2011 Tansley Lecture, April 19, Johnson-Shoyama Graduate School of Public Policy. Regina, Saskatchewan.

Cartwright, Nancy and Jeremy Hardie. 2012. Evidence-Based Policy: A Practical Guide to Doing It Better. Oxford: Oxford University Press.

Chalmers, Iain. 2005. "If evidence-informed policy works in practice, does it matter if it doesn't work in theory?" Evidence and Policy 1 (2): 227-^12.

Cherney, Adrian, Brian Head, Jenny Povey, Michele Ferguson and Paul Boreham. 2015. "Use of academic social research by public officials: Exploring preferences and constraints that impact on research use." Evidence and Policy 11 (2): 169-88.

Clark, Julien R. A. and Alun Jones. 1999. "From policy insider to policy outcast? Comite des Organisations Professionnelles Agricoles, EU Policymaking, and the EU's 'agrienvironmental' regulation." Environment and Planning C: Government and Policy 17 (5): 637-53.

Corbett, Thomas J. and Karen Bogenschneider. 2011. Evidence-Based Policymaking: Insights from Policy-Minded Researchers and Research-Minded Policy Makers. New York: Taylor and Francis.

Craft, Jonathan and John Halligan. 2017. "Assessing 30 years of Westminster policy advisory system experience." Policy Sciences 50 (1): 47-62.

Davies, Philip. 2012. "The state of evidence-based policy evaluation and its role in policy formation." National Institute Economic Review 219 (1): R41-52.

Daviter, Falk. 2015. "The political use of knowledge in the policy process." Policy Sciences 48 (4): 491-505.

Eichbaum, Chris and Richard Shaw. 2008. "Revisiting politicization: Political advisers and public servants in Westminster systems." Governance 21 (3): 337-63.

Etzkowitz, Henry and Loet Leydesdorff. 1995. "The triple helix: University-industry-government relations: A laboratory for knowledge-based economic development." EASST Review 14: 14-19.

Flinders, Matthew. 2013. "The politics of engaged scholarship: Impact, relevance and imagination." Policy and Politics 41 (4): 621-42.

Freiberg, Arie and W. G. Carson. 2010. "The limits to evidence-based policy: Evidence, emotion and criminal justice." Australian Journal of Public Administration 69 (2): 152-64.

Gregory, Robert and Zsuzaanna Lonti. 2008. "Chasing shadows? Performance measurement of policy advice in New Zealand government departments." Public Administration 86 (3): 837-56.

Hall, Julie L. and Edward T. Jennings. 2010. "Assessing the use and weight of information and evidence in U.S. state policy decisions." Policy and Science, 29: 137-47.

Halligan, John. 1995. "Policy advice and the public sector." In Governance in a Changing Environment, edited by B. Guy Peters and Donald Savoie. Montreal: McGill-Queen's University Press, pp. 138-72.

Head, Brian W. 2010a. "Reconsidering evidence-based policy: Key issues and challenges." Policy and Society 29 (2): 77-94.

--. 2010b. "From knowledge to knowledge-sharing? Towards better links between research, policy and practice." In Knowledge Brokering, edited by Gabriel Bammer, Annette Michaud and Ann Sanson. Canberra: ANU e-Press, pp. 109-23.

--. 2013. "Evidence-based policy-making for Innovation." In Handbook of Innovation in Public Services, edited by Stephen P. Osborne and Louise Brown. Cheltenham: Edward Elgard, pp. 143-56.

--. 2014. "Public administration and the promise of evidence-based policy: Experience in and beyond Australia." Asia Pacific Journal of Public Administration 36 (1): 48-59.

--. 2015. "Relationships between policy academics and public servants: Learning at a distance?" Australian Journal of Public Administration 74 (1): 5-12.

Head, Brian W., Michele Ferguson, Adrian Cherney and Paul Boreham. 2014. "Are policymakers interested in social research? Exploring the sources and uses of valued information among public servants in Australia." Policy and Society 33 (2): 89-101.

Howard, C. 2005. "Policy cycle: A model of post-Machiavellian policymaking?" Australian Journal of Public Administration 64 (3): 3-13.

Howlett, Michael. 2009. "Policy analytical capacity and evidence-based policy-making: Lessons from Canada." Canadian Public Administration 52 (2): 153-75.

--. 2011. Designing Public Policies: Principles and Instruments. New York: Routledge.

--. 2015. "Policy analytical capacity: The supply and demand for policy analysis in government." Policy and Society 34 (3-4): 173-82.

Howlett, Michael and Jonathan Craft. 2013. "Policy advisory systems and evidence-based policy: The location and content of evidentiary policy advice." In Evidence-Based Policy-Making in Canada: A Multidisciplinary Look at how Evidence and Knowledge Shape Canadian Public Policy, edited by Shaun P. Young. Oxford: Oxford University Press, pp. 27-44.

Howlett, Michael and Andrea R. Migone. 2014. "Making the invisible public service visible? Exploring data on policy and management consultancies in Canada." Canadian Public Administration 57 (2): 183-216.

Howlett, Michael, Andrea R. Migone and Seck L. Tan. 2014. "Duplication or complementarity? External policy consulting and its relationship to internal policy analysis in Canada." Canadian Journal of Political Science 47 (1): 113-34.

Howlett, Michael and Josh Newman. 2010. "Policy analysis and policy work in federal systems: Policy advice and its contribution to evidence-based policy-making in multi-level governance systems." Policy and Society 29 (2): 123-36.

Howlett, Michael and Adam Wellstead. 2011. "Policy analysts in the bureaucracy revisited: The nature of professional policy work in contemporary government." Politics and Policy 39 (4): 613-33.

Howlett, Michael, Adam Wellstead and Jonathan Craft, eds. 2017. Policy Work in Canada: Professional Practices and Analytical Capacities. Toronto: Toronto University Press.

Jennings, Edward T. and Julie L. Hall. 2012. "Evidence-based practice and the use of information in state agency decision making." Journal of Public Administration Research Theory 22 (2): 245-66.

John, Peter. 2013. "Political science, impact and evidence." Political Studies Review 11 (2): 168-73.

Knaggard, Asa. 2014. "What do policy-makers do with scientific uncertainty? The incremental character of Swedish climate change policy-making." Policy Studies 35 (1): 22-39.

Landry, Rejean, Moktar Lamari, Nabile Amara. 2003. "The extent and determinants of the utilization of university research in government agencies." Public Administration Review 63 (2): 192-205.

Lasswell, Harold. 1970. "The emerging conception of the policy sciences." Policy Sciences 1 (1): 3-14.

Lester, James P. 1993. "The utilization of policy analysis by state agency officials." Science Communication 14 (3): 267-90.

Lindquist, Evert. 1988. "What do decision models tell us about information use?" Knowledge in Society 1 (2): 86-111.

Mead, Lawrence M. 2015. "Only connect: Why government often ignores research." Policy Sciences 48 (2): 257-72.

Newman, Joshua. 2014. "Revisiting the 'two communities' metaphor of research utilisation." International Journal of Public Sector Management 27 (7): 614-27.

--. 2016. "Deconstructing the debate over evidence-based policy." Critical Policy Studies, 1-16.

Newman, Joshua, Adrian Cherney and Brian W. Head. 2017. "Policy capacity and evidence-based policy in the public service." Public Management Revieiv 19 (2): 157-74.

Newman, Joshua and Brian W. Head. 2015. "Beyond the two communities: A reply to Mead's 'why government often ignores research'." Policy Sciences 48 (3): 383-93.

Neylan, J. 2008. "Social policy and the authority of evidence." Australian Journal of Public Administration 67 (1): 12-19.

Parsons, Wayne. 2004. "Not just steering but weaving: Relevant knowledge and the craft of building policy capacity and coherence." Australian journal of Public Administration 63 (1): 43-57.

Prince, Michael J. 2007. "Soft craft, hard choices, altered context: Reflections on 25 years of policy advice in Canada." In Policy Analysis in Canada: The State of the Art, edited by Laurent Dobuzinskis, Michael Howlett, and David Laycock. Toronto: University of Toronto Press, pp. 95-106.

Ranga, Marina and HenryEtzkowitz. 2013. "Triple helix systems: An analytical framework for innovation policy and practice in the knowledge society." Industry and Higher Education 27 (4): 237-62.

Sanderson, Ian. 2006. "Complexity, 'practical rationality' and evidence-based policy making." Policy and Politics 34 (1): 115-32.

Savoie, Donald. 2015. "The Canadian public service: In search for a new equilibrium." In The International Handbook of Public Administration and Governance, edited by Andrew Massey and Karen K. Johnson Cheltenham: Edward Elgar, pp. 182-98.

Smith, Katherine. 2013. Beyond Evidence-Based Policy in Public Health: The Interplay of Ideas. Basingstoke: Palgrave Macmillan.

Van der Arend, Jenny. 2014. "Bridging the research/policy gap: Policy officials' perspectives on the barriers and facilitators to effective links between academic and policy worlds." Policy Studies 35 (6): 611-30.

Vesely, Arnost. 2017. "Policy advice as policy work: A conceptual framework for multi-level analysis." Policy Sciences 50 (1): 139-54.

Vesely, Arnost, Adam Wellstead and Evans, B. 2014. "Comparing sub-national policy workers in Canada and the Czech Republic: Who are they, what they do, and why it matters?" Policy and Society 33 (2): 103-15.

Watts, Rob. 2014. "Truth and politics: Thinking about evidence-based policy in the age of spin." Australian Journal of Public Administration 73 (1): 34-46.

Weiss, Carol H. 1979. "The many meanings of research utilization." Public Administration Revieiv 39 (5): 426-31.

--. 1980. "Knowledge creep and decision accretion." Science Communication 1 (3): 381-404.

Weller, Patrick. 2015. "Policy professionals in context: Advisors and ministers." In Policy Analysis in Australia, edited by Brian W. Head and Kate Crowley. Bristol: Policy Press, pp. 23-36.

Wesselink, Anna, Hal Colebatch and Warren Pearce. 2014. "Evidence and policy: Discourses, meanings and practices." Policy Sciences 47 (4): 339-44.

Wesselink, Anna and Andy Gouldson. 2014. "Pathways to impact in local government: The mini-Stern review as evidence in policy making in the Leeds City Region." Policy Sciences 47 (4): 403-24.

Wildavsky, Aaron B. 1979. Speaking Truth to Power: The art And Craft of Policy Analysis. Boston: Little, Brown and Company.

Young, Shaun P. ed. 2013. Evidence-Based Policy-Making in Canada: A Multidisciplinary Look at how Evidence and Knowledge Shape Canadian Public Policy. Oxford: Oxford University Press. Appendix Table A1. Research Engagement--Canadian Academics-University Survey Seldom Occasionally (not more (at least than twice every three Never per year) months) Receive an invitation from 53.7% 31.7% 12.5% members of the public service to serve on boards, panels, or committees Participate in meetings that 53.1% 32.6% 8.8% discussed academic research with members of the public service Attend workshop/training 63.4% 27.3% 5.9% session with members of the public service Receive a request for 59.6% 29.4% 7.1% research or research reports from members of the public service Provide informal advice to 50.6% 27.2% 14.2% members of the public service Participate in a network with 60.6% 21.2% 11.9% members of the public service Sit on a panel, roundtable, 59.8% 29.7% 5.0% etc. with members of the public service Second personnel to (or 90.2% 7.2% 1.7% from) a government agency or administrative unit Often Very often (at least (more than every twice per month) month) Receive an invitation from 1.2% 0.8% members of the public service to serve on boards, panels, or committees Participate in meetings that 4.6% 0.8% discussed academic research with members of the public service Attend workshop/training 2.5% 0.8% session with members of the public service Receive a request for 2.9% 0.8% research or research reports from members of the public service Provide informal advice to 5.4% 2.5% members of the public service Participate in a network with 3.8% 2.5% members of the public service Sit on a panel, roundtable, 4.2% 1.3% etc. with members of the public service Second personnel to (or 0.4% 0.4% from) a government agency or administrative unit Table A2. Research Engagement Canadian Academics--Public Policy, Public Administration and Political Science Survey Never Seldom Occasionally (not more (at least than twice every three per year) months) Receive an invitation from 57.6% 27.60% 10.8% members of the public service to serve on boards, panels, or committees Participate in meetings that 40.2% 38.4% 14.5% discussed academic research with members of the public service Attend workshop/training 52.2% 31.8% 13.4% session with members of the public service Receive a request for 51.2% 31.4% 14.8% research or research reports from members of the public service Provide informal advice to 38.8% 31.8% 16.9% members of the public service Participate in a network with 42.8% 29.9% 15.0% members of the public service Sit on a panel, roundtable, 43.7% 38.5% etc. with members of the 12.0% public service Second personnel to (or 88.8% 8.2% 1.8% from) a government agency or administrative unit Often (at least Very often every month) (more than twice per month) Receive an invitation from 2.9% 1.2% members of the public service to serve on boards, panels, or committees Participate in meetings that 4.3% 2.6% discussed academic research with members of the public service Attend workshop/training 1.7% 0.9% session with members of the public service Receive a request for 1.7% 0.9% research or research reports from members of the public service Provide informal advice to 6.7% 5.8% members of the public service Participate in a network with 7.6% 4.7% members of the public service Sit on a panel, roundtable, etc. with members of the 4.7% 1.2% public service Second personnel to (or 0.3% 0.9% from) a government agency or administrative unit Table A3. Research Engagement--Canadian Public Service Executives Never Seldom Occasionally (not more (at least than twice every three per year) months) Invite academics to 13.1% 25.6% 35.7% serve on boards, panels, or committees Participate in meetings 6.5% 30.4% 28.6% that discussed academic research Attend workshop/training 6.0% 28.7% 38.9% session led by academics Commission academic 16.2% 41.3% 26.9% research reports from public administration schools/programs Seek informal advice 6.5% 29.8% 30.4% from members of these institutions Sit on panel, roundtable, 11.4% 32.9% 28.7% etc. with members of research institutions Second personnel to (or 17.9% 31.5% 29.8% from) an academic or research institutions Second personnel to (or 39.3% 34.5% 18.5% from an academic or research institution Often Very often (at least (more than every month) twice per month) Invite academics to 20.2% 5.4% serve on boards, panels, or committees Participate in meetings 25.6% 8.9% that discussed academic research Attend workshop/training 17.4% 9.0% session led by academics Commission academic 10.8% 4.8% research reports from public administration schools/programs Seek informal advice 23.2% 10.1% from members of these institutions Sit on panel, roundtable, 15.6% 11.4% etc. with members of research institutions Second personnel to (or 14.3% 6.5% from) an academic or research institutions Second personnel to (or 4.2% 3.6% from an academic or research institution

Andrea Migone is Director of Research and Outreach, Institute of Public Administration of Canada, Toronto, Ontario. Kathy Brock is Professor, School of Policy Studies and Department of Political Studies (cross-appointed), Queen's University, Kingston, Ontario. Table 1. Response Rates for the Surveys--Canada Surveys Usable Response mailed answers rate Public Service Executives 471 190 40.3% Canada-wide Survey 1,654 374 22.6% University Survey 1,752 302 17.2% Table 2. Respondents by Department--University--Select Departments Department Percentage Response count English 3.5% 10 Kinesiology 3.5% 10 Economics 3.8% 11 Biology 4.9% 14 Physics and Astronomy 5.2% 15 Biomedical and molecular sciences 5.9% 17 Psychology 5.9% 17 Business 6.3% 18 Medical or health specialization 17.5% 50 Table 3. Key Roles--Differences Between Academics and Public Servants University PP & PA Public survey survey service executives Participate in meetings that 5.4% 6.9% 34.5% discuss academic research Provide/Seek informal advice 7.9% 12.5% 33.3% Participate in a network 6.3% 12.3% 27.0% Provide/Commission academic 3.7% 2.6% 15.6% research reports Receive/Extend an invitation 2.0% 4.1% 25.6% from members of the public service to serve on boards, panels, or committees Percentages are the sum of the Often and Very Often responses. Table 4. Engagement with the Non-Profit and Private Sector Seldom Occasionally (not more (at least than twice every three Never per year) months) Canadian public service executives Engage with 5.4% 16.7% 38.1% non-academic actors like social enterprises and not-profit? Engage with the private 2.4% 9.8% 14.0% sector? Canadian PP & PA academics Engage with 33.8% 29.7% 19.1% non-academic actors like social enterprises and not-profit? Engage with the private 56.9% 27.7% 11.2% sector? University academics Engage with 55.0% 20.4% 10.8% non-academic actors like social enterprises and not-profit? Engage with the private 51.9% 18.4% 17.1% sector? Often Very often (at least (more than every twice per month) month) Canadian public service executives Engage with 25.0% 14.9% non-academic actors like social enterprises and not-profit? Engage with the private 34.1% 39.6% sector? Canadian PP & PA academics Engage with 10.3% 7.1% non-academic actors like social enterprises and not-profit? Engage with the private 3.2% 0.9% sector? University academics Engage with 8.3% 5.4% non-academic actors like social enterprises and not-profit? Engage with the private 5.4% 7.1% sector? Table 5. Research Engagement by Partner Type Seldom (not Occasionally more than (at least twice every Never per year) three months) Canadian PP & PA academics Did you partner with a 65.8% 18.1% 8.5% member of the public service in undertaking academic research that included contact with members of government or the public administration? Did you undertake 46.9% 27.7% 14.0% academic research that included either surveying or interviewing members of government or the public administration? University survey Did you partner with a 70.3% 15.5% 7.9% member of the public service in undertaking academic research that included contact with members of government or the public administration? Did you undertake 78.7% 10.4% 6.7% academic research that included either surveying or interviewing members of government or the public administration? Canadian public service executives Participate or engage in 10.7% 33.3% 37.5% academic research, excluding participation in this research survey? Often (at Very often least (more than every twice month) per month) Canadian PP & PA academics Did you partner with a 5.6% 2.0% member of the public service in undertaking academic research that included contact with members of government or the public administration? Did you undertake 7.0% 4.4% academic research that included either surveying or interviewing members of government or the public administration? University survey Did you partner with a 2.9% 3.3% member of the public service in undertaking academic research that included contact with members of government or the public administration? Did you undertake 1.7% 2.5% academic research that included either surveying or interviewing members of government or the public administration? Canadian public service executives Participate or engage in 13.1% 5.4% academic research, excluding participation in this research survey? Table 6. Level of Satisfaction Engaging the Public Service in Academic Research PA & PP academics University Very dissatisfied 9.0% (27) 8.0% (11) Dissatisfied 18.0% (54) 13.8% (19) Neither satisfied nor dissatisfied 38.3% (115) 35.5% (49) Satisfied 28.0% (84) 30.4% (42) Very satisfied 6.7% (20) 12.3% (17) Table 7. In the Past 5 Years the Level of Engagement with the Public Service Became: PA & PP academics University Much worse 19.0% (56) 11.2% (15) Worse 27.9% (82) 22.4% (30) Remained the same 38.4% (113) 48.5% (65) Better 11.9% (35) 14.9% (20) Much better 2.7% (8) 3.0% (4) Table 8. Instruments Used by Academics in their Research Engagement Attempting research PA & PP academics University Survey instrument 19.8% (50) 12.4% (25) Interview 72.6% (183) 15.4% (31) Focus group 11.1% (28) 8.5% (17) Case study 29.8% (75) 9.5% (19) Informal discussion 63.5% (160) 16.4% (33) Other qualitative 6.0% (15) 2.0% (4) instruments Other quantitative 6.0% (15) 1.5% (3) instruments Experimental research N/A 8.5% (17) Laboratory research N/A 8.0% (16) Field research N/A 9.0% (18) Clinical research N/A 2.5% (5) Non-clinical research N/A 1.0% (2) Translational research N/A 5.5% (11) Successful research PA & PP academics University Survey instrument 17.2% (42) 13.0% (25) Interview 71.7% (175) 15.1% (29) Focus group 10.2% (25) 8.9% (17) Case study 27.5% (67) 10.9% (21) Informal discussion 61.9% (151) 15.1% (29) Other qualitative 5.3% (13) 2.6% (5) instruments Other quantitative 6.0% (15) 2.6% (5) instruments Experimental research N/A 8.3% (16) Laboratory research N/A 7.3% (14) Field research N/A 8.3% (16) Clinical research N/A 2.1% (4) Non-clinical research N/A 1.0% (2) Translational research N/A 4.7% (9) Table 9a. Over the Past Two Years Have Often Have You--Federal Government Never Seldom (not more than twice per year) Bid on a competitive University 80.3% 16.5% contract (184) (38) PA & PP 81.5% 15.8% (268) (52) Been selected for a University 82.5% 16.2% sole source contact (189) (37) PA & PP 80.8% 16.7% (266) (55) Provided policy University 85.2% 13.0% analysis (190) (29) PA & PP 77.7% 19.6% (254) (64) Occasionally Often (at least (at least every every three month) months) Bid on a competitive 2.6% -- contract (6) 2.4% 0.3% (8) (1) Been selected for a sol - 0.9% sole source contact (2) 2.1% 0.3% (7) (1) Provided policy 0.4% -- analysis (1) 2.7% -- (9) Very often (more than twice per month) Bid on a competitive 0.4% contract 0) -- Been selected for a sol 0.4% sole source contact (1) Provided policy 1.3% analysis (3) -- Table 9b. Over the Past Two Years Have Often Have You--Provincial/Territorial Government Never Seldom (not more than twice per year) Bid on a competitive University 82.0% 14.9% contract (187) (34) PA & PP 84.4% 12.9% (282) (43) Been selected for a University 89.9% 9.2% source contact (205) (21) PA & PP 85.0% 13.2% (277) (43) Provided policy University 87.8% 10.4% analysis (195) (23) PA & PP 84.5% 13.7% (278) (45) Occasionally Often (at least (at least every three every month) months) Bid on a competitive 2.6% -- contract (6) 2.4% 0.3% (8) (1) Been selected for a sol 0.9% -- source contact (2) 1.8% -- (6) Provided policy 0.4% 1.3% analysis (1) (3) 1.2% 0.6% (4) (2) Very often (more than twice per month) Bid on a competitive 0.4% contract (1) -- Been selected for a sol -- source contact -- Provided policy -- analysis -- Table 9c. Over the Past Two Years Have Often Have You--Local Government Never Seldom (not more than twice per year) Bid on a competitive University 97.0% 2.2% contract (222) (5) PA & PP 93.0% 7.0% (305) (23) Been selected for University 97.8% 2.2% a sole source (222) (5) contract PA & PP 92.6% 7.10% (300) (23) Provided policy University 96.9% 3.1% analysis (216) (7) PA & PP 91.1% 8.6% (296) (28) Occasionally Often (at least (at least every three every month) months) Bid on a competitive 0.4% -- contract (1) -- -- Been selected for -- -- a sole source contract 0.3% -- (1) Provided policy -- -- analysis 0.3% -- (1) Very often (more than twice per month) Bid on a competitive 0.4% contract (1) -- Been selected for -- a sole source contract -- Provided policy -- analysis -- Table 9d. Over the Past Two Years Have Often Have You--Aboriginal Government Never Seldom (not more than twice per year) Bid on a competitive University 97.8% 1.7% contract (223) (4) PA & PP 96.4% 3.0% (318) (10) Been selected for a University 99.6% 0.4% sole source contract (226) (1) PA & PP 95.1% 3.7% (309) (12) Provided policy University 99.6% -- analysis (221) PA & PP 95.1% 4.3% (312) (14) Occasionally Often (at least (at least every three every month) months) Bid on a competitive 0.4% -- contract (1) 0.6% -- (2) Been selected for a -- -- sole source contract 0.9% 0.3% -- Provided policy -- 0.4% analysis (1) 0.3% 0.3% (1) (1) Very often (more than twice per month) Bid on a competitive -- contract -- Been selected for a -- sole source contract -- Provided policy analysis -- Table 10. Federal Contracts Allocated to Universities by GSIN Code--January 2009 to June 2016 GSIN Description Universities AR910400 Astronautics (R&D) $71,192,086.00 AD917700 Military (R&D) $27,771,042.00 AN117500 Medical (R&D) $13,388,407.00 V502CA Accommodation and/or $12,157,893.00 Food Support Services AN417668 Vaccines (R&D) $11,250,000.00 AT919500 Transportation (R&D) $10,134,414.00 V502C Accommodation and Food $9,119,265.00 Services, N.E.S. R199H Consulting Services, $8,285,308.00 Change Management/ Organizational Development G009G Health and Welfare Services $7,651,295.00 AR210480 Spacecraft Instrumentation $7,116,562.00 Development (R&D) AJ412000 Communications (R&D) $5,478,606.00 G009A Dental Services $5,198,400.00 AJ716500 Humanities and Social $1,111,566.00 Sciences (R&D) R123AQ Evaluation & Performance $531,250.00 Measurement Services R199B Miscellaneous Business $450,979.00 Services AZ110210 Research Project Management $314,977.00 Services (R&D) R019U Professional Services/ $282,500.00 Program Research Analysis AE913500 Economics (R&D) $226.000.00 R019E Business Services $122,000.00 T001B $15,000.00 T002AK Public Opinion Research-- $9,605.00 Qualitative Research and Writing Total Expenditures $271,923,428.00 Social science and business $11,349,185.00 subset (Italicized entries) GSIN Total % AR910400 $1,296,381,910.00 5.5% AD917700 $819,044,709.00 3.4% AN117500 $56,178,960.00 23.8% V502CA $60,279,912.00 20.2% AN417668 $11,576,357.00 97.2% AT919500 $38,951,089.00 26.0% V502C $74,716,217.00 12.2% R199H $1,321,938,240.00 0.6% G009G $148,695,393.00 5.1% AR210480 $62,892,498.00 11.3% AJ412000 $25,429,437.00 21.5% G009A $14,013,721.00 37.1% AJ716500 $15,397,906.00 7.2% R123AQ $8,624,187.00 6.2% R199B $67,646,034.00 0.67% AZ110210 $4,288,223.00 7.3% R019U $22,522,563.00 1.3% AE913500 $6,645,976.00 3.4% R019E $399,088,130.00 0.03% T001B $10,874,532.00 0.1% T002AK $1,511,559.00 0.6% $20,718,252,439.00 1.3% $1,858,537,350.00 0.6% Source: Government of Canada, Contracting Open D.