Does public reporting measure up? Federalism, accountability and child-care policy in Canada.

Anderson, Lynell ; Findlay, Tammy

Governments in Canada have shown growing interest in accountability and citizen engagement, including within federalism. The 1999 accountability provisions of the Social Union Framework Agreement (SUFA) emphasize "collaborative" federalism, public reporting and the involvement of third parties in the social policy process (Boismenu and Graefe 2004). Since that time, public reporting has become the preferred mechanism in a range of policy areas, including early learning and child-care. This new approach departs significantly from federal/provincial/territorial (FPT) accountability mechanisms, such as legislated standards, audited information and reporting to legislatures, that were applied in the past to social policy. While we have serious questions about whether public reporting can replace traditional approaches in Canadian federalism, in this article we assess its effectiveness as an accountability measure. The article is based on our experience with the Child Care Advocacy Association of Canada's community capacity-building project that explores the relationship between public policy, funding and accountability mechanisms under the FPT agreements related to child-care. (1)

We argue that, in its current form, public reporting has not lived up to its promise of accountability to citizens. This evaluation is based on the standards that governments have set for themselves under the FPT agreements, as well as guidelines by the Public Sector Accounting Board, an independent body that develops accounting standards through consultation with governments. We begin by outlining the case study of early learning and child-care. Next, we elaborate on our research methodology and the criteria on which we base our analysis. We then move on to the review and critique of the process and substance of child-care reporting and the prospects for citizen engagement. Finally, we offer several policy recommendations and draw some conclusions about the future of public reporting.

Public reporting, accountability and child-care

Those concerned with Canadian federalism have long been interested in accountability between levels of government, and a substantial shift in the approach to social policy and FPT relations in Canada has renewed attention. This shift has involved a movement away from a regime based on conditionality and cost-sharing towards a new paradigm, initially spelled out in the SUFA.

Barbara Cameron has traced changes in intergovernmental relations over two different "accountability regimes:" the Canada Assistance Plan Act (R.S., c. C-1) and the Social Union Framework Agreement (2007:162). In the former regime, the accountability mechanism revolved around federal conditions placed on financial transfers to the provinces and cost-sharing. This regime included legislated standards, such as those in the Canada Health Act (R.S.C. 1985, c. C-6), bolstered by the federal spending power (Day and Brodsky 2007). Accountability for federal funds also required approval by elected legislatures and reporting to Parliament (Cameron 2007).

The Canada Assistance Plan regime certainly was not perfect. For instance, it problematically defined child-care in terms of welfare. It has also been criticized for its heavy-handed federal oversight of the provinces (Boismenu and Graefe 2004; Cameron 2007). Despite these weaknesses, federal conditions were attached to fiscal transfers (Friendly and White 2008), providing guarantees of social citizenship rights, and the procedural right to appeal (Cameron 2007), which have since been greatly weakened. By the mid-1970s, there was a shift in fiscal federalism and an accompanying loosening of accountability for federal transfers (Boismenu and Graefe 2004). New accountability techniques, especially public reporting, have gained currency in intergovernmental relations (Phillips 2001). Performance reporting has been developing in health care in Canada, as well as internationally, since 2000 (Morris and Zelmer 2005). According to Gerard Boismenu and Peter Graefe, with the decline of cost-sharing and conditionality, "[t]he vocabulary now centres on 'reporting' and having provinces submit reports on where money is spent, and on mutually agreed upon performance measures. While a much blunter tool than cost-sharing, it nevertheless provides the federal government with some leverage.... If the federal government can no longer hold the provinces to account, perhaps the public will do so for them if given the 'right' information" (2004: 77-78). According to Susan Phillips, citizens are emerging as the "third force of federalism" (2001), and Paul Kershaw notes that citizens are now expected "to fill the void by serving as policy watchdogs who will hold all levels of government to account" (2006: 200). Within this context, public reporting has become the central accountability mechanism for child-care policy and funding in Canada.

Over the last several years, governments have acknowledged the importance of public investment in services for early childhood. Since 2001-02, three agreements between FPT governments resulting in new federal transfers to provinces and territories have advanced such services and supports. In 2000, the Early Childhood Development Agreement was introduced to channel investments into four areas: healthy pregnancy, birth and infancy; parenting and family supports; early childhood development, learning and care; and community supports. In 2003, the Multilateral Framework Agreement on Early Learning and Child Care emerged to specifically direct federal transfers to improve access to affordable, quality, provincially and territorially regulated early learning and child-care programs and services. Finally, in 2005, the Bilateral Agreements-in-Principle on Early Learning and Child Care committed $5 billion over five years towards a national child-care system, working in cooperation with provinces and territories and building on the Multilateral Framework Agreement. The Bilateral Agreements were cancelled in 2006.

These agreements represent political rather than legal commitments. Federal transfers under both the Early Childhood Development Agreement and the Multilateral Framework Agreement (and, more recently, the $250-million annual federal transfer for child-care spaces confirmed in the 2007 federal budget) currently flow annually to provinces and territories through the Canada Health and Social Transfer, which does not have program-specific spending conditions attached to it. Therefore, in a range of circles emanating from the community (2) and from the Office of the Auditor General of Canada, questions have been raised about accountability for federal transfers. Without legal agreements or spending conditions in place, these agreements involve commitments by FPT governments to produce annual public reports that describe each government's priorities, investments and progress, relying on citizens reviewing these public reports and providing feedback to governments.

Such reporting is not a new concept for governments. What is new is the suggestion that public reporting, on its own, provides sufficient accountability. If the public is expected to utilize the reports to assess progress and provide feedback to government, at a minimum, the reporting process must be effective. Therefore, our aim here is to determine if it is so.

Methodology: making the connections

"Child Care Policy: Making the Connections" (see http://www.ccaac.ca/ mtc/en/about.html) was a project of the Child Care Advocacy Association of Canada, a pan-Canadian, non-profit, membership-based organization dedicated to promoting quality, inclusive, publicly funded, non-profit child-care that is accessible to all (Anderson and Findlay 2007). The project provided tools and resources to support analysis of the implications of childcare policy and investments under FPT agreements. The FPT public reporting formed the basis for the work, which focused on the connections between the public policy, funding and accountability mechanisms that indicate improved and expanded access to quality, affordable and inclusive child-care services. Beginning in November 2004, the project's mandate covered child-care policy and funding developments (outside of Quebec) under existing and any future FPT agreements. It was a three-year project funded by the federal Social Development Partnerships Program.

In addition to individuals, groups and governments interested in childcare services, the project was supported by two advisory groups with regional and sectoral community representation and academic expertise. The project worked with parents, caregivers, researchers, and other interested community members, based on the principle that everyone has a role to play in developing an effective early learning and child-care system.

The early stages of the work involved an initial assessment of public reports and sharing in meetings with various governments and communities some of the weaknesses observed. In addition to a series of fact sheets developed to increase public awareness and understanding of the FPT agreements and federal transfers, the project developed a "Child Care Planning Checklist" to assist communities and governments to produce a comprehensive child-care plan and the "Child Care System Implementation Model" to facilitate dialogue about the policy and funding approaches that were most likely to improve access to quality, affordable child-care services. The final step was the report, released in November 2007, containing detailed analyses of public reporting related to child-care services.

The "Making the Connections" project analysed five years of reporting for thirteen jurisdictions. This led to the creation of a common framework for analysis of public reporting to confirm if the FPT agreement commitments were being met and whether child-care services were improving as a result. A simplified investment chart was developed to track annual expenditures (see Table 1). This common framework for financial analysis was first drafted in the fall of 2005, using the federal government's annual reporting as an example, and has been used in various presentations, discussions and meetings with communities and governments since that time. It was a much-needed tool for community capacity-building, because, as expanded on later, the reports were often unclear and frequently not comparable.

All figures in the investment charts were taken directly from the public reports. They were totalled and summarized so the public can track the changes in investments from the baseline year up to the present (2000-01 to 2005-06). Blank areas in the chart indicate that the information was not available, or was not clear, consistent and/or comparable. The total investment in early childhood development programs, including child-care services, was broken down into two components--the federal transfers and the resulting provincial contribution--to help the public understand which investments were funded through federal transfers and/or provincial contributions. The federal transfers not yet invested in early childhood development, including child-care services, were also identified.

There was no follow-up with individual governments to discuss reporting gaps, because that would essentially be "private" information and the goal of this project was to analyse public reporting. However, in August 2007, governments did receive the standardized template used to summarize the financial analysis. And, in order to support ongoing dialogue between communities and governments about public reporting, the director of the Child Care Advocacy Association of Canada and FPT directors in each jurisdiction were provided with the detailed financial analyses. All of the analyses were submitted to an external reviewer, Gordon Gray, CGA.

Measuring the performance of public reporting

Public reporting is about involving citizens in the process of performance measurement. But in order to assess if communities can make meaningful use of public reporting, we need to know the extent to which that reporting meets the needs of citizens in holding their governments to account. How do we measure the performance of public reporting?

The accountability principle underlying public reporting is straightforward: governments must demonstrate to citizens that they did what they said they were going to do. Therefore, our first criterion for evaluating public reporting was to measure the reporting against governments' own standards. Governments made commitments in several documents. As part of the Early Childhood Development Agreement, governments signed on to the Shared Framework for Reporting on Progress in Improving and Expanding Early Childhood Development (ECD) Programs and Services. Governments made accountability commitments based on shared principles, namely, a respect for the diversity of jurisdictions, a pledge to improve reporting over time, and an agreement to consult with third parties on indicators and progress "as appropriate." In addition, they agreed that in their annual reporting, they would acknowledge the federal contribution; report on changes in expenditures from the previous year; report on investments in the four areas of early childhood development; describe the programs, including objectives, target population, program description, department(s) responsible and delivery agent(s); report on such program indicators as expenditures, availability, accessibility, affordability and quality (see http://www.ecd-elcc.ca/eng/ecd/ecd_sharedframe.shtml).

Furthermore, under the Multilateral Framework Agreement, governments made specific performance and reporting commitments from 2002-03 onward related to child-care. They agreed to provide public reports with descriptive and expenditure information on all early learning and child-care programs and services, as well as indicators of availability, affordability and quality. Since our focus is on child-care services, we paid particular attention to the Multilateral commitments and related reporting requirements. (3)

Because of the vagueness surrounding the FPT agreements' call for "clear public reporting," we also turned to other public reporting standards that have been developing over the last ten years in Canada and that are generally recognized by governments. The most current guidelines by the Public Sector Accounting Board (PSAB) have informed our analysis.

The PSAB recommendations emphasize that clear public reporting is easily accessible and identifiable and contains information that is reliable, valid, relevant, timely, fair, comparable, consistent and understandable. It should focus on critical performance measures and demonstrate a link between planned and actual results and between financial and non-financial information (Public Sector Accounting Board 2007). Together, the FPT agreements and the PSAB guidelines provide the foundation for determining whether the progress on child-care services can be clearly tracked using public reports. (4) Under the FPT agreements governments made commitments to improve and expand early childhood development programs for Canadian families, including child-care services and to provide clear reporting so that the public can track progress towards this goal. Therefore, our assessment of public reporting sought to answer two basic questions: 1) Can progress be clearly tracked through the public reporting? and 2) Have there been improvements in early childhood development, including child-care services? In other words, our analysis considered both the process and the substance of intergovernmental relations in child-care policy.

Public reporting on child-care: the process

In this section, we reflect on the procedural aspects of public reporting, and in the next section we will discuss the substantive outcomes.

Public reporting can be a vital tool of accountability. However, our review of public reporting under the FPT agreements indicates several challenges for the public in tracking progress on child-care. Our observations regarding government reporting, as of 28 August 2007, are summarized into two broad areas: 1) ease of access; and 2) clarity, comparability and comprehensiveness of reporting.

In terms of access, the annual reporting relies almost exclusively on the Internet, even though studies have shown wide disparities in access and skills across Canada based on region and aboriginal status. Even if one has regular Internet access, reports and information are not always easy to find or access. The links from the main federal Early Childhood Development/ Early Learning and Child Care web site to several provincial/territorial sites are not kept up to date. The links from the federal site do not always lead directly to the provincial/territorial reports--they may only lead to the relevant ministry, so additional searching is often needed. Some reports have been removed from the site, in some cases temporarily, but in others, permanently. The links to some reports have been removed. Several reports are available only in English and do not seem to be available in accessible formats for people with visual disabilities.

Once reports are found, many are quite long, and PDF files take time to download, especially those with a lot of pictures. In community meetings, rural participants expressed concerns about accessing reports with dial-up service. Such reports can also be costly for parents and caregivers to print out. In addition, community members frequently told us that they simply do not have the time to read reports that are more than fifty pages. Lengthy, detailed reports are important for some stakeholders, such as researchers. Others may be looking for a brief summary of the key policy and investment changes and the resulting achievements. In their study of health-care reporting, Kathleen Morris and Jennifer Zelmer also wonder "whether a single report can meet the needs of all audiences, or whether more targeted reporting is required" (2005: 16).

Furthermore, many reports are not released on a timely basis and are not dated, or not clearly dated. Under the Early Childhood Development Agreement, reports are to be published in September each year and under the Multilateral Framework Agreement, in November. Few governments have consistently achieved this goal. For instance, the federal government's 2004--05 and 2005-06 combined report was released in August 2007. (5) Late reporting is common and is a significant concern because it affects usability and relevance. As the CCAF-FCVI (the Canadian Comprehensive Auditing Foundation), a non-profit auditing foundation interested in public-sector performance, points out, late reporting "undermines Canada's system of government, hurts public trust, and limits accountability" (2006: 38). Communities want to know about and discuss child-care services in the here and now. There was little interest in older reports and disappointment that up-to-date information was not available.

In addition to issues of access, other central standards as outlined by the PSAB relate to clarity, comprehensiveness and comparability. The PSAB notes that, in effective public reporting, information is complete, financial and non-financial information are clearly linked, trends are visible, and information is unbiased (2007). The reporting related to the Early Childhood Development and Multilateral Framework agreements does not always meet these standards. As a result, project staff spent over thirty hours analysing some individual governments' reports, trying to extract both financial and non-financial information that was not clearly and consistently presented and summarized in a way that was comparable from year to year. This seems to be a common issue with public performance reporting. The CCAF-FCVI's study found that legislators, the public, non-governmental organizations, and the media have made similar observations about the clarity and usability of public performance reports (2006). These concerns were also reflected in feedback at community meetings from parents, child-care workers and academics, who view the FPT reports as too long, confusing and difficult to understand. One of this project's goals was to help communities to understand public reporting, but our experience indicates that not all reports are understandable.

Some provinces do not provide complete baseline information for early childhood development and/or child-care expenditure, or do so inconsistently. Without a baseline, overall progress and use of federal transfers cannot be tracked and confirmed. While federal transfers are generally acknowledged in the report text, they are often not clearly reflected in the financial reporting section. Some reports are downright confusing, with a mismatch between the information in the text of the report and in the financial section of the same report, or between information from previous years. Comparable information is not always provided, so the investments may be difficult to track from year to year.

Also, reporting templates vary considerably across governments. The agreements allow for diverse approaches, supporting provincial and territorial autonomy, but fundamental differences across reporting means it is much more difficult for the public to track the progress on child-care investments in some jurisdictions than in others, resulting in a lack of procedural equality across the country. Some jurisdictions (for example, the federal government) do not consistently total the financial information, so the public is unable to easily see and compare trends in total investments.

The FPT agreement requirements themselves allow for unclear reporting. Both the Early Childhood Development and Multilateral Framework agreements permit governments to report only on their chosen "priorities." Thus, some reporting appears to be skewed in favour of good news, providing no explanation of apparent funding and/or service reductions. The CCAF-FCVI findings also reveal that legislators, citizens, non-governmental organizations, and the media believe that public performance reports read like public relations tools and lack credibility. Some legislators commented that reports put information "in the 'best light' and tend to 'smooth out the bumps' too much" and give citizens the impression that they are "being spun all the time." Non-governmental organizations and the media were also concerned about "spin" (CCAF-FCVI 2006: 17, 32, 38, 42). This approach is at odds with community expectations and the PSAB's guidelines on unbiased reporting, and, when expenditure reductions are not explained, the credibility of public performance reporting--already a widely recognized concern--is further diminished.

The federal government's 2004-05 and 2005-06 combined report provides an interesting example. On the one hand, its report clearly and comparably includes all programs from previous years, even those that have been discontinued, and generally includes an explanation for any expenditure reductions. On the other hand, the report does not include any information on the now-terminated Bilateral Agreements, which were introduced by a previous government and transferred $700 million to provinces and territories in 2004-05 and 2005-06 combined. This omission appears to place partisan politics ahead of public accountability.

At the same time, some reports have already moved beyond the specific requirements in the Early Childhood Development and Multilateral Framework agreements, providing a table(s) showing the baseline and subsequent annual investments by early childhood development program. The federal government's most recent report suggests movement towards a PSAB recommendation to consider an external audit opinion on the public performance report. We support an external audit approach--a topic to which we will return in our recommendations below.

To this point, we have provided some observations on the existing public reports. We conclude this section by assessing the extent to which the specific requirements of the Early Childhood Development and Multilateral Framework agreements have been met.

Under the Multilateral, governments made specific performance and reporting commitments from 2002-03 onwards related to early learning and child-care services. For example, governments agreed "to further invest in provincially/territorially regulated early learning and child care programs for children under six." They agreed to "publicly recognize and explain the respective roles and contributions of governments to this initiative" and to use the federal transfers to improve "access to affordable, quality early learning and child care programs and services," providing evidence of progress achieved by publicly reporting: descriptive and expenditure information on all early learning and child-care programs and services; indicators of availability, such as number of spaces in early learning and childcare settings broken down by age of children and type of setting; indicators of affordability, such as number of children receiving subsidies, income and social eligibility for fee subsidies, and maximum subsidy by age of child; and indicators of quality, such as training requirements, child/caregiver ratios and group size, where available (see the "Multilateral Framework Agreement" at http://www.ecd-elcc.ca/eng/ elcc/elcc_multiframe.shtml#ftn1).

There are some basic public expectations of timely, clear, useful and accurate information that logically follow from these government commitments. Unfortunately, in many ways, these expectations are inadequately met. Public reports for the years up to and including 2005-06 should have been available by November 2006. Of the thirteen jurisdictions we reviewed in the summer of 2007, eight were missing reports for one or more years, so the public cannot track all of the federal transfers and investments in child-care services through 2005-06. (6) Of the five remaining jurisdictions with public reporting available through 2005-06, three have information gaps. The remaining jurisdictions provided information that allows the public to track total investments in child-care services for that year. However, these jurisdictions did not meet the public reporting requirements in other ways, including publicizing plans for using unspent funds.

With regard to reporting on indicators of availability, affordability and quality, information is often incomplete. There was wide variation in the level of detail and comprehensiveness in all three reporting categories, but it seems that public reporting on child-care programs and services is not unique in this way. Reporting in health care has been similarly criticized for its lack of detail, clarity, accessibility, applicability, relevance and comparability (Maxwell, MacKinnon, and Watling 2007: 5; Morris and Zelmer 2005: 11).

Public reporting could help to address public cynicism about FPT relations by bringing greater transparency to public policy like child-care. However, given the current state of reporting, this seems unlikely. Instead, what we found in our work with communities was a lack of interest and trust in public reporting, reinforced by the problems in access and credibility noted above.

Public reporting on child-care: the substance

Evaluating the process of public reporting is one objective, but equally relevant for our analysis are the actual results described in the reports. What do these reports tell us about real progress in child-care policy in Canada?

Our review of the FPT reports found some improvement in income supports, such as maternity and parental benefits federally and targeted programs for lower- and modest-income families in several provinces. In line with critics from the social policy community, the reports also show the proliferation of "boutique" programs across the country emerging out of the Early Childhood Development Agreement, such as programs that promote healthy pregnancy and early childhood, support families in parenting, strengthen community supports, and advance early learning. Wendy McKeen has found such targeting of "at-risk" or vulnerable groups under the National Children's Agenda problematic, and it is clear that areas of early childhood development other than child-care have been priority areas for investments (2007). Several new universal programs were created, notably in the areas of early literacy and support for families in their parenting roles. For many other pre-existing programs, the reports describe expanded access for targeted groups, such as children with disabilities, lower-income families, and aboriginal communities.

Notably, affordability is rarely mentioned in relation to early childhood development programs other than child-care, where reports indicate that federal transfers have been used to introduce or expand these programs with minimal or no user-fees. These programs, in contrast with child-care, are substantially publicly funded. With child-care services, the public reports show that federal transfers have generally been focused on a much narrower range of policies that aim to improve the quality of existing child-care spaces and, to a limited extent, affordability and expanded access through new spaces. However, since no jurisdiction has complete and up-to-date reporting of both financial and non-financial information, we were unable to confirm overall progress on improved access to quality, affordable childcare services through 2005-06. Therefore, our analysis is limited to evidence suggesting a trend towards improvements.

Policy and funding to improve the quality of child-care has been a primary focus of investment in most jurisdictions. Several reports describe recruitment and retention of qualified staff as a critical issue. The child-care community echoes these concerns, and research consistently shows that staff compensation is a key indicator of quality. Yet most reports do not include wages as one of the non-financial indicators, even though several governments have introduced programs that enhance wages. Some provinces report on other quality indicators for certain years, such as increases in trained child-care staff and improved quality measurement scores. This reporting meets or exceeds the requirements of the Multilateral Framework Agreement, which only requires training, certain child/caregiver ratios and group size data "where available," but we are limited to confirming evidence of partial progress, or trends, due to a lack of comparable information.

There is evidence of overall growth in the number of regulated spaces in several jurisdictions. Some reports also indicate heightened attention to the inclusion of children with disabilities. Information on the actual number of existing spaces is necessary to track progress, but what is more important, in terms of the number of spaces and population figures for children, is the percentage of children with access to a regulated child-care space. This is particularly important given that less than twenty per cent of children in Canada (outside of Quebec and the Yukon) currently have access to regulated child-care services. Only Manitoba publicly reported on both of these indicators for three or more years.

Affordability of child-care services is also widely recognized as a critical issue because, unlike other early childhood development programs, child-care services are primarily funded through parent fees. Typically, subsidies are provided for lower- and modest-income families. The Early Childhood Development and Multilateral Framework agreements identify changes in both parent fees and subsidies as indicators of affordability; however, most provinces do not report on parent fees (nor are they required to under the Multilateral). Some jurisdictions describe improvements to their subsidy infrastructure (eligibility levels, etc.), although a few provinces show reduced investments in subsidies in some years, with no explanation for the reduction. And, without public reporting on fees, we cannot confirm that affordability overall has improved. Increased subsidies do not necessarily advance the policy goal of affordability. In fact, communities know from experience that they can often do the opposite, by driving up fees for families.

Overall, improvements cannot be determined due to missing information, but there is evidence of improvement trends in some quality and availability indicators. What is especially clear is that dedicated federal transfers for child-care services were central to any progress trend. Under the Early Childhood Development Agreement, provinces and territories were not required to invest specifically in child-care services or to include child-care in their baseline and subsequent reporting on early childhood development, and some did not. Of the seven jurisdictions for which we can trace annual investment in total programs and services for early childhood development from the 2000-01 baseline onwards for at least three years, we note that the proportion of total invested in child-care services decreased between 2000-01 and 2002-03 in six of them.

However, 2003-04 was the first year of dedicated federal transfers for child-care services under the Multilateral Framework Agreement. For the remaining jurisdictions, once the dedicated Multilateral transfers are in place, the relative decline in child-care services starts to level off or reverses completely, reinforcing the community's desire for federal leadership in child-care policy and funding. The introduction of dedicated federal transfers helps to focus specific attention to child-care services, while at the same time promoting and protecting Canada's commitment to comparable social services across jurisdictions.

Overall, the records of FPT governments meeting both their reporting and performance requirements are spotty and raise questions about this method of accountability. We also have concerns about the capacity of communities to engage in the public reporting process.

Public reporting and citizen engagement

Richard Simeon and David Cameron trace concern over the democratic character of federalism to Donald Smiley, who took issue with executive federalism's secrecy, its lack of citizen participation, and its weak accountability (2002). They identify the period of the Meech Lake Accord as the height of Canadian impatience with executive federalism and note that after Meech, the Charlottetown Accord, and the Calgary Declaration, there were brief experiments in the 1990s with popular engagement in intergovernmental relations. Yet even though governments promised new space for "citizen engagement," by the time the Social Union Framework Agreement emerged, FPT negotiations settled back into old patterns of executive federalism (Cameron 2004; Simeon and Cameron 2008).

Public reporting has been raised as one alternative to top-down, closed-door intergovernmental relations. It is possible that public reporting could play a role in democratizing federalism, but there are significant challenges to this prospect. The "Making the Connections" experience confirms many of the criticisms of public reporting as an accountability mechanism. Shelagh Day and Gwen Brodsky conclude that "public reporting is not a sufficient means of holding governments accountable for rights and the adequacy of social programs.... [P]ublic reporting ... assumes that citizens, given good information, can intervene with political representatives if they are concerned about problems in programs and services, and can vote on the basis of the information. However, reported information can be difficult to understand and is generally inaccessible" (2007: 92). Cameron also voices concern that "[a]ccountability is framed not as accountability of the executive to the elected legislature but directly to the people by means of periodic reports replete with vague performance measures and unhelpful expenditure information" (2007: 175). Susan Phillips adds that

[t]he main implication of the SUFA accountability provisions is to place the onus on citizens, unrealistically in my opinion, to review outcomes, assess their meaning, compare them across provinces and take political action to achieve better results. In effect, it makes social scientists of us all. This is unrealistic not because citizens are apathetic or not up to the task. Outcome measurement is a complex task and public debate about it requires access to relevant data and technical information, the ability to assess the quality of measurement as well as institutional venues for debate on the adequacy and policy implications of the data (2001: 20).

There are some fundamental obstacles to substantive citizen engagement in the reporting process related to community capacity. The FPT agreements specifically note that governments are reporting to their publics and not to each other. (7) The only accountability mechanism for investments in childcare services is through public monitoring and pressure, which places a lot of responsibility on communities. Throughout the Child Care Advocacy Association of Canada's "Making the Connections" project, we heard from researchers and academics about the uniqueness of the project work. Yet it relied on only 1.5 paid staff and the volunteer work of the association's members in the provinces and territories committed to child-care, who, on their own time, found ways to try to engage in policy discussions. There were very practical constraints, including the inability of interested citizens to attend community meetings because of working schedules and other time limitations.

While we believe that this project has supported some capacity-building, the reality is that communities do not have the resources to actively participate in this kind of citizen engagement, even if they are invited to. It is not practical for citizens to track public reporting on their own, and this raises questions about the heavy reliance on this approach as the accountability mechanism when parents and community groups are already time-strapped and over-burdened. In Kershaw's early assessment of this experiment by the Child Care Advocacy Association of Canada, he raised the issue of downloading and privatizing the responsibility for accountability onto child-care advocacy organizations, the majority of whom are women (2006). It appears that citizen engagement is serving as a replacement for, rather than as a supplement to, the state's responsibility for accountability.

Such challenges for citizen involvement are actually exacerbated by public-sector downsizing and the reduction of resources to communities. Citizens need expertise, resources, access to information, structures for participation, and community and organizational capacity for meaningful engagement (Simmons 2008). As Phillips indicates, this requires attention to both the "how" and "where" of citizen engagement and the public funding of advocacy groups.

The type of public funding also matters. Under project-based, as opposed to core, funding, community-based organizations, like the Child Care Advocacy Association, are confined to meeting specific deliverables, with the potential for extensive government direction and possible interference. During the term of the project, for example, funding under the Bilateral Agreements was in place and therefore included in discussions with communities and governments. In addition, some provinces had publicly reported on their investments under these agreements in 2005-06. However, we were advised by Human Resources and Social Development Canada that the project report should analyse only the agreements currently in effect. This directive placed limitations on the scope of the work by taking the politically sensitive Bilateral Agreements largely out of the equation. (8) Tightly controlled projects could fall into the trap of what Boismenu and Graefe have called the "selective cultivation of research themes and expertise" (2004: 77).

There are certainly limitations to public reporting. But the answer is not to simply return to top-down--the "eleven-white-men-in-suits" executive federalism. Herman Bakvis and Grace Skogstad note that, too often, Canadian intergovernmental relations have eschewed democratic participation in favour of competition between governments (2008). Public reporting, if developed, can provide space for citizens in intergovernmental relations that has, thus far, been limited.

Research has shown that citizen engagement can produce better public policy by providing important knowledge (Phillips 2001). Even in the relatively limited "Making the Connections" experiment, community participation generated some valuable insight about federalism and child-care policy. Much of the federalism literature is spent discussing the need for provincial flexibility, or respect, for "[d]istinctive provincial needs and tastes" (Bakvis and Skogtad 2008: 16), but our experience asks the question, whose "needs and tastes"--governments' or citizens'? One of the interesting lessons we took from the project is that claims of provincial diversity are exaggerated. Through cross-Canada consultations, what struck us was the extraordinary consistency in objectives and challenges across communities around three aspects of child-care: wages, fees and spaces. If the policy process around child-care actively involved citizens, the policy consensus would be readily evident. It is only by excluding advocates from the discussion that such a divided perspective on child-care policy has emerged.

Our findings mesh with Martha Friendly's conclusions that Canada's childcare policy is less about respect for diversity than a lack of coherence and political will (2000). We also agree with Simeon and Cameron that "the presence of third parties, with policy goals, not governmental interests, at the centre of their concern helps to ensure that the intergovernmental politics of turf and blame avoidance does not dominate substantive policy discussion" (2002: 291-92). Too often, federalism and jurisdictional arguments act as a guise for blocking substantive policy progress and undermine democracy. As Bakvis and Skogstad remind us, Canadians are much more interested in policy outcomes than federal principles (2008). Furthermore, we must not forget that jurisdictional inflexibility is located in a profoundly gendered division of powers based on the public/private divide (Cameron 2004).

Public reporting, especially if improved, offers opportunities to bring new voices into the federal/provincial/territorial process. Given that public reporting is now taking such a central place in the accountability process, and considering the heavy responsibility this places on the public, effective public reporting is more crucial than ever.

Recommendations for public reporting

Generally agreed-upon recommendations for public reporting are a critical component of accountability. (9) Therefore, in order to promote clear reporting that supports the public in tracking the ongoing progress in child-care services, FPT governments should implement their own guidelines and those outlined by the Public Service Accounting Board.

To improve ease of access, the links on the main federal Early Childhood Development web site should take citizens directly to one central site where all reporting related to early childhood development, including child-care services, can be found. All reports must be readily available and on time. Printer-friendly versions of the reports, without pictures, and alternatives for those without regular Internet access, are needed. Summaries or pamphlets should be made available in places convenient to parents and child-care workers so that information about the reporting can be more widely available.

To improve clarity, all reports should contain the actual release date. Each report should include a summary that captures key public policy and investment changes over time, with investments clearly totalled. Tables in each subsequent report should show not only new financial and non-financial indicators but also trace changes from the baseline and in each subsequent year. Through the course of the project, FPT governments were provided with a suggested reporting template based on the community capacity-building work.

Governments should also adopt a comparable template. While FPT governments have broadly agreed on a shared framework for reporting, there is still a lack of consistency in reporting. There are common programs and approaches across the country, so a common template should be achievable. Individual programs may differ across the country, but at a certain level the reporting on financial and non-financial indicators can be done consistently.

Governments have made broad commitments to citizen engagement. While the "Making the Connections" project is an example of involving third parties in the final review stage of the reporting process, communities also have a right to be involved in developing goals and plans in addition to monitoring results. At a minimum, following the PSAB's recommendation, public reporting should describe the extent to which users were involved in the report and in the selection of performance measures (2007). The Public Service Accounting Board suggests that the chosen indicators should focus on the "few critical aspects of performance." Governments should work with communities to determine which indicators are critically important to improving access to quality, affordable child-care services. Based on the existing FPT objectives, the PSAB guidelines, the experience and views of the childcare community, and the research and international evidence, we recommend that FPT governments consider the following as the few critical indicators of performance for child-care: staff training, staff wages, parent fees and subsidy rates, percentage of children with access to a regulated child-care space, and percentage of children from various targeted groups (aboriginal communities, children with disabilities) with access to regulated space.

The PSAB recommendations speak to the importance of planning for effective management, yet the lack of public plans was notable in the reporting reviewed to date. This is perhaps not surprising because the Early Childhood Development and Multilateral agreements do not require a developed plan, only that governments demonstrate service improvements. Therefore, we recommend that once the few critical aspects of performance are agreed upon, FPT governments should develop long-term goals and shorter-term plans that specifically address each indicator. Phillips also emphasizes that infrastructure is necessary to facilitate participation (2001). Governments must establish a mechanism for the involvement of a broad range of citizens (including parents, child-care workers and employers, advocates, researchers and academics). Resources are required at FPT government levels to support a comprehensive and genuine approach to citizen engagement. Community members express frustration at how some government consultations are conducted and the limited impact that these consultations have on resulting child-care policy and funding. This reinforces a point made by Julie Simmons, that superficial involvement will simply increase citizen disillusionment (2008).

Citizens alone cannot monitor public reporting. Governments must develop and implement a defined role for legislators. As the CCAF-FCVI states, "Government should be the primary user of performance measurement and information," since this information can "provide a sound basis for helping the Legislature hold the government to account" (2006: 11). As Cameron argues, accountability through the principle of responsible government is fundamental for democratic intergovernmental relations (2007). Day and Brodsky also propose that government performance be measured against rights standards (2007).

Our experience affirms other research describing public cynicism about government's public reporting. As a result, we agree with the CCAF-FCVI that an external audit would add credibility (2006: 27). Even more fundamentally, the "Making the Connections" project's review of the reporting neither resembled nor could it take the place of an external audit. It did not have the mandate, the authority or the resources to confirm the accuracy of the information provided by governments in their reports, the reasonableness of the estimates used, or the appropriateness of the methodology employed. Given these constraints, and others noted earlier, these improvements in public reporting can only be one piece of the accountability puzzle.

Conclusion

The child-care community has long recommended that governments use additional mechanisms such as reporting to Parliament/legislatures, legislated standards and auditing, and continue to see the federal spending power and conditionality as critical for public accountability. Such accountability measures are well explored in the literature on federalism. At this point, public reporting seems to be the only mechanism for demonstrating accountability for federal transfers and effective use of public funding for child-care services. Projects like "Making the Connections" can support the community in their analysis of public reports, but many are concerned about the growing reliance on such reports as the accountability mechanism in the public sector. In the area of child-care, our findings indicate that the public reports issued to date under the FPT agreements are not yet living up to the promises of accountability and transparency. And, given the increasing primacy that governments are placing on public reporting, this is a serious problem. While several of the public reports analysed through this project show evidence of improvements for child-care service quality and availability, few have clear public reporting that allows the public to easily track progress and none meets all of the performance and reporting requirements outlined in the Early Childhood Development and Multilateral Framework agreements. This does not mean that public reporting has no value. We out-o lined several ways that public reporting could be more useful to citizens. Yet the question remains whether, even with improvements, public reporting on its own can provide sufficient accountability.

References

Anderson, Lynell, and Tammy Findlay. 2007. Making the Connections: Using Public Reporting to Track the Progress on Child Care Services in Canada. Written for the Child Care Advocacy Association of Canada (CCAAC). Ottawa: CCAAC. Available at http://www.ccaac.ca/mtc/en/ pdf/mtc_finalreport_en.pdf.

Bakvis, Herman, and Grace Skogstad. 2008. "Canadian federalism: Performance, effectiveness, and legitimacy." In Canadian Federalism: Performance, Effectiveness, and Legitimacy, 2nd edition, edited by Herman Bakvis and Grace Skogstad, Don Mills, Ont.: Oxford University Press Canada.

Boismenu, Gerard, and Peter Graefe. 2004. "The new federal tool belt: Attempts to rebuild social policy leadership.'" Canadian Public Policy 30 (1) March: 71-89.

Cameron, Barbara. 2007. "Accounting for rights and money in Canada's social union." In Poverty: Rights, Social Citizenship and Legal Activism, edited by Margot Young, Susan Boyd, Gwen Brodsky, and Shelagh Day, Vancouver: University of British Columbia Press.

--. 2004. "The social union, executive power and social rights." Canadian Woman Studies 23 (3/4) Spring/Summer: 49-56.

Canada. Human Resources and Skills Development. Child and Youth Policy. Early Childhood Development and Early Learning and Child Care. 2004a. First Ministers' Meeting Communique on Early Childhood Development. Ottawa: Public Works and Government Services Canada. Available at http://www.ecd-elcc.ca/eng/ecd/ecd_communique.shtml.

--, --, --, --. 2004b. Multilateral Framework Agreement on Early Learning and Child Care. Ottawa: Public Works and Government Services Canada. Available at http://www.ecdelcc.ca/eng/elcc/elcc_multiframe.shtml#ftnl.

--, --, --, --. 2004c. Shared Framework for Reporting on Progress in Improving and Expanding Early Childhood Development (ECD) Programs and Services. Ottawa: Public Works and Government Services Canada Access at: http://www.ecd-elcc.ca/eng/ecd/ecd_shared frame.shtml.

--. Parliament. Senate. Standing Committee on Social Affairs, Science and Technology. 2007. Proceedings. 39th Parliament, 1st Session, 7 June. Ottawa: Public Works and Government Services Canada. Available at http://www.parl.gc.ca/39/1/parlbus/commbus/senate/ Com-e/soci-e/24cv-e.htm?Language=E&Parl=39&Ses=1&comm_id=47

CCAF-FCVI Inc. 2006. Users & Uses: Towards Producing and Using Better Public Performance Reporting: Perspectives and Solutions. Ottawa: CCAF-FCVI Inc.

Day, Shelagh, and Gwen Brodsky. 2007. Women and the Social Transfer: Securing the Social Union. Written for Status of Women Canada. Ottawa: Public Works and Government Services Canada.

Friendly, Martha. 2000. Child Care and Canadian Federalism in the 1990s: Canary in a Coal Mine. Toronto: Childcare Resource and Research Unit.

Friendly, Martha, and Linda A. White. 2008. "From multilateralism to bilateralism to unilateralism in three short years: Child care in Canadian federalism, 2003-2006." In Canadian Federalism: Performance, Effectiveness, and Legitimacy, 2nd edition, edited by Herman Bakvis, and Grace Skogstad. Don Mills, Ont.: Oxford University Press Canada.

Kershaw, Paul. 2006. "Weather-vane federalism: Reconsidering federal social policy leadership." Canadian Public Administration 49 (2) Summer: 196-219.

Maxwell, Judith, Mary Pat MacKinnon, and Judy Watling. 2007. "Taking fiscal federalism to the people." Policy Options 47 (3) March, available at the Canadian Policy Research Networks web site http://www.cprn.org/documents/47307_en.pdf.

McKeen, Wendy. 2007. "The national children's agenda: A neoliberal wolf in lamb's clothing." Studies in Political Economy (79) Spring: 151-73.

Morris, Kathleen, and Jennifer Zelmer. 2005. "Public reporting of performance measures in health care." Canadian Policy Research Networks, Health Care Accountability Papers No. 4, Health Network. Ottawa: CPRN. Available at http://www.cprn.org/documents/34864_en.pdf.

Phillips, Susan D. 2001. "SUFA and citizen engagement: Fake or genuine masterpiece?" Policy Matters 2 (7) December: 1-36.

Public Sector Accounting Board. 2007. Public Performance Reporting: Guide to Preparing Public Performance Reports. Ottawa: PSAB. Available at http://www.psab-ccsp.ca/other-non-authoritative-guidance/item14604.pdf.

Simeon, Richard, and David Cameron. 2008. "Intergovernmental relations and democracy: An oxymoron if there ever was one?" In Canadian Federalism: Performance, Effectiveness, and Legitimacy, 2nd edition, edited by Herman Bakvis and Grace Skogstad, Don Mills, Ont.: Oxford University Press Canada.

Simmons, Julie. 2008. "Democratizing executive federalism: The role of non-governmental actors in intergovernmental agreements." In Canadian Federalism: Performance, Effectiveness, and Legitimacy, 2nd edition, edited by Herman Bakvis and Grace Skogstad, Don Mills, Ont.: Oxford University Press Canada.

Notes

(1) Throughout, we will use the terms "early learning and child-care" and "child-care" interchangeably because quality child-care includes early learning.

(2) "Community" refers broadly to those outside of government, including parents, child-care workers, advocates, organizations and researchers.

(3) The URL addresses for these documents were recently changed, so the links published in public education materials are no longer active. From the standpoint of citizen engagement, this difficulty of access is a problem.

(4) It must be noted that the standards that governments have set for themselves do not place the bar particularly high in comparison to, say, measurement against community expectations.

(5) In his 7 June 2007 comments to the Senate standing committee on social affairs, science and technology, Shawn Tupper, director general, Social Policy, Human Resources and Social Development Canada, acknowledged the problem of late reporting but insisted that governments have confirmed their commitment to public reporting (Canada, Parliament, Senate, Standing Committee on Social Affairs, Science and Technology 2007).

(6) Some jurisdictions have subsequently provided their 2005-06 reports.

(7) In fact, this message is repeated in all three of the central FPT documents, and, in meetings with government officials, there was palpable sensitivity about comparisons across jurisdictions and apprehension at being "centred out" (see Canada, Human Resources and Skills Development, Child and Youth Policy, Early Childhood Development and Early Leaning and Child Care 2004a, b, c).

(8) In the end, we were able to include the Bilateral Agreements in our financial analysis because funds had been transferred to the provinces and territories and some had reported on the use of that funding.

(9) There is cause for concern that even accountability through public reporting is on the decline, given that recent federal transfers are not (or are no longer) part of an FPT agreement. We have not found detailed public guidelines or provincial and territorial investment of federal transfers related to the now-terminated Bilateral Agreements (funding provided from 2004-05 through 2006-07) or the federal child-care space transfers introduced in the federal budget of 2007. Martha Friendly and Linda White have dubbed this the unilateral regime of federalism, where neither conditions nor public reporting is required (2008).

Tammy Findlay is a Post-doctoral Research Fellow at the Human Early Learning Partnership, University of British Columbia. Lynell Anderson, CGA, is senior researcher at the Human Early Learning Partnership. They acknowledge the helpful comments made by the Journal's anonymous reviewers.

Table 1. Summary of Annual Reporting on Early Childhood Development,
Learning and Care Activities

                                                  Annual investment
                                                  increase (decrease)
                                 Total baseline
                                 funding Year 0   Year 1    Year 2
                                    2000-01       2001-02   2002-03

Early learning and child-care
services

Income supports to families

All other programs and
services

Total investment--all early
childhood development and care
programs

Less: total federal transfers

Provincial contribution to
early childhood development,
learning and care programs or
(federal transfers not yet
invested)

                                       Annual investment
                                      increase (decrease)

                                 Year 3    Year 4    Year 5
                                 2003-04   2004-05   2005-06

Early learning and child-care
services

Income supports to families

All other programs and
services

Total investment--all early
childhood development and care
programs

Less: total federal transfers

Provincial contribution to
early childhood development,
learning and care programs or
(federal transfers not yet
invested)

                                 Total Year
                                 5 funding    Total change
                                  2005-06     over baseline

Early learning and child-care
services

Income supports to families

All other programs and
services

Total investment--all early
childhood development and care
programs

Less: total federal transfers

Provincial contribution to
early childhood development,
learning and care programs or
(federal transfers not yet
invested)