Readability of management's discussion and analysis for local governments.

Marsh, Treba Lilley ; Montondon, Lucille Guillory ; Kemp, Amanda M. 等

ABSTRACT

The Governmental Accounting Standards Board issued Statement 34 in June 1999, requiring governmental entities to prepare a Management Discussion and Analysis (MD&A) as part of their Comprehensive Annual Financial Reports. Statement 34 also reiterated that citizens are considered users of governmental financial statements and the MD&As. This paper discusses various readability measures and uses them to evaluate municipal MD&As. We found that most of the reviewed MD&As have average readability measures of grade 9 or higher. The finding is not as unequivocal as desired, however. The Flesch-Kincaid measure differs by four grades depending upon the source of the program used (Grammatik or Readability Plus for Windows). The results of the current study may influence local governments to write the governmental MD&A at a lower reading level and use a more understandable writing style.

INTRODUCTION

The Governmental Accounting Standards Board (GASB) issued Statement 34 in June 1999, requiring governmental entities to prepare a Management Discussion and Analysis (MD&A) as part of their Comprehensive Annual Financial Reports (CAFRs). The MD&A discusses financial performance, capital resources, budgets, and issues and trends relating to the governmental entity's financial position or results of operation. One of GASB's justifications for Statement 34 was "greater accountability by providing more useful information to more users" (Statement 34, paragraph 184). GASB Concept Statement #1 states that citizens are one of the three groups of users of governmental financial statements. According to Statement 34, "the fact that few citizens or legislators currently read their financial reports does not mean that they never will, especially if the financial reports can be made more understandable and more useful to them" (Statement 34, paragraph 194). To reach this new user group, particular care is required to explain the governmental entity's financial position simply and clearly. Readability is "that quality in writing which results in quick and easy communication. Readable writing communicates precisely and with a single reading" (Lesikar and Lyons, 1986, p.21). Since GASB identified three groups of primary users, and purposefully chose to set no group as having priority, readability is of particular importance.

This paper reports on readability of MD&As in CAFRs of early adopters of Statement 34. It discusses various readability measures and applies them to the MD&As of municipal entities of all sizes. It discusses five readability measures and correlates them to characteristics of the entities.

STATEMENT 34

The CAFR must provide governmental accountability to citizens, legislators and creditors. It is important for constituents of a governmental entity to understand how resources are being raised and how they are being utilized. The new requirements of GASB Statement 34 are designed to make CAFRs easier to understand and are considered "the culmination of years of planning and discussion on how to get governments to provide a clearer picture of their financial well-being" (Valuing Infrastructure, 2000).

GASB prepared guidelines for the MD&A, but style and format of presentation are at the discretion of management. An important aspect of the MD&A is the requirement of financial managers to share with constituents an analysis of the governmental entity's financial performance that is both readable and understandable. In essence, the MD&A section should provide citizens with information necessary to assist them in evaluating whether the entity's financial position has improved or worsened as a result of the current year's operation. It is crucial for the information to be displayed in a manner that is readable and understandable by the average United States citizen.

GASB structured the new financial reporting model to be implemented in three phases. Table 1 indicates the dates when governing bodies are required to implement Statement 34 although many are choosing to implement earlier.

READABILITY LITERATURE

Table 2 from Kinnersley and Fleischman (2001) provides the readability of commonly known documents. These documents are intended for the general public and are listed from the highest to the lowest reading grade level required.

Studies of readability of corporate financial reports show that annual reports have become less readable (Pashalian & Crissy, 1950; Sopor & Dolphin, 1964; Smith & Smith, 1971; Barnett & Loeffler, 1979). In addition, the readability of MD&As in corporate annual financial reports was noted to be no better than readability of footnotes (Schroeder & Gibson, 1990) which according to Golub and Kueppers (1983) "... are often written in a style and vocabulary that only accountants and attorneys can readily understand." According to Urbanic (1993), the reading level of tax instructions requires approximately two years of college.

A survey reported by the Financial Executives Research Foundation (1987) found individual investors uncomfortable with the language and complexity of traditional annual reports while professional investors did not report problems. Most of the individual investors thought writing the reports in more understandable language and without technical jargon would improve the reports. In summary, narrative disclosures receive poor ratings for readability in both academic research studies and from individuals.

READABILITY MEASURES

There are three aspects of writing known to affect readability (Flesch, 1948). These variables are passive voice, sentence complexity and vocabulary complexity. The passive voice data provides the percentage of finite verb phrases in the document. Too many passive constructions can make a passage difficult to understand. Short words and sentences communicate better and are easier to understand than long words and sentences. Sentence and vocabulary complexity is based on a scale of 0 to 100, with 100 denoting very complex.

Readability is assessed using either readability formulas such as Dale-Chall, Flesch Reading Ease, Flesch-Kincaid Grade Level Formula, FOG and SMOG. In tables 3 and 4, Flesch-Kincaid from the Grammatik program is labeled Flesch-Kincaid 1 with the same measure from the Readability Plus for Windows program as Flesch-Kincaid 2. Readability formulas are mathematical models that assign a score to a selected passage. Essentially, readability formulas measure sentence difficulty by counting the number of words per sentence and word difficulty by number of letters of syllables per word.

Dale-Chall and Flesch-Kincaid Grade level formulas express their results as grade levels. The Flesch Reading Ease expresses its result as a value on a scale from 0 to 100 with the lower the score indicating more difficult material. For example, a passage that scores 100 has an average sentence length of 12 words or less with no words over two syllables. Below 30 is very difficult. Above 70 is quite easy.

Similarly, the formula developed by Robert Gunning, FOG, takes into account the total number of words, words of three or more syllables, and number of sentences in a paragraph. "Most experts feel when using FOG no technical publication should score higher than 14, no general business publication higher than 12 and no clerical publication higher than 8" (Readability Calculations p. 5).

The SMOG formula relies solely on the number of words containing three or more syllables. Whereas most formulas predict the grade level for 75%-85% comprehension, SMOG focuses on 100%.

In summary, the five methods, Dale-Chall, Flesch, SMOG, FOG, and Flesch-Kincaid Indices, evaluate some or all of the following: average number of syllables per word, average number of words per sentence, and the average number of sentences per paragraph. All measures except the Flesch Reading Ease score reports results by grade levels.

METHODOLOGY

The governmental entities included in this study have implemented GASB 34, and their CAFRs contain the required MD&A section. Using the Governmental Accounting Standard Board web site of all early implementers, we chose to focus our analysis on city implementers as of February 1, 2003.

Fifty-eight of 86 cities provided an electronic CAFR. Although numerous governments provided electronic CAFRs, several were unusable for our purposes. In these instances, we requested a hard copy of the CAFRs. From a population of 86 municipal early implementers, our sample size of usable CAFRs was 84.

The demographic data are from the U.S. Census Bureau's web site and an interactive database engine called American FactFinder. The population and per capita income are from the 2000 census.

We used the readability program, Grammatik, contained in the word processing program Corel WordPerfect. Using the Flesch-Kincaid readability formula in its analysis, Grammatik creates a report of the readability of the MD&A section. The report includes the percent of sentences in the passive voice, sentence complexity, and vocabulary complexity. In addition, we used a software readability program from Micro Power & Light Company entitled "Readability Plus for Windows." The programs utilized in the analysis were Dale-Chall, Flesch Reading Ease, Flesch Grade Level, FOG, and SMOG formulas. Actual content is not assessed by any readability measure.

We analyzed each MD&A using five readability measures. Then the measures were compared to each other and to population and revenue of the reporting entities. Our findings follow.

FINDINGS

We compared the scores from each formula. The means for grade level measures ranged from 8.7 to 12.9. For a given entity, there was often a range of eight grade levels obtained by the five measures. However, the measures were consistent and significantly correlated at the .01 level of significance. This indicates that the measures apply different weights to the variables assessed but provide internally consistent results. See Table 3.

The sample was partitioned by size of the entity. The construct "size" was defined using the same revenue standards GASB used when developing the implementation schedule. The partition with the greatest revenue is designated as group 1, etc. Although smaller entities had higher readability measures (required higher grade level to comprehend) in the majority of the cases, the differences were not statistically significant. Table 4 reports the results of each of the various readability measures for the partitioned sample.

There are statistically significant differences between the largest and smallest entities' MD&As when applying the Flesch Reading Ease measure. The means differed by 4.68 and this is significant at the 0.015 level. Unfortunately, for citizens in smaller communities, the MD&As of these entities are more difficult to read than those of the largest entities. See Table 5.

Correlating size, revenue, and revenue per capital to the other four readability measures showed mixed results. There is a statistically significant negative correlation (at the .05 level) between revenue and two of the readability measures (SMOG and Flesch-Kincaid). This indicates that as revenue decreases, the readability level of the MD&A increases. There are also statistically significant positive correlations between size and two different measures (FOG and Dale-Chall). We can draw no conclusions from these results because the readability measures are highly correlated to each other (.000 to .018).

DISCUSSION AND ANALYSIS OF FINDINGS

Recent census data shows that about 84% of adults over 25 years of age have a high school degree and approximately 26% have one or more college degrees. Despite these levels of educational attainment, research shows that many people read three to five grades lower than their highest level of educational attainment. Thus, it is not unusual for someone with a high school diploma to be reading at a seventh to ninth grade reading level. Combining these facts, more than one-third of U.S. adults read below the eighth grade level. Because of this, Larocque recommends materials written for the general public should have a tenth grade reading level or below. In fact, many recommend that materials for the general public should have a junior high reading level. Yet, most of the MD&As reviewed have readability measures of grade 9 or higher. This finding suggests that if government officials wish to convey the financial state of the municipality to citizens, the reports must be simplified.

The readability measures are not as unequivocal as desired, however. The Flesch-Kincaid measure differs by four grade levels depending upon the source of the program used (Grammatik or Readability Plus for Windows). A clearer, but not encouraging, finding is that the Flesch Reading Ease measure averages 47.7 (maximum 68.8, minimum 31.7). This indicates fairly difficult reading since above 70 is considered easy and below 30 very difficult.

Another concern is that smaller entities have significantly more complex reports using the Flesch Reading Ease score. Although size is not correlated to revenue, smaller municipalities are hypothesized to have a larger percentage of citizens without the financial background or reading skills, making them less able to understand the financial performance reports of their government.

LIMITATIONS

Variations between readability measures were notable. However, in most cases, the results indicated fairly complex passages, measured by number of syllables per word, words per sentence, etc. Additionally, the content of financial reports is technical and requires comprehension of a basic vocabulary. This aspect was not measured. Another limitation was in relation to the various graphs and tables included within the MD&A sections. Primarily these graphs/tables consisted of numerical data. When we ran the readability analysis on a document before and after extracting the graphs and/or tables, the difference in the results was minimal.

CONCLUSIONS

This study reports on the readability of the MD&As in the CAFRs of early adopters of Statement 34. These results are based on their first MD&As. With refinement, writers will learn to present essential information in a more simplified format.

To increase the value of the financial reports, preparers should educate citizens through local newspapers and/or public forums. Even the best financial reports are of little value unless interested parties take an interest and devote some time and effort into understanding the reports.

Thus, municipalities should consider a two-phase approach. First, simplify the management discussion and analysis as much as possible. Second, enlist local civic groups to sponsor forums or discussions to explain the new reporting to citizens. Only then will GASB's purpose of three equal groups of users of government financial information be achieved.

REFERENCES

Barnett, A. & K. Loeffler (1979). Readabiltiy of accounting and auditing messages. The Journal of Business Communication, 16(3), 49-59.

Flesch, R. (1948). A new readability yardstick. The Journal of Applied Psychology, 32(3), 221-233.

Golub, S. J. & R. J. Kueppers (1983). Summary reporting of financial information: Moving toward more readable annual reports. Morristown, NJ: Financial Executives Research Foundation.

Governmental Accounting Standards Board (1987). Concepts Statement #1 Objectives of financial reporting. Norwalk, CT: GASB.

Governmental Accounting Standards Board (1999). Statement no. 34 of the governmental accounting standards board: Basic financial statements and management's discussion and analysis for state and local governments. Norwalk, CT: GASB.

Kinnersly, R. & G. Fleischman (2001). The readability of government's letter of transmittal relative to public company management's discussion and analysis. Journal of Public Budgeting, Accounting & Financial Management, 13(1), 1-22.

Larocque, P. (2003, May). Dumb or dumber?, Quill Magazine, 40.

Lesikar, R. L. & M. P. Lyons (1986). Report writing for business. Homewood, IL: Irwin.

Pashalian, S. & W.J.E. Crissy (1950). How readable are corporate annual reports? Journal of Applied Psychology, 34(4), 244-248.

Readability calculations (2002). Dallas, TX: Micro Power and Light Co.

Schroeder, N. & C. Gibson (1990). Readability of management's discussion and analysis. Accounting Horizons, 4(4), 78-87.

Smith, J. E. & N. P. Smith (1971). Readability: A measure of the performance of the communication function of financial reporting. The Accounting Review, 46(3), 552-561.

Sopor, F. J. & R. Dolphin, Jr. (1964). Readability and corporate annual reports. The Accounting Review, 39(2), 358-362.

SRI International (1987). Investor information needs and annual report. Morristown, NJ: Financial Executive Research Foundation.

Urbanic, F.R. (1993). An analysis of tax instruction readability for individuals in the northeastern states. Journal of Business and Economic Studies, 2(1), 59-68.

Valuing infrastructure assets: GASB 34 and you. (2000). Public Works, 131(10), 85.

Treba Lilley Marsh, Stephen F. Austin State University

Lucille Guillory Montondon, Texas State University--San Marcos

Amanda M. Kemp, Deloitte & Touche LLP Table 1: Implementation Guidelines Phase Total Annual Revenues Effective Beginning With 1 $100 million or more Fiscal year 2001-2002 2 At least $10 million but less than $100 million Fiscal year 2002-2003 3 Less than $10 million Fiscal year 2003-2004 Table 2: Flesch-Kincaid Grade Level of Other Writing Publication Flesch-Kincaid Grade Level Microsoft Word User's Guide 15 US Constitution including all Amendments 14 New York Times (one article) 14 Gettysburg Address 13 Associated Press Story (one article) 13 Wall Street Journal 11 Time 11 Newsweek 11 1040EZ Instructions 11 Microsoft Windows Manual 10 TV Guide 5 Cosmopolitan 5 Hemingway Short Story 4 National Enquirer 3 Table 3: Descriptive Statistics N Minimum Dale-Chall 84 7.9 Flesch-Kincaid 1 83 9.43 Flesch-Kincaid 2 84 4.4 FOG 84 7.4 SMOG 84 5.7 Maximum Mean Std. Deviation Dale-Chall 10.6 9.33 0.45 Flesch-Kincaid 1 16 12.91 2.21 Flesch-Kincaid 2 12.8 8.72 1.42 FOG 15.9 11.63 1.49 SMOG 14.6 10.63 1.39 Table 4: Group Statistics Size N Mean Std. Std. Deviation Error Mean Dale-Chall 1 11 9.245 0.398 0.120 2 39 9.203 0.481 0.077 3 32 9.484 0.375 0.066 Flesch-Kincaid 1 1 11 13.814 2.412 0.727 2 39 12.707 2.204 0.353 3 31 12.819 2.109 0.379 Flesch-Kincaid 2 1 11 8.164 1.765 0.532 2 39 8.703 1.538 0.246 3 32 8.913 1.126 0.199 FOG 1 11 10.882 1.818 0.548 2 39 11.636 1.586 0.254 3 32 11.838 1.228 0.217 SMOG 1 11 10.000 2.003 0.604 2 39 10.690 1.405 0.225 3 32 10.731 1.117 0.197 Table 5: Flesch Reading Ease Comparison of means of segmented sample Size N Mean Std. Dev. 1 11 50.76 6.68 2 39 48.22 6.47 3 32 46.08 4.76