The metrics of style: Adam Smith teaches efficient rhetoric.

Diamond, Arthur M., Jr. ; Levy, David M.

I. QUANTIFYING CLARITY: ADAM SMITH AND RUDOLF FLESCH

Teachers of English rhetoric often give lists of rules to follow. What is not so often done is to give reasons for the rules, to help one determine which rules are hard and fast and which are not. The lack of reasons for the rules leads some to question whether the rules are not therefore mere prejudice (Crystal |1987, 2~). For example, writing concisely is often on the list.(1) Adam Smith, also a teacher of rhetoric, sketched the reasons why conciseness matters.(2) What, for instance, is so good about short sentences? Smith's answer is "Short sentences are generally more perspicuous than long ones as they are more easily comprehended in one view" |1985, 7~.

"Make it easy for a busy reader to get the point" may serve as the guiding principle for the plain style. In particular, Smith points to the natural order of an English sentence as a key to clear communication |1985, 6~. Comparing English with inflected classical languages, such as Greek and Latin, Smith makes much of the fact that English links meaning to strict word order |1985, 225~. Breaking order means recourse to the passive voice and added prepositions. Change Robert beat John to John was beaten by Robert. Three words swell to five. The passive in English has two drawbacks: sentences become wordier and the natural order of English is compromised. Two centuries pass and we find grammarians offering the same advice: avoid the passive.(3) Modern guidance has now been reduced to formulae, the best known of which is the Flesch Index. This index was originally developed by Rudolf Flesch in a series of publications |1946; 1948; 1949~. If WL is the number of syllables per 100 words and ASL is the average number of words per sentence, the Flesch Index = 206.835 - .846WL - 1.015ASL. For example, a Flesch score of 100 would indicate great clarity and a score of 1, great opacity.(4) The Flesch Index and its competitors have been quite controversial.(5) Duffy |1985~, who summarizes and extends some of the doubts about these mechanical procedures, nevertheless concludes that the formulas "... do an excellent job of accounting for variations in the difficulty of prose". Some research in the psychology literature also suggests that the formulas may be valid measures of a text's clarity.(6)

What kind of evidence is there for the efficacy of such norms of style? On theoretical grounds there is reason to believe that normative advice can evolve to assist hard optimization problems (Levy |1988b~). We have noted two centuries of advice to avoid the passive. While no such evidence can be offered for the Flesch Index, it can pick out stylistic extremes from a sample of academic writing. McCloskey has identified a few contemporary economists as exemplary writers: George J. Stigler, Robert Solow, and John K. Galbraith. Using the Grammatik IV computation of the Flesch Index, we find J.B. Clark and Paul Samuelson coming in first and second. Restricting attention to the 1960s and later, we find Samuelson (of course), W. A. Lewis, Solow and Galbraith the top four.(7) At the other extreme, here is the first sentence from the lowest-ranked address: "In view of the fact that the war situation, though happily, in the end, rendering possible our assembly here, has prevented the

integration of our discussions with those of other social science organizations, it seems appropriate that this address should not confine itself to strictly economic matters."(8)

There is a terrible problem letting a few extreme observations force one's results. Hence, we employ regression techniques which are robust against extreme observations to determine whether either type of advice helps in the production of citations. An answer can be found by testing whether an increase in citations results from a decrease in the percentage of sentences in the passive voice or an increase in the Flesch Index. Two different dimensions of writing determine the Flesch Index. Increasing either syllables per word or words per sentence reduces clarity. The Flesch Index further asserts there is a linear relation, specified above, between the two in terms of reduction of clarity. In a regression of citations, the explanatory power of a function constrained to Flesch's specific functional form can be tested against the explanatory power of an unconstrained function.

II. DATA

Our data set consists of the American Economic Association presidential addresses.(9) We have two main reasons for choosing presidential addresses. First, presidential addresses are important documents, surely taken very seriously by those who write them. We would expect the clarity of Presidential addresses to be higher than the average article in economics. As E. W. Kemmerer said at the start of his own address: "The occasion offers an excellent opportunity to an economist to deliver an epoch-making message to the economic world; ..." |1927, 1~. Thus by choosing presidential addresses we expect to have a particularly stringent test of the hypothesis that the clarity of writing has declined over time. Second, if we believe that the election to presidency serves to select the best in the population of economists, we have some insight into the ability of the author of the paper. The larger the sample, the higher the expected sample extreme (Kendall and Stuart |1977, 352-54~. The more members in the AEA, the more capable we would expect the president to be. This gives us a proxy for the ability of the author. Our source for the number of members in the AEA is Diamond and Haurin |1994~. Ninety-seven addresses (the first published in 1888, the last in 1990) have either been typed into computer-readable form or else scanned-in and then corrected. Presidential addresses have not been delivered in all years, and in a couple of years the "presidential address" was delivered by an acting or vice president. The authors would be happy to provide the interested reader with an account of how these issues were resolved to build our data set.(10)

As a measure of the impact of an economist and her work, we use citations. The source for the citations is the Social Science Citation Index (SSCI) which has recently been extended back to 1956. Although it would be useful to have earlier citations, a cross section of recent citations is still useful in discussing earlier economists (see Levy |1988a~ and Anderson, Levy and Tollison |1989~). We constructed two kinds of counts: one to all of the economist's work prior to the period of citation, the other just to the presidential address. In the current estimation we only make use of the former. We use the presidential addresses as a sample to obtain information about their authors' style over all their works. The stability of an author's style over various writings has been heavily studied, (Crystal |1987, 66-69~). If there is in fact no correlation between the style measures on presidential addresses and the authors' usual style, then we would expect no relation between the style we measure and citations. Citations are subject to several standard qualifications (see Diamond |1986~). For instance, we may have failed in our attempt to discard citations to like-named social scientists.(11) Our citation counts are first-author-only, as are the vast majority of the counts used in the citation literature; hence, Modigliani does not get credit for work cited to Ando and Modigliani. For many purposes, the less-costly first-author counts have been shown to be adequate. Finally, we do not attempt to separate positive from negative citations. III. THE PRODUCTION OF CITATIONS BY CLEAR WRITING

Although we have elsewhere put forward models of scientific research in which time is employed by the individual scientist in some maximizing manner, for our present purposes we can be quite vague about incentives or motives. What we explore is only the narrow question of whether clarity of style is a contributor to citations.(12)

The econometric question which we pose is what explains the production of citations? Our econometrics focuses on the clarity of style and the passage of time. As knowledge accumulates and what was novel in past research becomes absorbed into the common stock of knowledge, we expect the citation rewards to fade. This hypothesis, unfortunately, does not speak to the question of what functional form we might expect to relate citations to the passage of time and to clarity of style. In the absence of knowledge of the form of the function, we systematically report Box-Cox regressions where the functional form and the regression coefficients are estimated simultaneously.

There are two good econometric reasons for employing a Box-Cox procedure. First, we reduce the bias in hypothesis testing which comes from rummaging around a box of functional forms until the results we hope for are realized (Leamer |1983~). Second, the Box-Cox procedure, by transforming variables in such a way that the estimated residuals approximate normality, is robust against outliers in the dependent variables (Zarembka |1974~). Our results are checked by employing both least squares and robust regression techniques on traditional transformations. The results we report are neither sensitive to the choice of technique nor the choice of functional form.

We have two competing views of what constitutes an objective indicator of effective prose. Adam Smith holds that adherence to the natural structure of English is key. This we can proxy with the percentage of the sentences which are passive. The conventional stylists hold that short sentences and simple words are key. We noted above that there is reason to expect positive correlation between the percent passive and words per sentence.(13)

If the profession undervalues clear writing, then we might not expect to observe much impact of clear writing on a measure of professional recognition, such as citations. To test the proposition that there is an impact of some indicator of good writing on the production of citations we form the specification for i = 1 to 97:

|Mathematical Expression Omitted~

For each ith AEA President, "CITATIONS+" is the total number of citations received in 1987 (to all of the President's work prior to 1987) plus a small positive number. The variable "CITATIONS" is thus the sum of the first-initial-only and the first-and-second initial citation counts. To deal with the occasional 0 citations in the data set, we follow convention and add 0.1 to each author's citation count. "YEAR" is the year the address is published. "STYLE" is one of several possible indicators of efficient writing.

We claimed above that we should expect that the larger the AEA membership, the more capable the president. Because membership has been growing in the AEA over time, we attempt to separate membership from time. To capture the effect of the growth in membership, which is not captured in the simple passage of time, we compute a Box-Cox regression explaining members by time and take the residuals, transformed to linear form, to be membership unexplained by time. These residuals are called BOXRES.(14)

The results of the production of citations are reported next in Table I. To understand the table clearly, recall that the Flesch Index is scaled so that a higher value is "good." For the other style variables in Table I, a higher value is "bad." It is interesting to compare equation 1-1 using the Flesch Index with equation 1-2 using the components of the Flesch Index entered in unconstrained form. One cannot reject the constraint of Flesch's specific functional form at conventional levels. Equation 1-2 does capture the intuition that a specialized vocabulary of big words might well be efficient and so ought to cost an academic less than wordy sentences. Nonetheless, a likelihood ratio test cannot reject the Flesch constraint at conventional levels. Clearly, Rudolf Flesch's specific functional form is worthy of respect. Equations 1-3 to 1-5 add, as an independent variable, the percentage of sentences which are written in the passive voice. Compare equation 1-4, which only counts the percent passive, with equation 1-5 which counts both the percent passive and the Flesch Index. The Flesch Index is not significant at conventional levels either by a t-test or a likelihood ratio test. The positive significance of the BOXRES variables is consistent with our hypothesis TABULAR DATA OMITTED that AEA presidents from years with more members--taking out the time trend in membership--are more productive than those from years with less members. The larger the membership, the higher one can expect the extremes to be.

The basic specification which we employ to test the proposition that there has been a secular decline in the writing style of the profession is, for observations i = 1 to 97:

|Mathematical Expression Omitted~

|STYLE.sub.i~ is one of the indicators discussed. |YEAR.sub.i~ is the year of publication of the presidential address. EARLY is a dummy variable which takes on the value of 1 before 1905 and 0 after. This variable is included to deal with the change in the AEA which Coats |1960; 1964~ documents. In 1905 the AEA unambiguously became a scientific society; hence, one would expect the presidents to be less concerned with communicating with the outside world. The results are reported in Table II.

Equations 2-2 and 2-3 document an increasing complexity of economists' prose style as measured by the Flesch Index. Words get bigger and sentences get longer. However, equation 2-1 documents that the passive voice is being expunged from academic economic writing. As the results in Table I report, it is the passive voice which significantly degrades citations. This result would not surprise Smith.(15) The Flesch Index, or its components, seem to significantly reduce citations only to the extent that it is correlated with the passive voice.

TABLE II
Dependent Variables: Indicators of Style (t-statistics in parentheses)

 Passive Flesch Flesch
Year -8.927 -0.0115 -0.0165
 (-2.62) (-2.03) (-3.40)
Early -0.277 15.853 --
 (-2.46) (1.33) --
Constant 56.489 506.83 632.41
 (2.54) (2.70) (4.23)
Lambda -0.04 1.42 1.41
LLF 161.54 -322.63 -323.53
Adjusted |R.sup.2~ 0.06 0.11 0.10
Observations 97 97 97
Equation 2-1 2-2 2-3

Equation 2-1 might well suggest that economists in the mid-twentieth century have read the same style manuals as everyone else. Strunk and White in America and Orwell in England have taught two generations to avoid the passive voice.

IV. CONCLUSION

If we learn our rhetoric from Adam Smith, we learn to appreciate the natural order. Keeping to the natural order, as Smith argues in many contexts, is the key to efficiency. And so it is with the English prose of American economists. As economists push the maximizing model toward aspects of human behavior we think unstudied, we may well find that the greatest analytical artist in economics was here before, sketching the solution.

1. Strunk and White |1979, 23~: "A sentence should contain no unnecessary words, a paragraph no unnecessary sentences, for the same reason that a drawing should have no unnecessary lines and a machine no unnecessary parts." Orwell |1968, 139~: "Never use a long word where a short one will do. If it is possible to cut a word out, always cut it out."

2. McArthur |1992, 866~: "The Scottish scholar Adam Smith chose English rather than Latin when giving his lectures; his friend Hugh Blair was appointed to the first chair of Rhetoric and Belles Lettres at the U. of Edinburgh in 1762, the precursor of all chairs of English language and literature around the world." 3. Orwell |1968, 139~: "Never use the passive where you can use the active." Strunk and White |1979, 18~: "The active voice is usually more direct and vigorous than the passive."

4. Flesch |1949, 216~ claims that the scale ranges from 0 to 100, while the addendum to one of the computer programs we used, Readability |1989, 12~ claims a range of 1 to 100. In fact, the Index can be negative and has a theoretical upper bound of 121.22 for the case where each word in the writing sample is a single syllable long and each sentence is a single word long.

5. When we presented an earlier version of the paper to the meetings of the American Economic Association, Mark Blaug rose from the audience to wager that the Flesch Index was flawed because it would not identify an early master stylist, such as Adam Smith, as a clear writer. After scanning-in Book I, Chapter I ("Of the Division of Labor") of the Wealth of Nations, we found that the Readability score--which counts periods and semicolons as terminators--for Smith was 44.3, as compared to a mean of 37.04 for the presidential addresses on the same convention. One might think that any formula which grades style on the basis of sentence length would give Adam Smith low marks. "Obviously," he writes in long sentences. in fact, this is not so. Adam Smith, and many other eighteenth century writers, used a great many semicolons or colons as sentence terminators. As grammarians before and after Smith would attest, a sentence is defined as a complete thought or sense. Smith |1985, 17~: "A Period is a set of words expressing a compleat sense without the help of any other." Thus, it is a matter of taste, not grammar, whether a sentence is terminated by a period, semicolon or colon.

6. Gilinski |1948~ found correlations ranging from .61 to .84 when she compared subjective evaluations of the readability of texts with Flesch Index scores. As part of a useful survey, Rothkopf |1972, 317~ notes:" ... readability formulae were originally developed for a very specific purpose, namely, matching written materials to the educational status of readers. Nevertheless, there are very plausible psychological translations of the main readability factors." Glazer |1974, 467~ supports this generalization. Based on her own correlations of sentence length with a measure of sentence complexity, she concludes that: "In most cases sentence length can be considered a good indication of difficulty." 7. When we use the percent passive we find the "best" four to be Gardner Ackley, James Tobin, Gary Becker and J. M. Clark.

8. The computations were performed in Grammatik IV because it allows one to include colons and semicolons as sentence terminators.

9. We assume that achieving the Presidency of the AEA is an indication of a high level of scientific attainments. But for the early period of the AEA, as documented by A. W. Coats |1960; 1964~, the agenda of many participants was more oriented toward policy and religious objectives, than toward science. Coats suggests that only with the joining of J. L. Laughlin of the University of Chicago in 1904 did the Association become unambiguously devoted to the pursuit of economic science. Coats notes |1960, 527~: "In retrospect it is clear that Laughlin's action marked the final disappearance of the suspicion of the organization which it had encountered, in varying degrees, since its inception, and betokened its permanent establishment as a strictly scientific and scholarly body." Apart from substantive content, and age, we would expect the pre-AER (i.e., pre-1910) presidential addresses to be less-cited because they are harder to discover and access. Referring to the pre-AER era, Coats has noted |1969, 57~ that: "The list of Publications printed annually in the Papers and Proceedings of the Association does not specify each item, and as the contents of the Publications were excluded from the A.E.A. Index of Economic Journals, a number of important papers and Presidential Addresses have been virtually consigned to oblivion."

10. In order to prepare the texts for processing by the programs, the texts had to be converted into ASCII files. Further preparation included the elimination from the ASCII texts of headings, footnotes, tables and bibliography. We also eliminated all mathematical symbols. Where a major part of a sentence was mathematics (one-fifth or more), the whole sentence was eliminated. Where a minor part of a sentence was math (one-fifth or less), the math was eliminated, but the rest of the sentence was retained.

11. There is a convention in the SSCI which deserves mention. To save space, the SSCI uses first and middle initials instead of full first and middle names. We have done the counts separately under the author's first initial-only listings and first-and-second-initial listings, so that, for example, we know how many times J. Galbraith was cited and how many times J. K. Galbraith was cited. Making the counts separately would be useful if the first-initial only counts were more likely to be contaminated by citations to other social scientists who share the last name and first initial with the AEA president whose citations we are trying to isolate.

12. Laband and Taylor |1992~ report that "bad" writing has no impact on citations. McCloskey |1992~ criticizes the report.

13. Strunk and White |1979, 19~. The Pearson correlation coefficient between the percentage of sentences in the passive voice and the words per sentence is 0.468. The Spearman correlation coefficient is 0.475. Both of these are decisive evidence against independence.

14. We have membership information only from 1893 to the present; hence, the first two observations are not included in these regressions. Redoing the estimation without the BOXRES variable produced the same pattern of results. All the variables which are significant in equations 1-1 to 1-5 remain so and conversely.

15. Stigler |1982, 4~: "...if on first hearing a passage of his you are inclined to disagree, you are reacting inefficiently; the correct response is to say to yourself: I wonder where I went amiss?"

REFERENCES

American Economic Association. "Biographical Listing of Members." American Economic Review. December 1981 |and other editions~.

American Economic Association. "Report of the Proceedings of the American Economic Association at the Fifth Annual Meeting, Chautauqua, N.Y., August 23-26, 1892." Publications of the American Economic Association, January 1893.

Anderson, Gary M., David M. Levy and Robert D. Tollison. "The Half-Life of Dead Economists." Canadian Journal of Economics, February 1989, 174-83.

Coats, A. W. "The American Economic Association's Publications: An Historical Perspective." Journal of Economic Literature, March 1969, 57-68.