Counting economics PhDs: how many new graduates do U.S. universities produce?

Finegan, T. Aldrich

Despite the many studies of the production of new economists, it is surprising that so little is known about the persistent differences in how many are graduated, as reported by different sources. For the decade from 1996-97 to 2005-06 (our "study decade"), the Survey of Earned Doctorates (SED) reported 11,524 new economics graduates (including those with specialty degrees), compared to 10,812 reported by the Integrated Postsecondary Education Data System (IPEDS) and 8,862 doctoral dissertations listed in the Journal of Economic Literature (JEL). After specialty degrees are dropped, the comparable numbers in general economics are 9,504, 8,534, and an estimate (ours) of 7,856 degrees. So far as we are aware, this is the first paper to explore the reasons for these differences and attempt to reconcile them. (2)

There are several reasons why such differences matter. Given the resources devoted to training new PhDs, the size of entering cohorts, and the quality of graduate education, a larger output implies more efficient production. Second, a better estimate of annual output will permit a better assessment of the balance (or imbalance) between supply and demand in the market for new graduates. Third, for economists who track the output of new PhDs, there is some value in knowing which of the two main data sources (SED and IPEDS) turns out numbers that are too large, which are too small, and where in between an accurate count likely falls. Fourth, this study finds that about 14 percent of the dissertations written each year in general economics are never listed in the JEL, reducing the value of these listings to scholars. Finally, we also find that large programs tend to underreport degrees to IPEDS. Since output is often used as a predictor of program rank, if such programs also underreport output to the organizations and researchers that create program rankings, then the latter may be biased downward for some departments.

In the course of seeking a more accurate count of the doctoral graduates of economics departments, this paper identifies some general economics PhD programs that have been misclassified by IPEDS as business/managerial economics programs, explains why some graduates of interdisciplinary programs and business schools mark "economics" on the SED survey as their primary field of graduate study, and identifies the economics departments with the largest deficits in dissertation listings submitted to the JEL.

In Part I we offer essential information on the data collection procedures of our three data sources. Part II presents a preliminary analysis of the degreecount gaps from these sources, appraises the elusive contribution of interdisciplinary and business doctoral degrees to the SED-IPEDS gap, investigates the undercount of general economics (GE) degrees by IPEDS, and looks for the sources of the huge IPEDS-JEL gap in total economics PhD degrees as well as the smaller but more worrisome JEL deficit in GE degrees. It concludes with an adjusted estimate of the output of GE degrees during our study decade.

Part III summarizes the findings. It also offers recommendations to the NSF that would make the data from the three sources more comparable and improve the accuracy of NSF data on first-year enrollments of PhD students.

The paper concludes with an Epilog that offers preliminary evidence on the extent of underreporting and inaccessibility of dissertations awarded in five other social sciences.

I. Data Sources and Collection Procedures

A. The Survey of Earned Doctorates (SED)

The SED sends its annual survey to institutional coordinators at some 420 research-doctorate-granting institutions. These coordinators are responsible for distributing a form to every student who received a first doctoral degree in any of 279 fields of study during the academic year. The coordinators send the completed surveys to the NSF's survey contractor, the National Opinion Research Center (NORC), for editing and processing. Since 2001, an alternative procedure for completing the survey on line has been available.

The SED reports a 92 percent student response rate, with just 21 unidentified institutions accounting for over half of all nonrespondents. Missing surveys from participating universities are not a problem here because basic information on these students' degrees and fields of study can be secured from university records. (3)

The SED asks graduates to identify both their primary field of dissertation research and their primary field of study for the doctoral degree, but only the latter responses are used in NSF reports. (4) Graduates respond with both written answers and by entering the applicable three-digit code from a two-page list of 279 fields. The only four that pertain to economics are "economics," "econometrics," "agricultural economics," and "business/ managerial economics." This list is problematic in several respects.

While 217 graduates marked "econometrics" as their primary field of study during our study decade, IPEDS reports no degrees conferred in it! At the same time, IPEDS also reports that 314 degrees were conferred in other minor specialties, such as applied economics, international, development, and natural resources-none of which is listed on the SED survey. (5) Presumably most of these graduates mark "economics" when they can't find their own specialty.

We recognize that space constraints on the SED survey prevent listing all specialties there, whereas IPEDS faces no such constraint. Nonetheless, in Part III we offer an alternative quartet of fields that would reduce these problems.

B. The Integrated Postsecondary Education Data System (IPEDS)

Each fall, IPEDS gathers information from institutional coordinators at more than 6,700 schools on numbers of degrees conferred, by level and highly detailed field. Completion of the surveys is mandatory for institutions receiving any federal aid. Unlike the SED, which counts degree recipients, IPEDS draws on administrative data from school registrars. Within the broad discipline of economics, degrees are tabulated under "economics, general" (the degrees conferred by economics departments), two major specialties (agricultural and business/ managerial economics), and six minor ones. (6)

IPEDS allows universities the most time to collect and report degrees conferred in each academic year. Nonetheless, any survey that attempts to count every undergraduate and graduate degree in each of over 1,100 narrow specialties will occasionally run into trouble. Notably, data processing problems for the 1998-99 academic year led IPEDS to withhold publication of degrees conferred by field that year. But these data were made available to researchers, and we chose to include them in our study so as to permit cross-source comparisons for a full decade. Such comparisons suggest that data problems in 1998-99 reduced IPEDS count of degrees in general economics that year by about 70 degrees, or eight percent (see the Appendix).

We also found some cases of program misclassification. Most involved programs housed in business schools that IPEDS reported under managerial economics, when curriculum information from the program's website and administrators convinced us that it was actually a small program in general economics. For more about these programs, see Part II, Section B-2.

The SED and IPEDS are both administered by the National Center for Science and Engineering Statistics (NCSES) of the National Science Foundation. For more information about the design and quality measures of each survey, see the three NSF references at the end of the paper.

C. Listings of Doctoral Dissertations by the JEL

The JEL does not offer (nor actively seek) a census of new economics PhD dissertations. It welcomes all U.S. and Canadian programs, both general and specialty, that wish to participate. But in soliciting submissions by U.S. programs in general economics, it tries harder and has more success.

Early each fall, the AEA's Pittsburgh Office sends e-mail invitations to all participating programs on its invitation list. The Fall 2011 list contained 161 programs at 138 North American universities; 145 of these programs were at the 124 U.S. universities on the list. Twenty-two of these invitations went to major specialty programs-12 in agricultural economics (AE), 10 in business, management, and finance (BMF). The other 123 went to programs in general economics (GE) and a small number of minor specialty programs (such as applied and international economics).

If the initial request to a program in general economics (or a minor specialty) elicits no response, more solicitations follow. Eventually the great majority of these programs reply. Even the few that do not will receive additional rounds of e-mails in future years. In contrast, the AE programs on the list that do not respond to the first e-mail receive fewer follow-ups, and the business schools receive none. If either specialty continues to ignore JEL requests for several years, it will be dropped from the list.

It is not clear why or how these 22 major specialty programs first expressed an interest in submitting at least some of their dissertations to the JEL. In the case of business schools, perhaps having an economics department on the same campus-and the opportunity to share in the teaching of some graduate courses-played a role. The absence from the list of any business school at a university without an economics Ph.D. program supports this conjecture.

A further constraint on the number of degrees that participating business schools submit (and perhaps on which ones choose to participate) is the implicit understanding that only those theses with an economics foundation are appropriate for publication by the JEL.

The net result of these differences is that dissertations from both major specialties are grossly underrepresented in JEL listings. Later, in Part II-E, we show that degrees in general economics (GE) are also underreported, but not as much. In practice, the completeness of each program's submissions depends in large measure on the timely efforts of overworked departmental administrative assistants and assistants to the directors of graduate studies, many of whom suffer from "survey fatigue."

Despite these shortcomings, JEL listings of dissertations have two important advantages over the more complete degree counts of SED and IPEDS. First, JEL listings and the JEL codes under which they appear provide the only published overview, by broad field, of the content of doctoral research being done in this country. Second, the JEL's strict requirements that every dissertation entry report its author, full title, a JEL code, the conferring university, and the month and year when the graduate's degree was conferred all combine to virtually guarantee its authenticity. Yet, Siegfried and Stock ran across a few listed graduates in two graduation cohorts who had not completed their degree. One adviser reported, "he has not earned his PhD and never will!" (7)

II. Empirical Findings

A. Introduction

Table 1 assembles annual data from SED, IPEDS, and the JEL on economics doctorates conferred by U.S. universities for AYs 1997 through 2006-our study decade, for short. (8) The differences in grand totals (including all specialty degrees) for the decade from the three sources are substantial.

The SED-IPEDS gap, the smallest of the three, is 712 degrees. When we compare the two distributions by program kind in Table 1, we see that the two sources report nearly identical numbers of degrees in business/managerial economics, while IPEDS's tally of agricultural economics degrees is 150 degrees larger. If the 217 degrees that SED reports in econometrics were the only minor specialty graduates in its overall count, then the SEDIPEDS gap in general economics (GE) would appear to be nearly one thousand degrees (970, to be exact). That is the difference between SED's tally in "economics" and IPEDS' tally in "general economics."

But that inference assumes no classification errors in either count. It also overlooks the possibility that some graduates of interdisciplinary programs and business schools might also contribute to this gap. We address these issues in the first two sections that follow.

B. A First Attempt at Reconciling the SED-IPEDS Gap in General Economics Degrees

Here we try to reconcile the GE totals reported by SED and IPEDS. Adjustments are limited to adding, deleting, and reclassifying the degrees in each data source. Much of this reassembly draws on data by university. IPEDS totals and group subset tallies by school are conceptually consistent with the totals and subtotals in Table 1. For SED, however, the only university counts available include degrees in agricultural economics. We attempt to adjust these counts for this inconsistency, but the corrections are only approximate.

1. Adjustments to the SED count of 9,504 degrees in "economics". The SED "economics" group in Table 1 is dominated by graduates of economics departments, but it also contains most graduates of the minor specialty programs that IPEDS combines into a separate category. In the same table, IPEDS shows a decade tally of 314 such degrees, but only 293 came from universities reporting at least one doctorate to at least one data source in this study. To avoid an over-correction, we further reduced SED's specialty program degree count to 246, so that the sum of specialty degrees at each university did not exceed the total number of economics graduates reported for that school by SED.

Given that the graduates of minor specialty programs have no appropriate "box" to check on the SED survey, they presumably check "economics" instead. So the SED and IPEDS tallies of general economics degrees in Table 1 would be more comparable if we reduced SED's "economics" total by these 246 specialty degrees.

Next, as explained earlier, the 217 graduates that the SED lists in "econometrics" must have earned their degrees elsewhere. Since the other listed specialties are unlikely homes, these graduates probably graduated from plain vanilla economics departments. So we move them from econometrics to economics.

Finally, we found 31 universities with at least one economics degree reported by SED during the decade, but none by IPEDS in "general economics". These schools fall into three subsets.

(a) The first contains 22 small to mid-size universities for which SED reported only one, two, or three economics graduates. So far as we can tell, none of these schools had a doctoral program in any kind of economics during that decade. (9) Since SED logged 30 economics degrees for all 22 combined, we deduct an additional 30 degrees from the SED "economics" total in Table 1.

(b) There are another six schools with programs that, based on their website information, we consider "applied economics." Together they awarded 83 degrees, according to SED. (10) Because none of these programs was classified as general economics by IPEDS, we also subtract their degrees from SED's count in "economics."

(c) The last subset consists of three schools for which SED reported 41 degrees, very likely in "economics," but which were actually earned in specialty programs. (11) The members of this trio were Cal Tech, with a multidisciplinary PhD in Social Sciences that includes a highly ranked track in economics; the University of Missouri at Kansas City, with a long-standing program in institutional and post-Keynesian economics; and the University of New Orleans, with a specialty program in financial economics. We also deduct these degrees from SED's total in "Economics."

2. Adjustments to IPEDS' tally of 8,534 degrees in general economics. There are just three. The problems leading to the first two were discussed earlier in Part I, Section B.

First, owing to tabulation problems in AY 1999, we would increase IPEDS' unpublished estimate of GE degrees that year by 70. (See the Appendix for the derivation of this estimate.)

Second, seven universities had programs that IPEDS classified as business/managerial economics, but we consider general economics. With estimated study decade output in parentheses, these were Auburn (30 degrees in 1997-2001, program then closed), Georgia (35), Miami of Florida (13), Oklahoma State (44), Tennessee (38), West Virginia (56), and Wyoming (21). The combined total is 237 degrees, which we add to IPEDS' GE count. (12)

The last adjustment involves 19 dissertations reported to JEL during AYs 2005 and 2006 from a start-up GE program at Clemson University. These also belong in IPEDS' GE column.

3. To sum up. The results of these adjustments are to reduce the SED count for "economics" from 9,504 degrees to 9,321, while increasing IPEDS' count for "general economics" from 8,534 to 8,860. The net effect is to narrow the published SED-IPEDS GE gap from 970 degrees to 461. (13)

C. Do New Doctorates from Interdisciplinary Programs and Business Schools Add to the SED-IPEDS Gap?

We already know that a newly minted PhD does not need a degree in economics to mark "economics" as his or her major field of study on the SED survey. Might enough graduates of interdisciplinary programs and business schools do so to contribute significantly to the SED-IPEDS gap?

There are at least six interdisciplinary (ID) specialty programs that typically include some empirical economics in their curriculum: area studies (e.g. Latin American), public policy analysis, urban studies, labor and industrial relations, human resources management, and international business. Fifty of the 102 universities that reported GE degrees to all three of our data sources also had a doctoral program in one of these ID fields. During our study decade, these universities conferred 2,440 doctorates in all six fields combined, along with close to 7,000 economics PhDs.

Likewise, nearly all doctoral programs in business require work in business finance that includes some economic theory and applications, and students who write dissertations in business finance learn much more. It seems likely that some of these graduates might choose economics as their major field. (Financial economics is not listed on the survey, although finance is.)

Seventy of the 102 universities with PhD-granting programs in GE also housed a doctorate-granting business school, and together they conferred a whopping 6,460 degrees in business, management and finance (BMF)-minus those in managerial economics, which is an economics specialty, and in accounting.

Unfortunately, we cannot identify (and count) the specific graduates in these programs who adopted economics as their primary field of graduate study. But we can see whether universities with more graduates from these programs had larger SED-IPEDS surpluses (or smaller deficits). (14)

To appraise the evidence, we employ an OLS multiple regression using the 102 universities as observations. The dependent variable is the university's adjusted SED-IPEDS gap in GE doctorates. The main predictors are two sets of dummy variables. The first contains four size groups of ID degrees; the other, four size groups of BMF degrees. A school is coded "1" if the number of degrees it conferred of a particular kind fell in the specified size interval, zero otherwise. For each set, schools with no such program are the reference group. (15)

We have added three independent variables. The first is QJEL, the number of dissertations from each university that were listed in the JEL during our study decade. This variable serves as a control from a source unrelated to SED and IPEDS on the output of new degrees at each economics department. (We discuss shortly why departmental output might matter.)

The second is AG, a dummy variable equal to 1 if the university also had a program in agricultural economics (AE), zero otherwise. As noted earlier, we adjusted SED-reported degree counts downward by the output of AE degrees, as reported by IPEDS. Unfortunately, this downward adjustment was, on average, about ten percent too large. Since AG is intended to capture how this over-adjustment influenced GAP, its expected sign is negative.

The last addition is another dummy variable, USP, coded one if the school had a minor specialty program, and zero otherwise. Since none of these programs is listed on the SED survey, we expect most graduates to choose "economics" instead. The result is to overstate the SED-IPEDS GE gap in the six universities of our sample with such programs, so USP should have a positive sign.

The results of the multiple regression appear in Table 2. The intercept predicts that a university with no specialty program and a very small GE program would have a SED deficit of four degrees. Most ID size groups have positive coefficients, but these are unrelated to group size and have by-chance probabilities that average 70 percent! The BMF coefficients rise with group size from "Small" to "Mid-Size" to "Large", which is barely significant at five percent-only to plummet to nearly zero for the 18 schools with an "Extra Large" number of these degrees. That plunge suggests that numbers of degrees are not the whole story here.

The extent to which economic theory is integrated into the business school curriculum and members of the economics department share in teaching its courses may be a more important influence on how many BMF graduates mark "economics" on their SED surveys. Scanning the entire list of business schools in this regression, we see that most of the elite programs have relatively large SED-IPEDS surpluses, and several are known for the important role of economic theory in the curriculum. So a careful study might find that the graduates of such programs do account for a small but significant fraction of the overall SED-IPEDS gap. (16)

AG and USP have the predicted signs but are not statistically significant. QJEL is the only highly significant predictor. Its 0.072 regression coefficient (with a t-value of 4.00) tells us that an economics department reporting one hundred additional dissertations (ten more per year) would be predicted to have a SED surplus larger by seven degrees.

Why should the spread between SED and IPEDS counts of GE degrees widen systematically with program size? One possibility is that larger economics PhD programs, which are also more likely to be higher ranked, tend to attract more ID and BMF graduates to mark "economics" on their SED surveys. Alternatively, as the output of new economics degrees rises across universities, a few more of these economics graduates may be recorded by SED than by IPEDS. While both conjectures may have merit, we think that the second is more important empirically.

The first story would be weightier if the empirical link between numbers of ID and BMF degrees and the SED-IPEDS gap were not so weak. If QJEL is picking up mostly specialty-program graduates, how could the additional number marking "economics" at a school be so significant, yet so small relative to additional department output, when the total number of specialty degrees varies so widely across universities and has so little association with GAP? (17) In other words, how could the output of economics degrees matter so much when the number of specialty degrees matters so little?

On the other hand, there is a more plausible reason why larger economics departments might be associated with larger SED surpluses of their own graduates. As reported earlier, degree information about the few graduates who do not return SED surveys to institutional coordinators can be obtained from internal university records. In contrast, so far as we know, IPEDS has no fallback source of information on the few new completers that graduate school registrars may have missed. So even a slightly higher reporting rate by SED would result in a larger SED-IPEDS gap in larger economics departments.

The regression coefficient for QJEL is robust to whether or not the ID and BMF predictors are included in the all-university regression, but not to large changes in the number and composition of schools within it. When we dropped all 70 universities with business schools, the QJEL regression coefficient fell to 0.027 and lost statistical significance (t = 1.14). That might be seen as evidence that business school graduates were making most of the economics "adoptions" on the SED. But it turns out that the remaining 32 schools have mostly smaller economics departments and much smaller variances in both output and GAP. (18) So comparing the results of the sub-sample run with those of the overall regression cannot inform us about the separate effect of either business schools or program size on the SED-IPEDS gap.

Conclusion. For every ten economics PhDs conferred by U.S. universities during 1997-2006, the same universities also awarded about six doctorates in business and 2.3 in interdisciplinary programs with some economic content. Our multiple regression finds only marginally significant evidence of a positive association between the number of BMF degrees that a university conferred and the size of its SED-IPEDS gap in general economics, and hardly any association between its output of interdisciplinary degrees and that gap.

We do find a highly significant positive association between the number of economics doctorates from a university (as measured by its JEL listings of dissertations) and its SED-IPEDS gap. Most of the additional surplus of SED-counted degrees probably consists of graduates of economics departments. While lack of supporting evidence precludes assigning shares by source to this flow, later we offer a conservative estimate of how many of these degrees can be imputed to those departments (see Part II-G).

D. The Huge IPEDS-JEL Gap in All Program Degrees

Much larger but less puzzling than the SEDIPEDS surplus in general economics degrees is the nearly 2,000 degree chasm between the all program total degree count from IPEDS and the JEL dissertations count. The respective ten-year sums, before any adjustments, are 10,812 and 8,862. About two-thirds of this difference can be attributed to the under-representation of specialty program dissertations in the JEL. Whereas IPEDS recorded 2,278 PhDs from specialty departments (both major and minor) during the study decade, we estimate that only about 1,000 of their theses appeared in the JEL. (19) The reasons for this under-representation were explained in Part I-C.

The remaining third of the difference resides in the gap between IPEDS ' count of general economics degrees and our estimate of GE theses published by the JEL. To that we now turn.

E. The Smaller IPEDS-JEL Gap in Degrees from Economics Departments

Over our study decade, IPEDS reports that 8,534 doctorates were conferred in GE, whereas we estimate that 7,856 of the dissertations listed in the JEL were earned in economics departments-a difference of 678 degrees. As a shortfall in total GE degrees earned, this difference is surely too small.

Earlier we found that IPEDS missed about 326 GE degrees. Adding them to its tally would raise it to 8,860 and the JEL deficit to 1,004. On the other hand, the adjusted gap is only approximate because we were able to obtain an actual count of JEL submissions by kind of program only for academic years 2006 through 2009. (20) During these four years, 88.6 percent of all JEL submissions from U.S. universities came from economics departments. Assuming that the same fraction applied to earlier years, we applied it to the total number of JEL listings for 1997-2005, and then tacked on the actual tally for 2006. That yielded the spuriously precise estimate of 7,856 GE dissertations.

While a substantial margin of error attaches to any estimate of the deficit of submissions from economics departments to the JEL, any in the vicinity of one thousand theses over a decade calls for an investigation of its causes.

F. Why Is There So Large a Deficit in JEL Submissions, Yet Two Subsets with Surpluses?

1. The net deficit. In light of the collection efforts described in Part I-C, the source of the deficit would seem to lie with some participating departments that fail to report all of the degrees that they conferred in the previous academic year. Table 3 lists the eleven universities with JEL submission deficits of 20 degrees or more, compared to IPEDS' count of GE degrees, during AYs 1997-2006. (21) All together, these schools evidently approved at least 350 dissertations that were never listed in the JEL.

Nearly all these doctoral programs are well known. Five were ranked in the top two tiers of the NRC's 1993 quality of faculty rankings, and three in the third tier (National Research Council, 1995).

Northwestern's eye-catching deficit of 81 degrees is so large that we attempted to look into it. By sheer luck, we found an unpublished report on the economics department's website that tabulated the economic progress of the 230 students who had entered its economics PhD program from fall 1994 through fall 2003. Two-thirds of them (154) had received their degree by 2010, and 80 percent of the completers had finished in five or six years. (22) Had Northwestern's program been in steady-state equilibrium, it would have been enrolling about 23 students a year and graduating about 15. Given the considerable overlap in the entering classes of this decade-long cohort and those who graduated in 1997-2006, it is very unlikely that Northwestern would have had only 67 dissertations to report for the latter decade. The IPEDS count of 148 appears to be much nearer the mark. (23)

Part of the 350-degree JEL deficit in Table 3 might result from an IPEDS over-count of these degrees--due, perhaps, to errors in tabulation or classification. Were that true, we would expect to find smaller deficits if we used adjusted SED counts of economics degrees instead of IPEDS GE counts as a benchmark. (The two sources use such different counting procedures that sizeable errors in the same direction by both sources at the same school are highly unlikely.)

It turns out that at six of these universities, the re-estimated JEL deficit was either the same or differed by only one or two degrees. In all of the other five, the SED-based deficits were larger than the originals-by four degrees at Hawaii-Manoa, six at Duke, 14 at Yale, and 27 at both Northwestern and Penn. (24) Therefore, the IPEDS over-count hypothesis is not supported.

Besides the eleven schools with a combined JEL deficit of 350 degrees (relative to IPEDS), another 17 reported deficits of 10 to 19 degrees each and a total of 223. For the two lists combined, the total deficit is 573 degrees.

2. Unexpected JEL surpluses. That there are subsets with JEL surpluses may come as a surprise. In fact, there are thirteen universities with surpluses of 20 or more dissertations (relative to IPEDS GE counts) and a combined surplus of 674 degrees! (25) The explanation is that twelve of these universities had specialty programs in agricultural economics, three had business schools, and two had both. All were solicited by the JEL, and all delivered dissertation lists. Since IPEDS GE counts excluded these specialty degrees but SED counts did not, the JEL surplus is easily explained.

The presence of large surpluses at some universities does not offset or mitigate shortfalls at others, since the two imbalances arise for different reasons. Nor does a surplus imply full reporting of theses by the economics department at that school. It tells us only that any underreporting of GE theses is more than offset by the addition of specialty degrees.

As expected, most universities with no JEL-solicited specialty degree programs have JEL deficits in GE degrees, although many are quite small. Yet we found eight such schools with JEL surpluses totaling 48 degrees. It is highly doubtful that these surpluses reflect over-reporting of degrees by economics departments to the JEL. A shortfall in the IPEDS count of GE degrees is far more likely.

Were that so, the adjusted IPEDS-JEL degree gap would widen by 48 degrees, while the adjusted SED-IPEDS degree gap would narrow by the same margin.

G. How Many General Economics Degrees Were Conferred During Our Study Decade?

Our estimate is 9,128 degrees. Table 4 shows the baseline IPEDS' GE total from Table 1 and lists the specific adjustments leading to this estimate. All but the last were explained earlier in the paper. The final adjustment is an extension of our findings in Part II, Section C, on the elusive contribution of new graduates from interdisciplinary programs and business schools to the SED-IPEDS gap in GE degrees.

The reader will recall the puzzle discussed in that section about the origin of the additional graduates in that gap associated with larger economics departments, as measured by QJEL. We concluded that most of these graduates had degrees from economics departments, rather than from ID programs and business schools; but we could not assign shares by source. We can, nonetheless, derive a conservative estimate of the number of additional graduates from GE programs picked up by SED but missed by IPEDS.

The starting point is the regression coefficient for QJEL of 0.072 (t=4.00) shown in Table 2, which predicts that an additional hundred theses reported to the JEL would be associated with a SED-IPEDS gap larger by 7.2 degrees. The range of QJEL values across the 102 universities in our sample stretches from 2 to 326, but we would not expect this coefficient to apply to the entire range. Below some level, the number of degrees is so small that IPEDS is unlikely to miss any of them. And above some level, output is so large and variable that the association between degrees and gap is unpredictable. There, BMF degrees may fill part of the gap.

So we have constrained the interval of QJEL values in this test to run from 30 degrees to 149, where the mean of the full distribution is 81.6 and the standard deviation is 68.2. Two-thirds of the 102 schools fell within this interval, and together they conferred 4,586 PhDs during the study decade.

Finally, we assume that only two-thirds of the additional 7.2 SED degrees associated with 100 additional JEL dissertations came from economics departments. Hence, the additional number of IPEDS degrees imputed to these departments works out to 220 (0.667*0.072*4,586=220), an increase of about five percent.

This final adjustment in Table 4 is the second largest and the most tentative, and its inclusion makes the final adjusted total degree count equally tentative. Further, this total applies to a decade that ended seven academic years ago. As a guidepost for approximating similar adjustments in later years, our estimate of GE degrees lies about three-fifths of the way between IPEDS' smaller reported total for "general economics" and SED's larger reported total for "economics".

III. Findings and Recommendations to the NSF

A. Findings

1. During the decade from AY 1997 to AY 2006, our three data sources reported substantial and persistent differences in the numbers of economics PhDs conferred. These variations arise from differences across surveys in their scope and objectives, their sources of information and response rates, and their classification frameworks and procedures. Each survey faces challenges but also has unique advantages.

2. Before adjustments, SED reported 970 more degrees in (core) "economics" than IPEDS reported in "general economics." After adding, deleting, and reclassifying degrees in each source, the SED surplus shrinks by about one-half, to 461 degrees.

3. There is no systematic, statistically significant evidence that universities conferring more doctorates in interdisciplinary specialty programs with economic content or in business schools have larger SED-IPEDS surpluses. But there is fragmentary evidence of larger surpluses at universities with elite business schools where economic theory is integrated into the curriculum. Some graduates of such schools may therefore be marking "economics" on their SED surveys as their primary field of graduate study.

4. As one moves up the output ladder of economics departments as measured by JEL dissertations, an additional 100 listed theses during our study decade is associated with a SED-IPEDS gap that is seven degrees wider. The most likely explanation for this highly significant association is that SED has backup information on those few graduates who do not return surveys, whereas IPEDS lacks such information on the few graduates who are overlooked by graduate registrars.

5. We estimate that only about 11 percent of all JEL listings are from specialty economics programs, compared to 18 percent of SED degrees and 21 percent of IPEDS degrees. The main reason is that while the JEL actively seeks a complete listing of new dissertations from all economic departments, it solicits submissions only from those specialty programs on the same campus that have shown an interest in participating.

6. At least one fourth of economics departments underreport new dissertations to the JEL. The eleven departments with the largest deficits relative to IPEDS-GE accounted for 35 percent of the total shortfall of about one thousand degrees during 1997-2006. (IPEDS' count also fell short, as the next finding reveals.) The loss is large enough to reduce the value of JEL lists to users. The value of EconLit is also reduced, because only the dissertations listed in the JEL are included in EconLit's database. Since full reporting depends on the voluntary efforts of overworked departmental staff, a solution to this problem is not obvious.

7. After adjusting IPEDS data for various omissions (see Table 4), we estimate that U.S. universities conferred 9,128 degrees in general economics during our study decade. This total lies about three-fifths of the way between the smaller reported total from IPEDS (8,534) and the larger reported total from the SED (9,504).

B. Recommendations to the National Science Foundation

1. Listing economics specialties on the SED Survey. As noted earlier, the SED lists only four fields of economics degrees on its survey: "Economics" and "Econometrics" under Social Sciences; "Agricultural Economics" under Agricultural Sciences/Natural Resources; and "Business/Managerial Economics" under Business Management/ Administration. We recommend (a) that "Economics" be renamed "General Economics"; (b) that "Econometrics" be replaced by "Other Economics Specialties," and (c) that all four fields be listed under Social Sciences, and in the following order: "General Economics", "Agricultural Economics", "Business/Managerial Economics", and "Other Economics Specialties".

Econometrics should be dropped because no degrees have been conferred in that specialty for over two decades, while other minor specialty programs turn out 20 to 45 graduates per year who are left homeless on the survey. The suggested changes would make the major specialty degrees easier for their graduates to find on the survey, would create an "other" category for the real minor specialties, and would increase the comparability of IPEDS and SED data on how many economics PhDs are earned in economics departments.

2. Improving data on first-time full-time (FTFT) enrollments of graduate students, by discipline. (26) These statistics report numbers of first-year students entering graduate programs in a large number of disciplines, including economics and agricultural economics, after deducting part-time students and those who entered a graduate program at the same university earlier. (The deductions are very small for doctoral students in economics.) Unfortunately, these data do not distinguish between first enrollments in Master's degree programs and in PhD programs. As a result, lagged FTFT enrollments are a flawed predictor of changes over time in the supply of new PhDs.

According to an NSF project officer, the rationale for aggregation is that many students beginning graduate studies in a field do not know how far they want to pursue them, which is true. Nonetheless, the degree offered by a given graduate program, unlike the different (and shifting) goals and plans of the enrollees, is an objective and important fact. Economic research on the production of new PhDs would benefit greatly if the NSF were to disaggregate its FTFT enrollment data by the degree that completers will receive. (27) Otherwise, it is not possible for users to assess accurately the influence of changing macro labor market conditions on first year doctoral enrollments, or the influence of lagged first-year enrollments on the output of new PhDs.

Epilog: Do Other Social Science Disciplines Have Problems Counting PhDs?

Are the problems in counting new PhDs unique to economics, or can they be found in other social sciences? (28) A full analysis would require far more research than could be reported here. Instead, we offer preliminary information on two likely indicators of trouble for five other disciplines. The indicators are (1) the relative size of the SED-IPEDS gap in doctoral degrees, and (2) evidence of underreporting or inaccessibility of dissertation lists. The other disciplines are anthropology, geography, history, political science, and sociology. (29)

A. SED-IPEDS Gaps

In Table 5, we show published tallies of new doctorates in each discipline, as reported by the SED and IPEDS for the study decade of 1997-- 2006, along with the SED-IPEDS gaps from these counts. The results for general economics are included as a benchmark.

Our discipline's SED surplus of 10.2 percent is only slightly larger than history's 9.2 percent, but this similarity masks huge differences (not shown in the table) in the size of subfields in each source. As we showed earlier, there are no major specialties in general economics, and the minor ones are very small. In history, however, SED lists nine specialties and IPEDS seven, and the counts by source for those that overlap seem to come from different worlds. For example, in AY 2006 (a sample year), SED reported 391 new PhDs in "American History," but IPEDS only eight! For "European History," the score was SED 216, IPEDS 1. Yet IPEDS swamped SED in "History, General," 790 to 59.

The explanation seems to be that most PhD graduates in history chose one of the subfield specialties on the SED survey, whereas most history departments don't offer subfield specific degrees. Further, the IPEDS contact person for history at a university is unlikely to know the area specialties of history graduates, and so she assigns the "History, General" code to most of them. (30) Fortunately, these SED-IPEDS inequalities are largely offsetting, so they do not signal counting troubles for the discipline. But we have no explanation for the nine percent residual surplus that remains. (Perhaps there are a few interdisciplinary degree programs where a subfield of history is part of the curriculum and where some graduates with theses in that subfield chose it as their primary field of study on the SED questionnaire.)

In contrast to economics and history, Table 5 shows nearly equal counts of new PhDs by SED and IPEDS for sociology and anthropology (with gaps of 1.6 and -0.9 percent, respectively). An important reason is that SED does not list a single subfield in either discipline on its survey, while the two listed by IPEDS in anthropology (physical and other) are virtually ignored, with counts near one percent.

The last two fields listed in Table 5--political science and geography--have unexpected SED deficits near five percent. Why?

Consider political science (PS) first. Here, most doctoral programs require work drawn from several areas, the most common of which are American politics, comparative politics, international relations, models and methods, and political theory. (31) What distinguishes international relations is that it is also listed as a separate social science by SED and IPEDS, at the same time that five PS departments have international relations (or studies) in their names. (32) If enough new PhDs in PS mark the separate discipline of international relations as their primary field of study on the SED survey, while the graduate registrars at these universities report them as PS doctorates to IPEDS, they could produce the observed 279-degree SED shortfall. Nonetheless, there could also be a "flow" of graduates in the opposite direction-from international relations to political science. It is unclear on a priori grounds which flow should be larger.

Finally, geography is a small discipline that graduated only about 1,800 PhDs over 1997-2006. The National Research Council (2011) reports four subfields for it: physical and environmental geography, human geography, nature and society relations, and geographic information sciences. Despite their large differences in content, none appears on either SED's or IPEDS' list of subfields. Nor could we find a listed SED subfield that looked potentially more attractive to some geography graduates than geography itself. So the small SED deficit here is a mystery.

B. Evidence of Underreporting and Inaccessibility of Degrees

Perhaps the most important finding of this study is the substantial underreporting of doctoral dissertations (DDs) by a minority of departments of economics. Do other social sciences face a similar problem? Table 6 compares dissertation counts for each of our disciplines (again including economics as a benchmark) with the number of doctorates reported by SED and IPEDS for a sample of academic years in the mid-2000s. We wanted to use 2005, 2006, and 2007 for all disciplines, but data problems led to the loss of 2005 for political science and a decision to add 2004 for history. (33) Disciplines are listed in rank order of the average percent of dissertations received (i.e., their "yield," for short) for all sample years combined.

Scanning these average yields reveals that sociology and geography had nearly complete returns (about 98 percent), while anthropology garnered only 77 percent. In the middle were history, economics, and political science, with yields of 91, 88, and 84 percent, respectively. These are substantial differences. But a further look at the data tells us that a discipline's average yield is a flawed measure of how well its dissertation lists are likely to encourage additional research.

First, these averages conceal large differences across years within some major fields, casting doubt on the reliability of their data. In geography, for example, the yield drops from 111 percent (a surplus!) in 2005 to 93 percent in 2006 and 87 percent in 2007. Likewise, the yield in political science plummets from 97 to 72 percent between 2006 and 2007. Why is this statistic so unstable in some fields?

We have no answer for geography, but here is at least a partial one for political science. This is the only social science discipline that collects new dissertations only by calendar year, and currently reports them biennially for both adjacent years combined. The entries, which include the year of completion, are published by PS in a single list early in the following year. The problem is that the respondent is asked to submit the results of two years' surveys in one calendar year.

For example, early in 2013, directors of graduate studies were asked to submit information on DDs completed in 2012 (a tight fit, but doable). However, the call for dissertations completed in 2013 went out early in the fall of that year, before the titles of many dissertations completed in the last quarter of the year were known. The expected result will be the omission of many of the latter theses from the forthcoming list in 2014 and a decline in the yield of DDs between 2012 and 2013. (34)

The most recent report on new dissertations in PS, which appeared in April 2012, reveals a 14 percent drop in the count of new dissertations between 2010 and 2011, compared with only a four percent decline in PhDs. If the same timeline of data collection was in place back in 2007, that would help to explain at least some of the 30 percent drop in dissertations reported between 2006 and 2007 in Table 6, compared with a five percent decline in degrees. (35)

There may be some benefits to ASP A from this data collection policy, but it clearly imposes costs on users, who are able to gain access to information on new dissertations only every two years. There is also a question of how many late year dissertations not reported in odd years are belatedly submitted the following year, or perhaps between surveys, as APSA's instructions invite. But some permanent attrition seems inevitable, since directors of graduates studies have more important things to do.

Another limitation of the reported yields in Table 6 is that they do not take into account either the benefits to the profession of DD lists that go back many decades (like those of history and economics), or the losses from gaps in reporting (like those in political science during the 2000s). While recent research is more relevant than what came earlier, the use of a source is encouraged by predictable availability. For historians, not surprisingly, annual information on DDs can be found back to 1882!

The content and organization of citations also matter. While the content of the typical entry varies little across disciplines, the inclusion of the principal adviser's name, as most entries for geography and some from history do, helps researchers contact graduates who have left the campus. More importantly, for a DD list to be to be useful, it needs to be organized by subfield, preferably at several levels, as the JEL codes of economics provide. Anthropology, geography, and sociology fall short on this count. Anthropology identifies the subfield in which each dissertation falls, but only within each listing (American Anthropological Association, 2008). Would not a list assembled by subfields be much more useful?

The accessibility of a discipline's list is probably its most important characteristic. Among social sciences, history is the exemplar (American Historical Association, 2013). With over 29,300 dissertations in its data bank as of October 2013, history's on-line search engine allows creation of subset lists for any year--and for all years that precede or follow it--organized by 19 major categories, 126 specializations, and 352 narrow fields. Users can also search by university, by keywords in a title, and by author and principal advisor (although some advisors are missing).

The most basic accessibility that a dissertation researcher requires is that a discipline combine the dissertations from all responding departments in a single list, organized at least by broad subfields. Surprisingly, neither geography nor sociology does so. Instead, these disciplines report new dissertations in their annual guide to departments and programs -but only by department, listed alphabetically by state and university/6 It helps that all of geography's dissertations appear in a separate section of its handbook, whereas those in sociology are part of the baseline information presented for each department offering graduate degrees. As a result, someone wishing to search one year's complete harvest of dissertations in geography need go through only about 35 pages of entries. In contrast, a researcher with the same task in sociology (with its larger harvest and less user-friendly format) must scan about 300 pages of departmental information. (37) Unfortunately, useful information by subfield cannot be presented efficiently for individual departments.

Departmental lists may nonetheless serve interests that departments value more than assisting searches! For example, they may help to recruit graduate students by showing how many complete the program, and with research on what topics, in a sample year. Further, if they have tangible benefits, departments may work harder at reporting all new degrees. Still, nothing prevents a discipline's association from producing a composite list from those it reports by department, and that list would attract far more attention.

We close with an important qualification that leads to a multidisciplinary challenge. The extent to which unreported or inaccessible dissertations inhibit new research within a discipline depends on the availability and quality of alternative sources of new findings. In this context, abstracts of dissertations surely trump titles! In economics we found that theses never submitted to the JEL were also lost to our main source of dissertation abstracts, EconLit. Compounding that loss is another one: nearly half of the dissertations that are delivered to the JEL do not include the requested abstract, which would also have been sent to EconLit. But other social science associations may receive abstracts from other sources, including graduate schools and new graduates themselves. We must leave it to scholars in these fields to see what additional sources are available to them.

APPENDIX

This technical note explains the basis of our conclusion that IPEDS' unpublished estimate of 773 PhDs conferred in general economics in AY 1999 was about 70 degrees too small.

As reported in the text, owing to processing problems IPEDS withheld publication of degrees awarded by discipline in that academic year. These data were, however, made available to researchers; and we have included the (uncorrected) values in Table 1. Visual inspection of the data for SEDE-con, IPEDS-Econ, and total JEL for AYs 1998, 1999, and 2000 leaves little doubt that the IPEDSE-con estimate for the middle year is too low. The 15.4 percent drop from 1998 to 1999 is much larger than the declines reported by SED-Econ (6.7 percent) and JEL (4.7 percent). Likewise, the 7.5 percent rebound from 1999 to 2000 by IPEDS-Econ compares with a 2.4 percent bounce by SED-Econ and a continued decline of 2.2 percent by JEL.

One way of estimating a corrected estimate of IPEDS-Econ in 1999 would be to take an average of the reported declines in GE degrees by SED and JEL between 1998 and 1999 (namely, 5.7 percent), and assume that IPEDS-Econ's actual output in 1999 was 0.943 times the reported output of 914 in the previous year, or 862 degrees.

Alternatively, one could take the average of the percent changes in the reported outputs of the two benchmark sources between 1999 and 2000, which was a gain of 0.1 percent, and use that to solve for the value that IPEDS would have reported for 1999, if that number had risen by 0.1 percent from 1999 to 2000. That estimate is 830 degrees.

Since neither estimate of the corrected figure for IPED-Econ in AY 1999 rests on stronger a priori premises, we take the average of them, or 846, which is 73 more than IPEDS reported. To avoid spurious accuracy, the adjustment is rounded down to 70 degrees.

REFERENCES

American Anthropological Association (AAA). 2006, 2007, 2008. The AAA Guide to Departments of Anthropology. "PhD Dissertations in Anthropology." Arlington, VA: AAA.

American Economic Association (AEA). "Doctoral Dissertations in Economics." Journal of Economic Literature. December issues, selected years. Pittsburgh, PA: AEA.

American Historical Association. Website of the "Directory of History Dissertations" (www .historians.org/pubs/dissertations), last accessed 6-26-13.

American Political Science Association (APSA). 2008, 2012. "Doctoral Dissertations in Political Science." PS: Political Science and Politics 41(2): 444-468, and 45(2): 364-394. Washington DC: APSA.

American Sociological Association (ASA). 2006, 2007, 2008. Guide to Graduate Departments of Sociology. Washington DC: ASA.

Association of American Geographers (AAG). 2006, 2007, 2008. Guide to Geography Programs in the Americas. "Titles of Theses and Dissertations Completed." Washington DC: AAG. (Note: publication of the printed version of the Guide is soon to be replaced by free website access at (http://www.aag.org), last accessed 1/11/2013.)

National Research Council. 1995. Research-Doctorate Programs in the United States. Washington DC: National Academy Press.

National Research Council. 2011. A Data-Based Assessment of Research-Doctorates in the United States. Washington, DC: National Academies Press.

National Science Foundation (NSF), National Center for Science and Engineering Statistics (NCSES). 2006. U.S. Doctorates in the 20th Century, Special Report NSF 06-319, Appendix C, "Technical Notes," subsection on "Data Sources," pp. 5-8. Available on NSF website. (http://www.nsf.gov/statistics/nsf06319/appc.cfm), last accessed June 2013.

National Science Foundation, NCSES, Survey of Earned Doctorates. 2006. Sections 2 and 3. Available on NSF website http://www.nsf.gov/ statistics/srvydoctorates/

National Science Foundation, NCSES, Integrated Postsecondary Education Data System Completions Survey. 2006. Sections 2 and 3. Available on NSF website: http://www.nsf.gov/ statistics/srvyipeds/

Northwestern University, Department of Economics. 2000. "Graduation Rates." Graduate Connection 16(1): 7-8. Retrieved April 2012 from departmental website (http://www.econ.northwestem .edu/phd/graduate-connection).

PhDs.org. 2013. Author's tabulation of names of departments conferring PhDs in political science. Retrieved September 2013.

Siegfried, John J., and Wendy A. Stock. 1999. "The Labor Market for New Ph.D. Economists." Journal of Economic Perspectives, 13(3): 115-134.

Siegfried, John J., and Wendy A. Stock. 2004. "The Labor Market for New Ph.D. Economists in 2002." American Economic Review, 94(2): 272-285.

Notes

(1.) I thank John Siegfried for helpful comments, Chris Bennett for econometric advice, Elizabeth Braunstein for JEL dissertations data, Mark Fiegener and Paulette Sears for information on SED and IPEDS degrees, Mary Ann Gilbert for ProQuest data and information, Doug Quint for information on EconLit, and Zhongzhong Hu for excellent research assistance. The author bears sole responsibility for all errors, conclusions, and recommendations.

(2.) Extensive on-line searches with EconLit, Google, and Google Scholar failed to reveal any citations of works that discussed such differences.

(3.) National Science Foundation, Sur\'ey of Earned Doctorates, 2006, Section 3, "Survey Quality Measures," Subsection (c), "Nonresponse."

(4.) We thank Mary Ann Latter, a project officer at NORC, for this information.

(5.) Programs currently entitled "natural resources" often house programs in agricultural economics, but IPEDS continues to classify the two specialties separately.

(6.) The latter (with number of degrees during our study decade in parentheses) are applied (117), development and international development (51), international (68), natural resources (16), pharmaceutical (5), and other (57). A seventh, econometrics and quantitative economics, is also listed but conferred no degrees.

(7.) Information from John Siegfried based on research by Siegfried and Stock (1999, 2004).

(8.) We have added comparable data for the next five years so that the reader can see how little impact the Great Recession of 2008 and 2009 and the weak recovery that followed in 2000 and 2001 appear to have had on the output of new doctorates.

(9.) The 22 are Bowling Green, Drew, Florida Atlantic, Jackson State, Louisiana State, New Hampshire College, New Mexico State, Nova Southeastern, Ohio University, Old Dominion, Rush, Rutgers-Newark, Tufts, U.S. International, Walden, and the universities of Alaska, Central Florida, Maine, Maryland-Baltimore County, North Texas, South Florida, and Texas-Arlington. A few of these may have opened PhD programs since 2006.

(10.) These universities are Drexel, Lehigh, Mississippi State, and the universities of Alabama, Memphis, and Mississippi. Programs in applied economics have some required courses outside the usual core and fewer elective specialties than GE programs.

(11.) None of these programs belonged to earlier subsets for which adjustments were made.

(12.) Except for Auburn, each school's estimated GE output is based on its SED count, minus any degrees in agricultural economics (AE) reported by IPEDS. For Auburn, which closed its GE program in the late 1990s but retained a program in AE, we used the number of dissertations it reported to the JEL over 1997-2001, since no AE theses were submitted during those years.

(13.) We are unable to estimate the effects of the SED-IPEDS deficit of 150 degrees in agricultural economics (AE) on the SED-IPEDS surplus in general economics (GE).

(14.) Bear in mind that while the aggregate SEDIPEDS gap is positive, it was zero or negative for 44 of the 102 schools in our sample.

(15.) A very large variance across universities in the output of each specialty, together with the presence of a good many schools with no program in it, preclude using the actual counts of specialty degrees as a predictor.

(16.) We cannot judge whether the same inference applies to ED programs, because we have no information on their rankings or use of economics.

(17.) The regression coefficient for QJEL "holds constant" the influence of BMF and ID degrees on GAP. But this control is weak when there is so little covariance between the numbers of these degrees and GAP.

(18.) The standard deviations of QJEL and GAP in the all university sample are 68.2 and 10.8 degrees, respectively, compared to 31.5 and 3.3 degrees in the 32-school subset with no BMF degrees.

(19.) Our estimate of roughly 1,000 specialty thesis listings in the JEL is the difference between total JEL listings (8,862) and those imputed to economics departments (7,856), as shown earlier in Table 1. The imputation procedure is explained in the next section. The comparable IPEDS total of 2,278 specialty degrees is the sum of its degrees awarded in agricultural, business/managerial, and minor (smaller) specialty programs, also shown in Table 1.

(20.) Elizabeth Braunstein of the AEA's Pittsburgh office kindly obtained these data for us. Similar data are not available for earlier years.

(21.) To the extent that JEL listings included dissertations from specialty programs, the deficits in Table 5 are understated. Only Hawaii-Manoa and Northwestern had specialty programs that were solicited in 2006-09, and neither reported any specialty degrees. Nonetheless, each might have done so earlier--and so might have a few other schools.

(22.) Source: Northwestern University website, Department of Economics, "Graduate Connection," September 2010, pp. 7-8.

(23.) The inference of a large departmental shortfall is also supported by data from ProQuest, which reported 150 economics PhDs from Northwestern over calendar years 1997-2006. We reported this shortfall (and its basis) to the chair at Northwestern.

(24.) The even larger SED-based deficits at Yale, Northwestern, and Penn might be explained by the unusual sharing of teaching with economics faculty and integration of economics into the curriculum in their schools of management-leading, perhaps, to an unusual number of their graduates adopting economics as their primary field of study on the SED survey.

(25.) With surpluses shown in parentheses, they are UC-Berkeley (100), Purdue (77), Cornell (72), Ohio State (67), Illinois at Urbana-Champaign (58), Stanford (52), Minnesota--Twin Cities (52), Michigan State (41), Washington State (38), UC-Davis (36), Wisconsin-Madison (33), Texas A&M (27), and Harvard (21).

(26.) The source of these data is the annual NSF-NIH Survey of Graduate Students and Post-doctorates in Science and Engineering, accessible to researchers on WebCASPAR, and published in the Digest of Education Statistics.

(27.) Some graduate schools have a division of unclassified studies for students preparing to enter degree programs. That division could be a third category on the NSF-NIH survey.

(28.) I thank an anonymous referee for raising this issue.

(29.) We included history because the National Research Council (NRC) listed it under Social and Behavioral Sciences in 1993. We omitted two others: (a) psychology, because some specialties confer professional rather than research doctorates, reducing SED-IPEDS comparability; and (b) linguistics, because the SED and IPEDS list it under different disciplinary divisions, with the same result.

(30.) We thank Mark Fiegener at the NSF for this explanation.

(31.) Source: National Research Council (2011).

(32.) Count by the author from data in PhDs.org (2013).

(33.) Some history departments report dissertations by academic year, others by calendar year. Use of the latter creates a six-month misalignment between the interval for completion of theses and that for conferring degrees, since SED and IPEDS degree counts are based on AYs. Adding another year (2004) to history's data set mitigates the mismatch. Political science, in contrast, reported dissertations only for calendar years 2006 and 2007, so data from additional years were not available.

(34.) We thank Cindy Kam, Director of Graduate Studies in Political Science at Vanderbilt, for information on these procedures.

(35.) We were unable to reach anyone at APS A headquarters who knew what timeline of data collection was used in earlier years.

(36.) See Association of American Geographers (2006) and American Sociological Association (2006).

(37.) Each sociology handbook contains an alphabetical list of all PhD recipients in the U.S. and other countries during the designated year. Each citation reports the university and program where the degree was earned, but does not give the title of the dissertation or the subfield in which it was earned.

by T. Aldrich Finegan, Professor of Economics, Emeritus, Vanderbilt University. E-mail address: t.aldrich.finegan@ vanderbilt.edu.

Postal address: American Economic Association, 2014 Broadway, Suite 305, Nashville, TN 372032418. Phone numbers: Office, direct line: (615) 322-7016, Office, message: (615) 322-2595, Home: (615) 352-9344.

TABLE 1.
Economics PhDs Conferred, by Data Source, Specialty, and Academic
Year: 1997-2011

 Academic Years in Study Period

Source and Specialty 1997 1998 1999 2000 2001 2002

NSF: SED: Total (a) 1,211 1,214 1,117 1,138 1,131 1,066
 Agricultural 133 156 149 138 154 119
 econs.
 Business/ 48 57 42 52 50 39
 managerial
 econs.
 Econometrics 31 25 15 15 13 14
 Economics (b) 999 976 911 933 914 894

NSF: IPEDS: Total" 1,150 1,162 1,032 1,061 1,066 1,002
 Agricultural 137 178 172 150 165 135
 econs.
 Business/ 45 56 50 59 46 41
 managerial
 econs.
 Minor econ. 28 14 37 21 30 24
 specialties (d)
 General economics 940 914 773 831 825 802

AEA: JEL 966 916 873 854 876 826
 dissertations:
 Total (e)
 General economics 860 810 770 760 780 730
 (f)

 Academic Years in Study Period

Source and Specialty 2003 2004 2005 2006 Total Share
 of
 Total

NSF: SED: Total (a) 1,094 1,128 1,238 1,187 11,524 1.0
 Agricultural 118 110 122 113 1,312 0.11
 econs.
 Business/ 44 59 55 45 491 0.04
 managerial
 econs.
 Econometrics 23 18 30 33 217 0.02
 Economics (b) 909 941 1,031 996 9,504 0.82

NSF: IPEDS: Total" 1,009 1,042 1,171 1,117 10,812 1.0
 Agricultural 131 129 138 127 1,462 0.14
 econs.
 Business/ 38 60 57 50 502 0.05
 managerial
 econs.
 Minor econ. 39 42 37 42 314 0.03
 specialties (d)
 General economics 801 811 939 898 8,534 0.79

AEA: JEL 779 863 994 915 8,862 1.0
 dissertations:
 Total (e)
 General economics 690 760 880 816 7,856 0.89
 (f)

 Later Years

Source and Specialty 2007 2008 2009 2010 2011

NSF: SED: Total (a) 1,217 1,220 1,259 1,220 1,217
 Agricultural 182 111 116 118 107
 econs.
 Business/ 31 18 24 29 26
 managerial
 econs.
 Econometrics 30 36 31 34 28
 Economics (b) 974 1,055 1,088 1,039 1,096

NSF: IPEDS: Total" 1,164 1,225 1,223 1,193 1,218
 Agricultural 166 147 157 140 139
 econs.
 Business/ 56 50 53 54 66
 managerial
 econs.
 Minor econ. 40 46 36 59 49
 specialties (d)
 General economics 902 982 977 940 964

AEA: JEL 956 975 956 944 1,008
 dissertations:
 Total (e)
 General economics 824 868 862 840 890
 (f)

(a) Source: NSF Survey of Earned Doctorates (SED), annual reports.
Totals are author's summations of group totals.

(b) Most of these degrees are in general economics, but the total
includes several hundred from minor economics specialties and a
smaller number from interdisciplinary programs and business schools.

(c) Source: NSF Integrated Postsecondary Education Data System
(IPEDS), annual reports. Totals are author's summations of group
totals. Data for 1999 are unpublished.

(d) These are degrees in the following economic specialties:
applied, development, international, natural resources,
pharmaceutical, and "other."

(e) Source: Author's counts of dissertations from U.S.
universities listed in December issues of the JEL.

(f) Actual counts are available only for 2006-2009; other
years show rounded estimates (see text).

TABLE 2.
OLS Regression Predicting SED-IPEDS Gaps in Output of General
Economics Degrees during 1997-2006 at 102 Universities (a)

Predictor (b) (Range NOBS Coeff. Std. t-stat Prob.
of Degrees) (c) error (d)

Constant term -3.86 * (2.09) -1.85 0.07

ID-FEW (1-3) 7 2.18 (4.22) 0.52 0.61
ID-SML (10-32) 18 0.96 (2.78) 0.34 0.73
ID-MID (37-69) 15 -0.08 (2.94) -0.03 0.98
ID-LGE (82-261) 10 2.76 (3.73) 0.74 0.46

BMF-SML (11-57) 19 1.83 (2.84) 0.65 0.52
BMF-MID (64-84) 13 3.68 (3.26) 1.13 0.26
BMF-LGE (91-120) 20 5.86 ** (2.97) 1.98 0.05
BMF-XLG (125-296) 18 0.27 (3.56) 0.08 0.94

QJEL (2-326) 0.072 *** (0.018) 4.00 0.000
AG (1,0) -1.42 (2.45) -0.58 0.56
USP(1,0) 5.39 (4.28) 1.26 0.21

(a) Dependent variable is the university's SED-IPED difference in
GE degrees conferred (mean = 4.63, S.D. = 10.8, median = 1, minimum
= -7, maximum = 73). Overall results: [R.sup.2] = 0.28, adj.
[R.sup.2] = 0.19, S.E. of Reg. = 9.69, F stat. = 3.19 ***

(b) See text for descriptions of explanatory variables.

(c) NOBS is the number of universities in each size subset
of ID and BMF degrees. Universities with no such program
are the omitted reference groups.

(d) Shows the probability that the observed regression
coefficient would differ from zero by chance.

*** Significant at the 0.01 level, ** at the 0.05 level,
* at the 0.10 level (two-tail tests).

TABLE 3.
Universities with Deficits of Twenty or More
Degrees in JEL Submissions, Relative to IPEDS
Reported Degrees in General Economics: AYs
1997-2006 (a)

University IPEDS JEL JEL
 General Total (b) Deficit
 Economics

Northwestern 148 67 81
Princeton 162 124 38
Rochester 107 69 38
UC-San Diego 113 81 32
Penn 156 131 25
North Carolina State 100 76 24
New School 98 74 24
Duke 100 77 23
Hawaii-Manoa 53 30 23
UNC-Chapel Hill 96 74 22
Yale 201 181 20
Totals 1,334 984 350

(a) Two universities (Emory and Oklahoma) with no JEL
submissions but with over 20 PhDs reported by
IPEDS-GE are excluded from the list because these
programs were evidently not solicited by the JEL.

(b) May include some dissertations submitted by specialty
programs.

TABLE 4.
Adjustments to IPEDS' Baseline Count of General Economic
Degrees for Decade of 1997-2006

 GE Degrees

Baseline: IPEDS' total for GE (from Table 1) 8,534

Plus output of seven GE programs misclassified +237
 by IPEDS as managerial economics (see Part
 II.B.2)
Plus estimate of degrees not reported by IPEDS +70
 in AY 1999 (see Part II.B.2 and Appendix)
Plus degrees reported by JEL for a start-up + 19
 program at Clemson in 2005, 2006 (see Part
 II.B.2)
Plus JEL-IPEDS surpluses at eight universities +48
 with no JEL solicited specialty programs
 (see Part II.F.2)
Plus conservative estimate of additional +220
 economics department degrees recorded by SED
 but not IPEDS and associated with larger
 department output (see this section)
Equals overall estimate of total GE degrees 9,128

TABLE 5.
SED-IPEDS Degree Gaps, Social Science Disciplines: 1997-2006

Discipline Ph.D. Degrees Conferred: 1997-2006
 Granting
 Programs,
 2006 (a)

 SED IPEDS SED- Percent
 (b) (c) IPEDS Difference
 GAP

Economics: General 118 (d) 9,504 8,534 970 10.2%
History 138 9,890 8,984 906 9.2
Sociology 120 5,685 5,592 93 1.6
Anthropology 82 4,600 4,643 -43 -0.9
Political Science 106 6,386 6,665 -279 -4.4
Geography 49 1,778 1,873 -95 -5.3

(a) Source: National Research Council (2011).

(b) Source: NSF, Survey of Earned Doctorates, annual reports.

(c) Source: NSF, Integrated Postsecondary Education Data System,
annual reports.

(d) May include some minor specialty programs.

TABLE 6.
Doctoral Dissertations Reported and PhDs Conferred by Social Science
Disciplines in Selected Years

Discipline, DDs PhDs Conferred DDs as %
by Year Reported of Mean
 (a) PhDs

 SED IPEDS Mean

Sociology
 2005 522. 536 527 532. 98.1.
 2006 546. 578 562 570. 95.8.
 2007 583. 576 569 573. 101.7
 2005-07 1,651. 1,690 1,658 1,674. 98.6
Geography
 2005 226. 195. 210. 203 111.3.
 2006 199. 205. 222. 214 93.0.
 2007 180. 200. 211. 206 87.4.
 2005-07 605. 600. 643. 622. 97.3.
History
 2004 862. 975. 855 915 94.2
 2005 766. 924. 819 872 87.8.
 2006 838. 973. 852. 913. 91.8.
 2007 766. 940. 807 874 87.6.
 2004-07 3,232. 3,812. 3,333. 3,573. 90.5.
Economics, General
 2005 880. 1,031. 939. 985. 89.3.
 2006 816. 996. 898. 947. 86.2.
 2007 824. 974. 902. 938. 87.8
 2005-07 2,520. 3,001. 2,739 2,870 87.8
Political Science
 2006 610. 615. 646. 631. 96.7.
 2007 430. 588. 614. 601. 71.5.
 2006-07 1,040. 1,203. 1,260. 1,232. 84.4
Anthropology
 2005 327 456. 445. 451. 72.5
 2006 386. 472. 475. 474. 81.4
 2007 395. 511. 507. 509. 77.6.
 2005-07 1,108 1,439. 1,427. 1,433. 77.3.

(a) For information on the sources, content, organization, and
publication of dissertation lists, see text and references cited
there.