1. INTRODUCTION

The evaluation of research quality is important for funding agencies, publishers, and academic departments that make labor market decisions about hiring, promotion, and tenure. The rise in performance-based university research funding systems (Hicks 2012) gives one impetus for systematically evaluating research quality. For individual academics, the expected or revealed quality of their research has always had profound impacts on careers, as noted by Liebowitz and Palmer (1984, 77) in one of the seminal studies of economics journal rankings:

   Where articles are published can affect one's
   promotion, tenure, and salary at one's present job; it
   can also affect one's brand name and the ability to
   change jobs.

Notably, it is where the article is published that the quotation suggests has an effect. While the improved coverage, timeliness, and ease of use of citations databases are aiding direct evaluation of individual research outputs (Hamermesh 2017), using the journal where something published as a proxy for the quality of the research remains the basis for many assessments of economics departments and of economists (e.g., Coupe 2003; Macri and Sinha 2006). (1)

There are several arguments for relying more on citations. The prestige of the journal publishing an article should become less informative over time, since articles are accepted based on their predicted likelihood of having an impact, while citation-based measures of actual impact become available as time passes. Thus, Sgroi and Oswald (2013) show how citations give new information, in a Bayesian sense, and as time passes the weight on this information should rise while the weight on the prestige of the journal should fall toward zero. Also, citations may be less affected by nepotism; according to Liebowitz (2013, 1):

   Because a journal's decision depends on the opinion
   of only an editor and a few referees chosen by the
   editor, there is a great deal of latitude for gratuitous
   decisions. By way of contrast, the number of citations
   that a paper receives is determined by the entire
   academy, likely reducing the influence of gratuitous
   behavior. (2)

Yet there is little comprehensive evidence on the role of citations compared with that of journals, either in research assessment exercises or in labor market decisions made by academic departments. Exercises like the Research Excellence Framework (REF) in the United Kingdom may ask panelists to ignore journal rankings and to treat each article on its merits (where those merits may include citations) but reviewers will be aware of rankings and ex post one can find which rankings mimic peer evaluations (Anderson, Smart, and Tressler 2013). For labor market decisions, there is some direct evidence from small surveys. For example, Liebowitz (2014) finds that only 9 out of 47 economics departments give citations equal or more weight than that given to the rank of the publishing journal, when evaluating Associate Professors seeking promotion to full Professor. Amongst 43 departments surveyed by Hamermesh and Pfann (2012), all used data on recently accepted or published journal articles when considering annual merit/salary review requests, but only eight requested similar data on recent citations.

While regression studies of economist salaries do find significant effects of citations, these studies often have a different focus. (3) Moreover, most of these studies lack comprehensive controls for journal quality. For example, Hamermesh (2017) reviews ten studies of economist salaries that have citations as explanatory variables and just two of these use journal quality weights. Citations and journal quality are highly correlated so using weak (or no) journal quality measures may cause the estimated effects of citations to be overstated.

In this paper, we comprehensively compare effects of citations and journals on academic salary. We relate salary of tenured University of California (UC) economists to their lifetime publications of 5,500 articles in almost 700 different academic journals, and the 140,000 citations to these articles. (4) For research-intensive universities, salary should vary with perceived research quality. The UC is the largest research-intensive public university system in the United States, so provides external validity, and California has a high-quality public salary database. We build on the seminal contribution of Ellison (2013) in studying how the academic labor market uses citation data--while he compares citation indices, we examine a broader issue of citations versus journals. While citations and journals are used in prior studies of academic salary, our approach is more comprehensive. We use nine different journal assessment schemes to provide quality weights for calculating size-, coauthor-, and quality-adjusted lifetime output in journals. We also compare highly ranked and less highly ranked departments, and we search for the optimally weighted combination of citations and journals for articles of different vintages.

The results show citations hardly affect salary, especially in top-ranked UC departments where citations have impacts less than one-tenth those of journals. In lower ranked departments, and when journal quality is less comprehensively controlled for, the estimated effects of citations increase. For example, using article counts rather than quality-weighted journal output causes estimated effects of citations to triple or more (depending on specification). Our results contrast with a recent study by Hilmer, Ransom, and Hilmer (2015), hereafter referred to as HRH, who used a larger sample with lower ranked schools, and found that controlling for journals (with article counts in tiers of journal quality) did not affect the magnitude of estimated citation effects. If the HRH specification is applied to the UC data, in conjunction with our approach of using quality-weighted journal output, the quality-weighted approach dominates. In particular, the journal tiers variables are never statistically significant while quality-weighted journal output remains significant in almost all salary equations.

To further advance the argument that how one controls for journal quality may alter the apparent effect of citations, we use the HRH sample and supplement their salary equations with our quality-weighted measures of journal output. For assessment schemes that most heavily discount lower ranked journals, the estimated effects of citations on salary are up to 40% larger if the HRH journal tiers approach to measuring journal output is used rather than using our approach of calculating quality-weighted journal output. The sensitivity of estimated citation effects with respect to the different ways of controlling for journals is much more apparent for the top schools in the HRH sample. (5) Specifically, for top schools, the apparent effect of citations on salary is up to 60% larger if the HRH journal tiers are used to measure journal output compared to if our quality-weighted journal output measures are used.

Our results suggest that salary determination differs by school rank, which has previously been discussed by Hamermesh and Pfann (2012) and Liebowitz (2014). At the top UC schools, quality-adjusted journal output matters to salary while citations hardly matter, while in the lower ranked UC schools it is a mix of both citations and journals that matters. At the top schools in the HRH sample, both matter with journals slightly more important than citations. In contrast, salary setting for the bottom schools in the HRH sample seems to be more of a counting exercise, with total counts of articles (with no quality tiers) and total counts of citations affecting salary but no premium for journal quality. These results supportive an argument made by Liebowitz (2014) that faculty at lower ranked schools may rely on easily available proxies, such as counts of articles or of citations, rather than making their own determination based on reading the articles when they have to evaluate a research record in order to make a labor market decision. (6)

The rest of the paper is structured as follows: Section II reviews previous studies, noting that the growing literature on journal quality weights is largely ignored in studies of academic salary for economists, where simple counts of articles in journal quality tiers are typically used. Section III describes the UC data and the nine different sets of journal quality weights, which are used in the Section IV analyses that compare salary impacts of citations with those of journals. In Section V, we reconcile our results with other recent studies, showing that two patterns from the UC results--that effects of citations on salary are higher in lower ranked departments, and when journal quality is less comprehensively measured--also hold in the larger sample used by HRH (2015). Section VI has the conclusions.

II. PREVIOUS LITERATURE

The relationship between academic salary for economists and various metrics of research output and quality has been studied since the 1970s. This literature often uses classifications of journals in broadly defined excellence tiers, and citation-based metrics, to represent research quality. In the same era, literature on formal journal assessment schemes developed from the seminal work of Liebowitz and Palmer (1984), who rank economics journals using impact-adjusted citations. (7) An important feature of these journal assessment schemes is that they provide quality weights that can be used to convert articles in different journals into a constant quality output measure. For example, Economic Inquiry gets a weight of 8.70 (the highest is 100) in the impact-adjusted citations rankings of Liebowitz and Palmer (1984). These weights are informative because even if there was consensus on ordinal ranks for journals (which there is not, beyond the top-5) the weights for comparing journals may not be agreed upon. Thus, the Combes and Linnemer (2010) ranking has two weights for each journal, from differentially discounting lower ranked journals (for Economic Inquiry these weights are 24 and 6, where 100 is for the top journal). If journals are simply grouped by rank with no accounting for weights--as is typical in academic labor market studies--potentially relevant information (that correlates with citations) is ignored.

Hamermesh, Johnson, and Weisbrod (1982) were the first to use citations to represent research quality in an academic economist salary model. They argued that "... one scholar's productivity should be measured by the sum of direct and indirect influences on other producers as well as by direct contributions (publications)" (473). Using 5 years of citations data from the Social Science Citation Index for 148 professors of economics from seven large public universities, they fitted a model in which citations were the only measure of research output or quality. For six of the universities, citations had a positive impact on salaries, and explained from 6.5% to 27.3% of salary variation. For the two universities for which counts of publications were also available to represent research output, citations continued to be significant determinant of salaries and explained a similar proportion of salary variation as that explained by publications.

Effects of seniority are a focus for several studies that exemplify how publications data and citations are used in salary models. (8) Ransom (1993) found that salaries exhibited negative returns to seniority in faculty surveys across many disciplines, attributing this to monopsonistic discrimination. Controlling for research productivity with publication counts did not alter this pattern. In contrast, Moore, Newman, and Turnbull (1998) found that the negative seniority effect disappeared in a study of 142 economists at nine state universities, as more comprehensive measures of research productivity were added. Once counts of articles in two tiers of journals, and citations (as a count and interacted with the article counts), were added to the salary equations, the coefficient on the seniority variable became insignificant. Although comparing the impact of journal quality with that of citations was not the focus of Moore, Newman, and Turnbull (1998), a feature of their approach persists in much of the subsequent literature, which is to use counts of articles in journal tiers as a measure of research quality. (9) Their top tier of ten journals included Economica, yet this journal ranked just 28th in the Liebowitz and Palmer (1984) rankings. Thus, two problems for the tiers approach are that journals with higher quality weights are often excluded from the top tiers, and within-tier variation in quality weights is ignored; for example, Economica had a weight of just 5.9% of that of the top journal (the American Economic Review [AER]) in the same tier.

Similar issues of poorly specified tiers and ignoring within-tier variation affect two later studies of seniority by Bratsberg, Ragan, and Warren (2003, 2010). (10) Bratsberg, Ragan, and Warren (2003) use two journal tiers; the top one is the same as what Moore, Newman, and Turnbull (1998) use and the second tier consists of the rest of the 36 journals used by Scott and Mitias (1996). Bratsberg, Ragan, and Warren (2010) further divide other journals outside these two tiers into a further two tiers. The within-tier heterogeneity can be seen with two weighting schemes toward opposite ends of the scale of discounting lower ranked journals: the Combes and Linnemer (2010) medium-convexity CLm scheme based on citation measures and the Kalaitzidakis, Mamuneas, and Stengos (2010) KMS scheme based on impact-adjusted citations. With the CLm scheme, journals in the Bratsberg et al. top tier have quality weights that vary between 100 for the Quarterly Journal of Economics (QJE) to 27.2 for Economica. In the KMS scheme they vary between 100 for the AER and 2.2 for Economica. For their second tier, the CLm weights range from 80.6 (Journal of Financial Economics) to 13.6 (Journal of Regional Science). Under the KMS scheme, several journals in this tier are not ranked (so implicitly get a weight of zero), and weights of the ranked journals range from 27.8 (Journal of Monetary Economics) to 0.4 (Journal of Regional Science). Irrespective of views about appropriate rates for down-weighting lower ranked journals, it is clear that a lot of information is lost through the treatment of all articles in a journal tier as representing the same research quality.

In contrast to the tiers approach, Sauer (1988) is a rare example of using journal weights to study academic labor markets. He uses Liebowitz and Palmer (1984) weights to quality-adjust size-standardized and coauthor-adjusted journal pages, in a salary model for 140 economists from seven "top 40" schools. The degree of prorating for coauthorship and the extent to which salary varies with the original journal weights compared with an all-journals-are-equal scheme is determined from the data. (11) The results show that citations and quality-adjusted journal pages both affect salary although these two impacts are not compared. Relatedly, Gibson, Anderson, and Tressler (2014) use labor market data to sift between nine commonly used journal assessment schemes and find that more inclusive schemes, where weights do not decline sharply with the rank of the journal (such as the CLm scheme), have the greatest congruence with salaries of UC economists.

Ellison (2013) also uses labor market data to sift between competing metrics, for variants of the citation-based h-index of Hirsch (2005). Ellison develops a class of generalized h-like measures, [h.sub.(a,b)] for the number h such that one has at least h papers with a[h.sup.b] citations. With the rank of the employing department as the labor market indicator, using Google Scholar citations Ellison finds a wide range of values of the parameters a and b outperform h = [h.sub.(1,1)] and suggests [h.sub.(15,2)] as a metric for younger scholars and [h.sub.(5,2)] for seasoned academics. These generalized measures concentrate attention on a few, highly cited, publications and this may be a by-product of using Google Scholar, which includes a wider range of research outputs, and far higher citations counts than those from Web of Science (WoS). (12) Perhaps for this reason, HRH (2015) find contrary evidence to Ellison; using WoS citations the original h-index outperforms the [h.sub.(15,2)] index, in explaining salary in their sample of 1,009 economists at 53 public universities.

While HRH (2015) experiment with different citations metrics, in also using a quadratic in log citations plus one, they just use a conventional journal tiers approach to measuring the quality and quantity of publications. Specifically, counts of items published in an "elite" tier of top-5 journals (but using Review of Economics and Statistics rather than Review of Economic Studies, which is typically thought of as the fifth of the top-5), and an "excellent" tier of the other 31 journals in Scott and Mitias (1996) are used, along with counts of all other items recorded in WoS.13 Two issues with the tiers approach, of higher ranked journals being omitted from tiers and of ignoring within-tier variation, also apply to this study and may explain why adding these tier counts hardly changed citations coefficients in the HRH salary models. Journals in the HRH "excellent" group occupy places between 5th and 155th place in the CLm rankings, with an average rank of 45th rather than 21st that would be expected (they should be 6th to 36th placed, since the top-5 are in the "elite" group). In terms of within-tier variation, the "excellent" group has journals whose CLm quality weight ranges from just 13.5% of the weight for the QJE all the way up to 81 % of the QJE and yet these journals are all treated as equally excellent.

Many studies include citations and some measure of journal output in their salary models but impacts of the two types of variables are not compared. One exception is Hamermesh and Pfann (2012); they report that going from the 25th to the 75th percentile of total citations raises salary by 33% for their sample of 564 economists, while the same percentile increase in the number of articles raises salary by only 6%. This comparison does not control for journal quality, which is highly correlated with citations, so the absence of journal quality measures may cause citation impacts to be overstated. In general, article counts may not be the best summary measure of journal output, and a head-to-head comparison shows that counts of articles explain less variation in salary than is explained by size-, coauthor- and quality-adjusted pages (Gibson 2014). Thus the question of what a comparison of citations versus journals might reveal when a more complete representation of journal quality is used remains open.

III. DATA DESCRIPTION

The sample is all Professors and Associate Professors in economics departments of UC campuses, except adjuncts, affiliates, part-timers, and anyone with primarily teaching or administrative roles. This group of n = 167 is a subset of the sample of Gibson, Anderson, and Tressler (2014), excluding Assistant Professors, who typically have few citations. The findings are, thus, most germane for comparing impacts of citations and journals on salary for seasoned scholars. The dependent variable is the (log of) base salary for the 2010 academic year. Control variables are a quadratic in experience, a gender dummy, fixed effects for each UC campus, the proportion of lifetime articles that were in journals not covered by WoS at the time of publication, and a dummy for individuals not on a standard 9-month academic year and pay scale. (14)

The citations are to the lifetime articles published by these 167 tenured professors up to the end of 2010. The articles are mainly from WoS, although it was often easier to find them by first checking EconLit, RePEc, and curriculum vitae, especially for people with common names. Moreover, some economics journals (defined by the union of EconLit and RePEc) are only listed in IPOS' in more recent years so the other databases are needed for finding these articles. (15) In total, there were about 5,500 articles in almost 700 different academic journals. The 140,000 citations to these articles are from the WoS, which is the most established citations database, with stricter coverage than Google Scholar which includes citations to and from various unpublished works. For articles in journals not covered by WoS at the time of publication, their citations appear to be zero even if citations show up in other databases. Hamermesh (2017) reports a rough conversion rate of 17 WoS citations for every 100 in Google Scholar. We did not use this to impute these missing citations since the conversion factor is calculated from junior scholars and the lack of WoS coverage is more of a problem for senior academics with older articles. Instead, our salary models include an indicator for the proportion of articles published by each economist in journals that were not in WoS at the time, so as to account for this potential undercoverage.

In order to compare citations with a comprehensive measure of quality-adjusted journal output we need weights to convert output to a constant quality level (equal to the highest ranked journal, which is the QJE for most schemes). A wide range of journal assessment schemes have been proposed by economists, with no consensus on which is best. Thus, we use nine different schemes to ensure that results do not depend on the particular weights used. (16) Journal assessment schemes differ in terms of coverage, and aggressiveness (or convexity), which is the rate weights decline for lower ranked journals. In our application, coverage can be indicated by the percentage of lifetime articles by UC economists that receive zero weight. The elasticity of weights with respect to rank provides a measure of aggressiveness. The full descriptions for each scheme are in Gibson, Anderson, and Tressler (2014), with their brief details as follows:

* Mason, Steagall, and Fabritius: [MSF] reputational weights for 142 journals from a survey of economics department chairs. This is the least aggressive (a rank elasticity of -0.22) but has low coverage (30% of articles by UC economists are not counted).

* Coomes and Linnemer: [CLm, CLh] is the most comprehensive, covering 1,168 journals by using a Google Scholar /2-index to extrapolate from citations for EconLit journals to all journals (thus, just 11% of articles by UC economists are not counted). Two different rates for down-weighting lower ranked journals are used; a medium variant (CLm) with a rank elasticity of -0.67, and a high variant (CLh) with a rank elasticity of -1.35.

* RePEc is an impact factor from unweighted citations, covering 984 journals (when we collected the data in May 2012), and is the fourth least aggressive of the nine schemes with a rank elasticity of -1.17 and a coverage rate of 87%.

* Coupe is an average of 2-year impact factors for 1994-2000 from the Journal Citation Reports for 273 economics journals; this is the third least aggressive of the nine schemes with a rank elasticity of -0.78 and a coverage rate of 78%.

* Kodrzycki and Yu [2006; K&Y_all, K&Y_econ] is an "eigenfactor" approach where a journal is deemed influential if cited often by other influential journals. Subdiscipline citing intensity is adjusted for, with cites from all social science journals [K&Y_all] and just from economics [K&Y_econ], The rank elasticities are -1.48 and -1.53, and 74% of articles by UC economists are covered by the ranked journals.

* Kalaitzidakis, Mamuneas, and Stengos: [KMS] is an eigenfactor approach, using the average of citations from 2003-2008 to articles published in the previous 10 years. This is the second most aggressive scheme, with a rank elasticity of -1.74. It has the second lowest coverage rate, with 30% of articles by UC economists having zero weight.

* Laband and Piette: [LP] is an eigenfactor approach using citations to 130 economics journals over 1985-1989 by articles published in 1990. It has the lowest coverage, with 36% of articles by UC economists given zero weight, and the largest rank elasticity (-1.92).

Appendix Table A1 has normalized weights under each of these nine schemes, for each journal in the "elite" and "excellent" tiers of HRH, and also mean weights and ranks, and the highest and lowest rank. The issue of poorly specified tiers is shown by the six journals in the "excellent" tier that have a higher mean rank than the bottom journal of the "elite" group. The issue of within-tier variation is seen in mean weights that range from 0.37 to 0.80 for the "elite" journals and 0.12 to 0.63 for the "excellent" journals. Moving across the columns of the table from the least aggressive to most aggressive schemes shows the variation in these schemes, which should pick up different aspects of the representation of research quality. For example, under the MSF scheme, the range in the top two tiers of HRH journals is just from 1.00 to 0.64 so it is close to treating all journals as equal, whereas KMS is one of the most aggressive schemes with a range of weights from 1.00 to 0.01 (and omits five journals whose implicit weight is zero).

Using each set of weights (including zero for unranked journals), the pages for every article published by sample members are multiplied by the quality weight, adjusting for the number of authors and standardizing to the size of a typical page in the AER (17):

Article Pages X Size Correction

X (1 /number of authors)

X Journal Assessment Weight.

In comparison to the journal tiers approach, we have more granular data on the quality of journal output for each economist. Our quality-weighted output approach should deal with issues of poorly specified tiers and within-tier variation, and also captures variation in the size of articles.

The definitions and summary statistics for our various citations metrics, our measures of lifetime (to 2010) quality-adjusted journal output, and the control variables are in Table 1. The lifetime journal output ranges from an average of 44 pages using LP journal weights (which are the most discriminating) to 174 pages using MSF weights (the least discriminating), from an average of 33 articles published in the career to date. The total citations to these articles for each economist average just over 800, with a maximum of over 15,000, while the most highly cited article per economist averages 200 citations. The average h-index for these academics is just over 11, while the generalized h-index proposed by Ellison (2013) averages 2.5. We use the [h.sub.(5.2)] variant because Ellison estimates a = 5 and b = 2 for a sample from top-50 departments who average 22 years post-PhD, which is same level of experience as the current sample.

IV. RESULTS

The basic academic salary equations are reported in Table 2 and these explain about two-thirds of the variation in log salary for these economists. All nine regressions show a highly significant (at p < .01 level) positive effect on salary of quality-adjusted journal output. The journals and citations variables are standardized throughout, to facilitate comparisons, so the coefficients in Table 2 show that a standard deviation increase in quality-adjusted journal output raises salary by 10%-15%. The effect is highest when quality-adjusted output is based on less convex journal weighting schemes like MSF and CLm, which tend to give the best-fitting salary equations.18

In contrast to the always significant effect of quality-adjusted journal output, citations are statistically significant in only three of nine salary regressions (at just a 10% level for two). Also, citations are significant in some of the worst-fitting models (with [R.sup.2] values from 0.639 to 0.655, compared to 0.684 for the best-fitting model) that have smaller journal effects. Citations and quality-adjusted journal output correlate closely (with coefficients of 0.56 to 0.66), so if poorer-fitting journal weights like KMS are used, estimated effects of citations tend to strengthen. A clear demonstration that how one controls for journal quality alters estimated effects of citations comes from a model with no variables for journals; in this model, the citations coefficient jumps from an average of 0.018 in Table 2 to 0.068 and becomes highly significant (t = 3.03).

This model without journals variables is not reported in Table 2, but it lies behind the other results in Table 2, that come from econometric "horse races" between models of salary that either use quality-adjusted journal output or citations (with control variables common to the two sets of models). The last row of Table 2 shows that Vuong (1989) non-nested tests significantly favor the journals model in all nine iterations. These tests do not presume that either competing model is "true," and instead determine which has verisimilitude (that is, is closer to the truth). Thus, we infer from the results in Table 2 that a model putting weight on quality-adjusted journal articles rather than on citations to those articles is closer to the true model of departmental and university decision-making that produces the faculty salaries that we observe.

In Table 3 we report the results of six sensitivity analyses that are designed to expand upon the comparisons between quality-adjusted journals and citations. The first analysis adds an extra covariate--the count of articles ever published by each academic. Citations are higher for academics who publish more articles, conditional upon their total number of coauthor-, size-, and quality-adjusted pages (Gibson 2014). Thus in Table 2, the citation counts may act as a proxy for an excluded relevant variable--the total number of articles. Indeed, if this count is put in the model, citations become statistically insignificant in all nine salary equations (Panel A of Table 3) while quality-adjusted journal output stays highly significant (p < .01) in all nine. (19)

The next two sensitivity analyses change the specification of the citations variables. In Panel B, the logarithm of citations plus one is used, following the transformation used by HRH (2015). This difference in functional form does not appear to matter, with citations remaining statistically insignificant in all nine models while quality-adjusted journal output stays highly significant in all nine. In Panel C, the citations are divided by the number of authors on each paper, whereas the other analyses do not prorate citations in common with most prior studies. If prorated citations are used, the effect of quality-adjusted journals on salary is slightly stronger, with an average coefficient of 0.095 versus 0.091 in Panel A, while the effect of citations gets even smaller and remains statistically insignificant in all nine salary equations.

The sensitivity analyses in Panels D and E allow for the fact that citations may vary over fields, with theorists likely to get fewer citations than applied economists. If a dummy variable for the Journal of Economic Literature (JEL) letter category that each economist most frequently publishes in is added to the model the effect of journal output is slightly increased while citation effects are decreased and remain statistically insignificant in all nine equations. Along the same lines, if a diversity index is calculated, based on the spread of articles across 137 2-digit JEL categories, the small, positive, effect on salary of contributing to more fields is apparent. The inclusion of the diversity index brings a slight reduction in magnitude of the journals effect, from an average of 0.091 in Panel A to 0.086 in Panel E, but otherwise does not change the statistical significance of journals and the insignificance of citations.

The final sensitivity analysis in Table 3 shows what happens when campus fixed effects are excluded from the regressions. These fixed effects are meant to capture cost-of-living and other amenity differences between campuses but unobservable productivity that is negatively correlated with citations and publications may also be picked up. (20) When these fixed effects are dropped, the impact of quality-adjusted journals is almost doubled, going from an average effect of 0.091 to 0.173, while the impact of citations becomes smaller, and is even negative and statistically significant with LP weights. Considering all of these sensitivity analyses together, it appears to be a robust finding that for UC economists their salary depends on quality-adjusted journal output but hardly at all on citations.

The results in Table 4 exploit variation amongst the nine campuses of the UC system that has economics departments. The top four schools are Berkeley, UCLA, San Diego, and Davis, and salary equations just for these four are in Panel A, with results for the other, lower ranked, departments in Panel B. Tests for differences in coefficients between the two subsamples are in Panel C. In the top-ranked departments, quality-adjusted journal articles matter to salary while citations do not, while in the lower ranked departments both are significant and have effects of similar magnitude. Specifically, a standard deviation increase in quality-adjusted journal output is always statistically significant in the top four departments and has an salary impact that ranges from 9% to 16%, with an average of 13%, while citations impact range from -1% to 3%, average 1%, and are (weakly) significant in just one equation. Comparing the two indicators of research productivity, for top departments there is a 12:1 ratio for journals over citations.

In contrast, the lower ranked departments have effects of citations and journals that are roughly equal, with average standard deviation impacts of 0.08 for journals and 0.09 for citations. Moreover, citations are statistically significant in all nine salary equations for the lower ranked departments, yet were insignificant in eight equations for the top departments. The tests of the interaction effects show that differences in salary equations between the two subsamples are driven by citations; journals coefficients are higher in the top departments but the difference is statistically significant only with RePEc weights while differences in the citations coefficients are significant in all nine equations. The result that citations matter more in lower ranked departments is consistent with the argument made by Liebowitz (2014) that faculty in these types of departments may be less likely to read articles and make their own determination about the research quality of candidates, when making labor market decisions, than they are to rely on easily available indicators for research activity such as counts of articles and of citations.

The results thus far do not consider vintage effects, with quality-adjusted journal output aggregated into a single measure, regardless of when articles published. The argument by Sgroi and Oswald (2013) is that more weight over time ought to be put on the accrued citations and less weight put on the journal where the article published. Based on this reasoning, one could adopt a conservative rule for academic labor market decisions, of letting article-specific quality be revealed by citations before committing to costly salary decisions. Indeed, the tenure system where new hires have 6 years to prove the quality of their research before being offered a permanent appointment might be broadly thought of in these terms. We are not advocating this as a good description of how salary decisions are actually made, since it would be undermined by a faculty member who scored a solid hit, such as publishing in a top-5 journal, and threatened to leave for another department that promotes based on expected research impact before citations accumulate. Instead, we think of this as a scenario that should give citations their best possible chance to show an influence on academic salary.

Therefore, we recalculated the measure of lifetime production in journals by ignoring articles published in either the prior 3 or prior 6 years. However, for the articles published by those cut-offs (that is, by the end of 2004 or the end of 2007), we count all citations that had accrued by the end of 2010. This calculation attempts to mimic the conservative rule of waiting for the quality of an article to be revealed by citations before making labor market decisions. The 6-year period matches the typical length of time that new hires have on a "tenure clock" while midterm reviews at 3 years are when junior hires may consider moving to a lower ranked department and resetting their tenure clock. Six years also corresponds to the typical window used for research assessment exercises like the REF in the United Kingdom. According to Sgroi and Oswald (2013), the earliest articles in that window are ones where the greatest weight should be placed on their accrued citations, and the least weight on where they were published.

The results of the salary equations with these new information sets are in Panels B and C of Table 5. The results from Panel A of Table 3 that use all information on citations and articles to the end of 2010 are repeated for ease of comparison. Measures of (truncated) career output in journals remain highly significant in all nine salary equations. In contrast, the accumulated stock of citations is (weakly) statistically significant in just three equations. In terms of magnitudes, the average impact of quality-weighted journal output falls slightly, from 0.091 to 0.085 to 0.079 if articles are considered to the end of 2010, 2007, or 2004 while the citation impacts (counting to the end of 2010) rise slightly, from 0.009 to 0.012 and 0.020. Nevertheless, even with the most favorable assumption for citations, that articles published in the last 6 years are ignored but citations to earlier articles continue to be counted, it is still the case that citations have impacts on salary that are just one-quarter of the impacts of quality-adjusted journal output.

We also form weighted averages of citations and of quality-adjusted journal output, in the spirit of Sgroi and Oswald (2013) who suggest research evaluators should use such an approach. The aim is to find the optimal weights on citations and on journals, in terms of maximizing the log-likelihood of the salary equations. Rather than do this for all nine weighting schemes, we use two that are toward opposite ends of the spectrum for down-weighting lower ranked journals, CLm and KMS. It is also the case that KMS shows some of the biggest effects for citations and CLm some of the smallest. The maximized log-likelihoods are illustrated in Figures 1A (CLm) and IB (KMS). This iterative search procedure is repeated using the measures of lifetime journal output that omit the articles published in the last 3 and 6 years (while citations to earlier articles continue to accumulate), corresponding to the analysis in Table 5.

With CLm weights, the optimal combination of citations and quality-adjusted journals puts 2% weight on citations and 98% on journals. Ignoring the last 3 years of articles, the weights are 4% and 96%, and with a very conservative rule of waiting 6 years before an article is considered for influencing salary, to give time for citations to accumulate, the weight on citations is just 11% while there is a 89% weight on where the articles are published. (21) If the KMS weights are used, results are a little more favorable for citations, which get optimal weights of 17%, 16%, and 23% as ever more journal output is ignored. (22)

The results thus far consider total citations but not their distribution. Yet it may be the most heavily cited articles that show research impact, which is the issue that the A-index and its generalized variants are designed to deal with. We therefore carry out a further three analyses that (a) consider total citations and the citations to the most-cited article, (b) use the h-index for citations, and (c) use the [h.sub.(5,2)] index that Ellison (2013) finds works best for senior academics. Since the number of articles and the total quality-, size-, and coauthor-adjusted pages in those articles are both statistically significant, we continue to use these, so the baseline model, before the distribution of citations is considered, is the one previously reported in Panel A of Table 3.

Allowing for the distribution of citations does not change the finding that salary depends on journal articles but is hardly affected by citations. In Panel A of Table 6, we see that salary is higher if there are more citations to the most-cited article, conditional on journal output, while total citations have a negative but largely insignificant effect. The Vuong tests show that a model using total citations and citations to the most-cited article is significantly rejected against models using the journals variables. In Panel B, the h-index is not statistically significant, although its use does somewhat weaken the Vuong test evidence in favor of the journals models. Similarly, the generalized [h.sub.(5,2)] index in Panel C has no significant effect on salary and using this generalized index tends to strengthen the effect of the journals variables (and strengthens the Vuong tests in favor of models using journals variables over those using citations).

V. RECONCILIATION WITH PREVIOUS STUDIES

The results in Section IV suggest that, for UC economists, citations hardly affect salary, especially in top-ranked departments. This holds for various specifications of the citations variables, such as using logs, prorating for coauthored articles, and using variants of the h-index, and also holds in other sensitivity analyses. Impacts of citations stay small, relative to those of quality-adjusted journals, if recent articles are ignored while citations to older articles are allowed to accrue. It is only in the lower ranked departments, and if journal quality is less comprehensively measured, that effects of citations on salary become larger, and comparable to effects of journals. While these results have implications for several parts of the literature, they differ in particular respects from five recent studies (Ellison 2013; Hamermesh and Pfann 2012; Liebowitz 2014; Sgroi and Oswald 2013; and HRH 2015). The aim of this section is to discuss possible reasons for these differences and, where possible, to reconcile our findings with the literature.

The easiest reconciliation is with Sgroi and Oswald (2013) who make a normative point that research evaluators should put more weight on citations over time and less weight on the journals where research was published. Our results show that, irrespective of the logic of that argument, citations do not seem to be used in this way for setting salary of UC economists. It is also true that direct survey findings, from Hamermesh and Pfann (2012) and Liebowitz (2014), support the patterns we find, with far fewer departments relying on citations than on information about journal articles, when making labor market decisions.

The next easiest reconciliation is with Ellison (2013) who finds academic labor market evidence in favor of generalized h-index measures that concentrate attention on a few, highly cited, publications. In contrast, the generalized measures provide no advantage over the original h-index for explaining salaries of UC economists. This same pattern occurs in a larger sample (HRH 2015). Both the current study and HRH differ from Ellison (2013) in the labor market outcome studied, and in their citations database, using WoS rather than Google Scholar. It is not clear which difference matters most, and it is hard to retrospectively create a database of Google Scholar citations with the same timing as our WoS database (that ended in 2010) to explore this further. Thus, future work could usefully create parallel databases from Google Scholar and WoS in order to test if conclusions about labor market payoffs to research quality depend on the citations database used, helping to extend the limited comparisons in Hamermesh (2017).

The issue of how research-related determinants of faculty salary vary with the rank of the department is the third area for reconciling our findings with the literature. In the top-ranked UC departments, citations do not seem to affect salaries while journal quality matters a lot. In lower ranked UC departments citations become more important. We show below that a similar pattern holds in the larger sample used by HRH (2015). These results support Liebowitz (2014), who notes faculty in lower ranked departments may be less likely to form their own determination of the quality of a research record by reading the articles, and instead may rely on easily available proxies, such as counts of articles and of citations, when making labor market decisions. They are less supportive of an argument of Hamermesh and Pfann (2012) that higher ranked schools may put more weight on citations, since citations contribute to reputation. In general, differences in salary determination by school rank is an underresearched topic and one advantage of studying a single university system like the UC is that institutional factors, such as the UC system of merit pay rises, are held constant while still having departments at widely different points in the rankings.

The strand of the literature requiring the most effort to reconcile with our results concerns the importance of citations as salary determinants. Explicit comparisons of their impacts with those of publications are rare, except in Hamermesh and Pfann (2012), but implicit comparisons occur in papers that emphasize the role of citations. For example, in the HRH (2015) results, adding article counts (in journal tiers) to salary regressions did not affect the magnitude of estimated citation effects, making it seem that publications are weak and citations strong in explaining salary. As noted at multiple points above, one reason the literature may support such claims is because of imperfect controls for journal quality; since citations are correlated with journal quality, the citations variables will pick up the slack. For example, if quality-weighted journal output is dropped from the models in Table 3A, and just a count of articles is used (following Hamermesh and Pfann 2012), the citations coefficient increases almost fourfold.

In order to further show that article counts in tiers are weak controls for journal quality, we added the HRH journal tiers variables and one of their specifications of citations (a quadratic in the logarithm of citations plus one) to salary models, along with our quality-weighted journals variables. (23) The results in Panel A of Table 7 show that tier count variables are never statistically significant while quality-weighted journals are statistically significant in all but two equations. In other words, the approach used to control for journal quality in much of the literature--counting articles within journal tiers--ignores relevant information that instead is captured by the size-, coauthor-, and quality-adjusted journal pages. (24) Citations are correlated with quality-adjusted journal output so omitting this relevant variable sees some of its effect picked up by citations.

Since the journal tiers variables are statistically insignificant, they are dropped in models reported in Panel B of Table 7. These models have the HRH specification of citations variables along with our specification of journals variables. The quality-adjusted journal output variables are statistically significant at the 1% level in eight equations and at the 5% level in the other, but the citations variables are significant in just a single equation at the 10% level, using LP weights. Thus, the result that salary of UC economists depends on quality-adjusted journal output while citations hardly matter still holds when using the HRH specification of citations variables. Also, the HRH approach to measuring journal quality lacks explanatory power compared to when formal journal assessment schemes are used to calculate quality-weighted journal output.

To further advance the argument that how journal quality is measured alters the apparent effect of citations, we obtained the data that HRH (2015) use so we could supplement their salary equations with our quality-weighted measures of journal output. (25) Specifically, we use a model with the most comprehensive controls, reported in their Table 6B, column (5), and we calculate a statistical indicator and an economic indicator for the impact of adding their citations variables to models where journal output is measured either using their tier counts or our quality-weighted variables. The statistical indicator is the incremental [R.sup.2] from adding the HRH citations variables (a quadratic in the log of citations plus one). The results in Table 8 show that apparent effects of citations are overstated by an average of 20% if HRH journal tiers are used rather than using our quality-weighted approach. The exaggeration of citation impacts from using journal tiers is most, at 40%, when compared to using LP journal weights that aggressively down-weight lower ranked journals. In contrast, MSF weights, which are closer to an "all-journals-are-equal" scheme, hardly change the estimated citation effects over what is seen with the HRH journal tiers.

The economic indicator reported in Table 8 simulates the predicted salary increase from a standard deviation rise in citations, using the Duan (1983) method to predict level variables from log-dependent variables. If journal output is controlled using HRH journal tiers, predicted salary increases by $29,000 from a standard deviation increase in citations. In contrast, the increase in predicted salary is just $24,000 if journal output is controlled for using our quality-weighted approach (with LP weights). Thus, the HRH tiers approach to measuring journal output leads to a 20% overstatement in the estimated salary impact of a citations shock.

In the remaining panels of Table 8, we repeat these exercises, separately for top-ranked schools and other schools in the HRH sample. The top-ranked schools are a better comparison to the UC sample, and split-sample results also are motivated by the findings in Table 4 that salary models differ by school rank. For the top schools in the HRH sample, the way that one controls for journals has a big effect on the apparent impact of citations; the incremental [R.sup.2] is about 60% higher and the salary effect about 30% higher when using the HRH journal tiers rather than using our quality-weighted approach (for the most convex weights, such as LP and CLh). In contrast, for lower ranked schools in the HRH sample, it does not matter which approach is used to control for journals, with little variation across the nine sets of journal weights or when using the HRH approach of counting articles within journal tiers.

In Table 9, we expand upon this difference between top schools and other schools, using the HRH model from their Table 6B, column (5) but with our journal variables. (26) For top schools in the HRH sample, the average effect of a standard deviation in quality-weighted journal output is to raise salary by just over six log points. This is slightly larger than the average marginal effect of citations (with the quadratic evaluated at the mean). In contrast, for lower ranked schools, quality-weighted journal output has no effect. The citations effects are about the same for the two groups of schools, with average marginal effects of 0.061 for top schools and 0.068 for lower ranked schools (in terms of precision, squared citations are always highly significant for lower ranked schools while hardly significant for the top schools). The other difference between the salary models for the two groups of schools is that the impact of article counts, with no splitting into journal tiers, is roughly doubled for lower ranked schools.

The evidence in Table 9 suggests that in lower ranked schools in the HRH sample, salary setting is more of a counting exercise, with total counts of articles (with no quality tiers) and total counts of citations affecting salary but no premium for journal quality. In contrast, top schools in the HRH sample pay more according to journal quality than to article counts. A lower premium for journal quality as one move down the ranks of schools was also apparent in the UC data, where salary at the top UC schools varied with quality-adjusted journal output but hardly at all with citations, while for lower ranked UC schools a mix of both citations and journals matter. These results help account for why the UC sample has weaker effect of citations than the HRH sample, even with the same controls for journals; the UC sample has more top schools and these schools reward journal quality more than citations. The lower ranked schools in the HRH sample dampen the sensitivity of estimated citation impacts to different ways of controlling for journal quality since those schools do not seem to pay a premium for journal quality.

VI. CONCLUSIONS

The salary of UC economists depends far more on journal quality than on citations. This pattern holds in many different sensitivity analyses and even if recent articles are ignored while citations to older articles are allowed to continue accumulating. It is only for the lower ranked UC departments, or if journal quality is less comprehensively controlled for, that effects of citations increase. For example, if quality-weighted journal output is not accounted for, estimated impacts of citations on salary increase almost fourfold.

The typical way that studies of economist salary control for journal output is to count articles within tiers of journals. This approach is imperfect because higher ranked journals are often omitted from tiers and within-tier quality variation is ignored. Citations correlate with journal quality, so using imperfect controls for journal output increases estimated effects of citations. In both the UC sample and the HRH sample, there is information that is relevant to salary that is captured by size-, coauthor-, and quality-adjusted journal pages; if this information is ignored, it leads to estimated impacts of citations being overstated. This overstatement is especially apparent at top schools where the returns to journal quality appear to be highest. However, if the focus is on lower ranked schools, the different ways of controlling for journal output may matter less since these schools do not appear to reward publication in the highest ranked journals to the same degree as do top schools.

ABBREVIATIONS

AER: American Economic Review

HRH: Hilmer, Ransom, and Hilmer (2015)

JEL: Journal of Economic Literature

NRC: National Research Council

QJE: Quarterly Journal of Economics

REF: Research Excellence Framework

UC: University of California

WoS: Web of Science

doi:10.1111/ecin.12453

APPENDIX

TABLE A1 Examples of the Journal Weights and Ranks, for
the 36 Journals Used by Hilmer, Ransom, and Hilmer (2015)

Journal                                     MSF    CLm    CLh

Quarterly Journal of Economics              0.95   1.00   1.00
Journal of Political Economy                1.00   0.96   0.93
Econometrica                                1.00   0.96   0.92
American Economic Review                    1.00   0.98   0.96
Review of Economics & Statistics            0.92   0.74   0.55
Excellent Journal Group
Review of Economic Studies                  0.86   0.81   0.66
Journal of Finance                          0.89   0.72   0.52
Journal of Financial Economics              0.72   0.81   0.65
Journal of Monetary Economics               0.82   0.76   0.58
Rand Journal of Economics                   0.83   0.64   0.41
Journal of Econometrics                     0.81   0.69   0.47
Economic Journal                            0.88   0.65   0.42
Journal of Economic Theory                  0.87   0.73   0.53
Journal of Labor Economics                  0.81   0.55   0.30
Journal of International Economics          0.76   0.62   0.38
International Economic Review               0.84   0.55   0.30
Journal of Public Economics                 0.75   0.62   0.39
Journal of Money Credit & Banking           0.81   0.42   0.18
Journal of Law & Economics                  0.81   0.41   0.17
Journal of Business & Economic Statistics   0.68   0.48   0.23
Journal of Human Resources                  0.73   0.42   0.18
Journal of Business                         0.76   0.29   0.09
Journal of Law, Economics & Organization    0.67   0.32   0.10
Journal of Development Economics            0.70   0.43   0.18
Journal of Economic Dynamics & Control      0.75   0.36   0.13
Journal of Urban Economics                  0.72   0.40   0.16
Journal of International Money & Finance    0.78   0.23   0.05
Economica                                   0.83   0.27   0.07
Economic Inquiry                            0.74   0.24   0.06
Industrial & Labor Relations Review         0.70   0.19   0.03
National Tax Journal                        0.68   0.23   0.05
Journal of Legal Studies                    0.64   0.16   0.03
Journal of Economic History                 0.76   0.31   0.10
Southern Economic Journal                   0.73   0.19   0.04
Public Choice                               0.64   0.27   0.07
Journal of Regional Science                 0.72   0.14   0.02

Journal                                     RePEc   Coupe   K&Y_all

Quarterly Journal of Economics              1.00    0.59     0.86
Journal of Political Economy                0.52    0.43     0.64
Econometrica                                0.83    0.41     0.44
American Economic Review                    0.46    0.32     0.29
Review of Economics & Statistics            0.29    0.16     0.18
Excellent Journal Group
Review of Economic Studies                  0.55    0.28     0.38
Journal of Finance                          0.26    0.40     0.81
Journal of Financial Economics              0.49    0.37     0.84
Journal of Monetary Economics               0.50    0.20     0.24
Rand Journal of Economics                   0.38    0.19     0.22
Journal of Econometrics                     0.34    0.20     0.17
Economic Journal                            0.33    0.22     0.12
Journal of Economic Theory                  0.25    0.14     0.13
Journal of Labor Economics                  0.48    0.21     0.13
Journal of International Economics          0.32    0.18     0.17
International Economic Review               0.22    0.13     0.18
Journal of Public Economics                 0.30    0.14     0.11
Journal of Money Credit & Banking           0.20    0.15     0.16
Journal of Law & Economics                  0.21    0.21     0.12
Journal of Business & Economic Statistics   0.31    0.18     0.15
Journal of Human Resources                  0.26    0.21     0.12
Journal of Business                         0.14    0.18     0.37
Journal of Law, Economics & Organization    0.21    0.21
Journal of Development Economics            0.21    0.11     0.07
Journal of Economic Dynamics & Control      0.16    0.09     0.07
Journal of Urban Economics                  0.15    0.13     0.05
Journal of International Money & Finance    0.20    0.10     0.07
Economica                                   0.13    0.10     0.03
Economic Inquiry                            0.11    0.11     0.04
Industrial & Labor Relations Review         0.08    0.25     0.07
National Tax Journal                        0.09    0.11     0.05
Journal of Legal Studies                    0.08    0.53
Journal of Economic History                 0.01    0.12     0.05
Southern Economic Journal                   0.07    0.07     0.05
Public Choice                               0.06    0.06     0.02
Journal of Regional Science                 0.06    0.09     0.01

Journal                                     K&Y_econ   KMS     LP

Quarterly Journal of Economics                0.78     0.60   0.42
Journal of Political Economy                  0.59     0.41   0.63
Econometrica                                  0.41     0.45   0.78
American Economic Review                      0.27     1.00   0.40
Review of Economics & Statistics              0.18     0.24   0.07
Excellent Journal Group
Review of Economic Studies                    0.37     0.33   0.41
Journal of Finance                            0.60            0.34
Journal of Financial Economics                0.64     0.16   1.00
Journal of Monetary Economics                 0.23     0.28   0.42
Rand Journal of Economics                     0.20     0.13   0.40
Journal of Econometrics                       0.16     0.16   0.19
Economic Journal                              0.12     0.21   0.08
Journal of Economic Theory                    0.13     0.23   0.35
Journal of Labor Economics                    0.13     0.09   0.15
Journal of International Economics            0.16     0.19   0.08
International Economic Review                 0.17     0.12   0.12
Journal of Public Economics                   0.11     0.22   0.09
Journal of Money Credit & Banking             0.15     0.12   0.09
Journal of Law & Economics                    0.09     0.03   0.12
Journal of Business & Economic Statistics     0.15     0.07   0.08
Journal of Human Resources                    0.11     0.08   0.05
Journal of Business                           0.26            0.21
Journal of Law, Economics & Organization               0.03
Journal of Development Economics              0.07     0.10   0.01
Journal of Economic Dynamics & Control        0.06     0.07   0.05
Journal of Urban Economics                    0.05     0.05   0.02
Journal of International Money & Finance      0.06
Economica                                     0.03     0.02   0.03
Economic Inquiry                              0.04     0.04   0.04
Industrial & Labor Relations Review           0.07            0.04
National Tax Journal                          0.05     0.02   0.00
Journal of Legal Studies                                      0.02
Journal of Economic History                   0.05     0.04   0.03
Southern Economic Journal                     0.05     0.02   0.01
Public Choice                                 0.02     0.04   0.02
Journal of Regional Science                   0.01     0.01   0.00

Journal                                      Mean    Highest
                                            Weight    Rank

Quarterly Journal of Economics               0.80       1
Journal of Political Economy                 0.68       1
Econometrica                                 0.69       1
American Economic Review                     0.63       1
Review of Economics & Statistics             0.37       6
Excellent Journal Group
Review of Economic Studies                   0.51       5
Journal of Finance                           0.57       4
Journal of Financial Economics               0.63       1
Journal of Monetary Economics                0.45       4
Rand Journal of Economics                    0.38       7
Journal of Econometrics                      0.35      11
Economic Journal                             0.34      10
Journal of Economic Theory                   0.37       8
Journal of Labor Economics                   0.32      13
Journal of International Economics           0.32      12
International Economic Review                0.29      14
Journal of Public Economics                  0.30       9
Journal of Money Credit & Banking            0.25      19
Journal of Law & Economics                   0.24      19
Journal of Business & Economic Statistics    0.26      22
Journal of Human Resources                   0.24      25
Journal of Business                          0.29      11
Journal of Law, Economics & Organization     0.26      31
Journal of Development Economics             0.21      23
Journal of Economic Dynamics & Control       0.19      28
Journal of Urban Economics                   0.19      28
Journal of International Money & Finance     0.22      24
Economica                                    0.17      15
Economic Inquiry                             0.16      36
Industrial & Labor Relations Review          0.18      27
National Tax Journal                         0.14      52
Journal of Legal Studies                     0.24       6
Journal of Economic History                  0.16      30
Southern Economic Journal                    0.14      37
Public Choice                                0.13      45
Journal of Regional Science                  0.12      41

Journal                                     Lowest   Mean
                                             Rank    Rank

Quarterly Journal of Economics                5       2
Journal of Political Economy                  10      5
Econometrica                                  11      5
American Economic Review                      16      8
Review of Economics & Statistics              59      20
Excellent Journal Group
Review of Economic Studies                    22      9
Journal of Finance                            30      11
Journal of Financial Economics                43      12
Journal of Monetary Economics                 41      14
Rand Journal of Economics                     42      18
Journal of Econometrics                       37      19
Economic Journal                              30      20
Journal of Economic Theory                    70      23
Journal of Labor Economics                    36      23
Journal of International Economics            52      24
International Economic Review                 83      28
Journal of Public Economics                   72      28
Journal of Money Credit & Banking             64      30
Journal of Law & Economics                    46      31
Journal of Business & Economic Statistics     53      31
Journal of Human Resources                    39      31
Journal of Business                           65      35
Journal of Law, Economics & Organization      55      41
Journal of Development Economics              93      44
Journal of Economic Dynamics & Control       109      44
Journal of Urban Economics                    79      46
Journal of International Money & Finance      98      56
Economica                                    101      60
Economic Inquiry                              96      61
Industrial & Labor Relations Review          110      61
National Tax Journal                          91      65
Journal of Legal Studies                     109      73
Journal of Economic History                  290      75
Southern Economic Journal                    143      80
Public Choice                                148      83
Journal of Regional Science                  155     111

Notes: Weights are indexed so that the highest weighted
journal = 1.00. Missing values are journals excluded from
a particular scheme. Highest, lowest, and mean rank allow
for ties. Sources for each weighting scheme in text.

REFERENCES

Anderson, D., W. Smart, and J. Tressler. "Evaluating Research--Peer Review Team Assessment and Journal Based Bibliographic Measures: New Zealand PBRF Research Output Scores in 2006." New Zealand Economic Papers, 47(2), 2013, 140-57.

Bratsberg, B., J. Ragan, and J. Warren. "Negative Returns to Seniority: New Evidence in Academic Markets." Industrial and Labor Relations Review, 56(2), 2003, 306-23.

--. "Does Raiding Explain the Negative Returns to Faculty Seniority?" Economic Inquiry, 48(3), 2010, 704-21.

Bruns, S., and D. Stem. "Research Assessment Using Early Citation Information." Scientometrics, 108(2), 2016, 917-35.

Combes, P-P., and L. Linnemer. "Inferring Missing Citations: A Quantitative Multi-Criteria Ranking of all Journals in Economics." Groupement de Recherche en Economie Quantitative d'Aix Marseille (GREQAM), Document de Travail, no. 2010-28, 2010.

Coupe, T. "Revealed Performances: Worldwide Rankings of Economists and Economics Departments, 1990-2000." Journal of the European Economic Association, 1(6), 2003, 1309-45.

Duan, N. "Smearing Estimate: A Nonparametric Retransformation Method." Journal of the American Statistical Association, 78(383), 1983, 605-10.

Ellison, G. "Evolving Standards for Academic Publishing: A q-r Theory." Journal of Political Economy, 110(5), 2002, 994-1034.

--. "How Does the Market Use Citation Data? The Hirsch Index in Economics." American Economic Journal: Applied Economics, 5(3), 2013, 63-90.

Gibson, J. "Returns to Articles versus Pages in Academic Publishing: Do Salary-Setters Show 'Article Illusion'?" Economics Letters, 125(3), 2014, 343-46.

Gibson, J., D. Anderson, and J. Tressler. "Which Journal Rankings Best Explain Academic Salaries? Evidence from the University of California." Economic Inquiry, 52(4), 2014, 1322-40.

Hamermesh, D. "Citations in Economics: Measurement, Uses and Impacts." Journal of Economic Literature, 2017.

Hamermesh, D., and G. Pfann. "Reputation and Earnings: The Roles of Quality and Quantity in Academe." Economic Inquiry, 50(1), 2012, 1-16.

Hamermesh, D., G. Johnson, and B. Weisbrod. "Scholarship, Citations and Salaries: Economic Rewards in Economics." Southern Economic Journal, 49(3), 1982, 472-81.

Hicks, D. "Performance-Based University Research Funding Systems." Research Policy, 41(2), 2012, 251-61.

Hilmer, C., and M. Hilmer. "How Do Journal Quality, Co-Authorship, and Author Order Affect Agricultural Economists' Salaries?" American Journal of Agricultural Economics, 87(2), 2005, 509-23.

--. "Negative Returns to Seniority and Job Mobility across the Program Quality Distribution: Are Top Public PhD-Granting Programs Different?" American Economic Review: Papers and Proceedings, 101(3), 2011, 466-70.

Hilmer, C., M. Hilmer, and J. Lusk. "A Comparison of Salary Structures between Economics and Agricultural Economics Departments." Applied Economic Perspectives and Policy, 34(3), 2012, 489-514.

Hilmer, M., M. Ransom, and C. Hilmer. "Fame and the Fortune of Academic Economists: How the Market Rewards Influential Research in Economics." Southern Economic Journal, 82(2), 2015,430-52.

Hirsch, J. "An Index to Quantify an Individual's Scientific Research Output." Proceedings of the National Academy of Science, 102(46), 2005, 16569-72.

Kalaitzidakis, P., T. Mamuneas, and T. Stengos. "An Updated Ranking of Academic Journals in Economics." Working Paper 9/2010, Economics Department, University of Guelph, Guelph, Canada, 2010.

Kodrzycki, Y., and P. Yu. "New Approaches to Ranking Economics Journals." B.E. Journal of Economic Analysis and Policy: Contributions to Economic Analysis and Policy, 5(1), 2006, Article 24.

Laband, D., and M. Piette. "The Relative Impact of Economics Journals." Journal of Economic Literature, 32(2), 1994, 640-66.

Liebowitz, S. "Our Uneconomic Methods of Measuring Economic Research." Voxeu December 6, 2013. Accessed June 2015. http://www.voxeu.org/article/ouruneconomic-methodsmeasuring-economic-research.

--. "Willful Blindness: The Inefficient Reward Structure in Academic Research." Economic Inquiry, 52(4), 2014, 1267-83.

Liebowitz, S., and J. Palmer. "Assessing the Relative Impacts of Economics Journals." Journal of Economic Literature, 22(1), 1984, 77-88.

Macri, J., and D. Sinha. "Rankings Methodology for International Comparisons of Institutions and Individuals: An Application to Economics in Australia and New Zealand." Journal of Economic Surveys, 20(1), 2006, 111-56.

Mason, P., J. Steagall, and M. Fabritius. "Economics Journal Rankings by Type of School: Perceptions versus Citations." Quarterly Journal of Business and Economics, 36(1), 1997, 69-79.

Moore, W., R. Newman, and G. Turnbull. "Do Academic Salaries Decline with Seniority?" Journal of Labor Economics, 16(2), 1998, 352-66.

--. "Reputational Capital and Academic Pay." Economic Inquiry, 39(4), 2001, 663-71.

Ransom, M. "Seniority and Monopsony in the Academic Labor Market." American Economic Review, 83(1), 1993,221-33.

Sauer, R. "Estimates of the Returns to Quality and Co-authorship in Economic Academia." Journal of Political Economy, 96(4), 1988, 855-66.

Scott, L., and P. Mitias. "Trends in Rankings of Economics Departments in the U.S.: An Update." Economic Inquiry, 34(3), 1996, 378-400.

Sgroi, D., and A. Oswald. "How Should Peer-review Panels Behave?" Economic Journal, 123(570), 2013, F255-78.

Spiegel, M. "Reviewing Less--Progressing More." Review of Financial Studies, 25(5), 2012, 1331-38.

Vuong, Q. "Likelihood Ratio Tests for Model Selection and Non-nested Hypotheses." Econometrica, 57(2), 1989, 307-33.

Wilhite, A. W., and E. A. Fong. "Coercive Citation in Academic Publishing." Science, 355, 2012, 542-43.

JOHN GIBSON, DAVID L. ANDERSON and JOHN TRESSLER *

* We are grateful to audiences at Monash and the WEAI conference, and especially to two anonymous referees and the editor for helpful comments. We particularly acknowledge the assistance of Michael Ransom and his coauthors in providing the data from their study. All remaining errors are those of the authors.

Gibson: Professor, Department of Economics, Waikato Management School, University of Waikato, Hamilton 3240, New Zealand. Phone 64 7 838 4289, Fax 64 7 838 4331, E-mail jkgibson@ waikato.ac.nz

Anderson: Emeritus Professor, Smith School of Business, Queen's University, Kingston K7L 3N6, Canada. Phone 1-790-470-0408, Fax 1-613-533-2301, E-mail dla@queensu.ca

Tressler: Research Associate, Department of Economics, Waikato Management School, University of Waikato, Hamilton 3240, New Zealand. Phone 64 7 856 2889, Fax 64 7 838 4331, E-mail tressler@waikato.ac.nz

(1.) In terms of timeliness, long-run citations in economics are predictable from early citations (Bruns and Stem 2016).

(2.) Of course, citations may also be prone to manipulation. Wilhite and Fong (2012) highlight coercing journals, while Ellison (2002) and Spiegel (2012) highlight the growth in the length of bibliographies, which may reflect a response by authors to an expected desire of editors and potential referees to be cited. We leave it for future work to assess whether citations or publications are more prone to manipulation.

(3.) For example, they may focus on things such as the returns to seniority (Ransom 1993; Moore, Newman, and Turnbull 1998; Bratsberg, Ragan, and Warren 2003, 2010; Hilmer and Hilmer 2011) or on effects of coauthorship (Sauer 1988; Moore, Newman, and Turnbull 2001; Hilmer and Hilmer 2005).

(4.) This is based on the same salary and publications data as Gibson, Anderson, and Tressler (2014) but adds the citations data and omits Assistant Professors who typically have not accumulated many citations.

(5.) According to the National Research Council (NRC) ranks, the top half of schools in the HRH sample is a better comparator to the UC schools.

(6.) The journal of publication is a less easily used proxy for quality, since there is no agreement on ranks beyond the top-5 journals, nor agreement on weights for converting output from one journal to another. Thus, there is still a need for user judgment and this need is increased by dynamics in the journal marketplace. Debates about where to rank new journals like the American Economic Journals, or when societies break from publishers (e.g., European Economic Review versus Journal of the European Economic Association or Journal of Environment Economics and Management versus Journal of the Association of Environmental and Resource Economists) show this uncertainty. Indeed, the diversity of views about journals is one reason why we use nine different assessment schemes.

(7.) Widely used journal rankings and weights are provided by: Mason, Steagall, and Fabritius (1997), Combes and Linnemer (2010), Coupe (2003), Kodrzcki and Yu (2006), Kalaitzidakis, Mamuneas, and Stengos (2010), and Laband and Piette (1994).

(8.) Bratsberg, Ragan, and Warren (2003) summarize nine, earlier, seniority studies.

(9.) Moore, Newman, and Turnbull (2001), Bratsberg, Ragan, and Warren (2003, 2010), Hilmer and Hilmer (2005), Hilmer, Hilmer, and Lusk (2012), and HRH (2015) are further examples.

(10.) These studies had useful innovations, by accounting for coauthorship and the different size of articles. These are two of the three adjustments that we make, but neglect the most important, which is the quality weights for journals.

(11.) Specifically, a search over values between 0 and 1 for an exponent to the weights yields an optimal [alpha] = .3.

(12.) Hamermesh (2017) shows that for a sample of younger faculty from top-30 economics departments with Google Scholar profiles, their citations in Web of Science increase at a rate of just 17.3 per 100 citations in Google Scholar.

(13.) We use "items" rather than "articles" to describe the data used by HRH (2015) because they include book reviews, editorials, announcements, and other nonarticle material that WoS attributes to authors.

(14.) A previous version included a quadratic in seniority but a referee urged that this be dropped since (low) seniority conditional on experience is likely to proxy for quality of that particular academic, and therefore is likely to be correlated with publications and citations. Our main results are not changed by including or excluding seniority.

(15.) For example, 13 of the 31 journals in the HRH "excellent" tier have incomplete coverage in WoS. Mostly this is for newer journals which had published for an average of 5 years (and a range from 2 to 12) before being included.

(16.) These quality weights are as originally published. We have not attempted to update these with more recent data.

(17.) There is mixed evidence on whether coauthored articles are fully prorated (Hilmer, Hilmer, and Lusk 2012; Liebowitz 2014). For UC economists, Gibson, Anderson, and Tressler (2014) show tests for prorating depend on whether older articles are discounted.

(18.) Model comparison and non-nested tests for a larger sample of UC economists that included Assistant Professors also showed that the CLm journal weights were the ones most congruent with the salary data (Gibson, Anderson, and Tressler 2014).

(19.) If the quality-adjusted journals variable is omitted, and just counts of articles and of citations are used, the coefficient on citations increases almost fourfold over its average in Table 3A, to a statistically significant 0.036.

(20.) We are grateful to an anonymous referee for this argument, which is that someone hired at a prestigious school without a strong publication or citation record must have a component of their productivity that is observable to colleagues in the hiring department but not observable to us in the record of their journal articles and citations.

(21.) Using this rule also lowers the log-likelihood a lot (Figure 1A), showing the unreality of these assumptions.

(22.) Figure 1 includes article counts, which remain significant even with the last 6 years of articles excluded. If the counts are omitted, the optimal weights on citations are 2%, 3%, and 9% (CLm) and 23%, 22%, and 26% (KMS).

(23.) We prefer this to using the h-index, since the h-index also depends, partly, on the number of articles and so may not be a pure measure of citations, as Hamermesh and Pfann (2012) also argue.

(24.) The statistical significance of the total count of articles--with no differentiation into journal tiers--for the journal assessment schemes that are most convex (LP, KMS, K&Y, and CLh) suggests that quality and quantity of journal output is relevant to pay, as also shown in Gibson (2014) but this pattern also is hidden by the journal tiers approach.

(25.) We are grateful to Michael Ransom and coauthors for providing these data, which consisted of an estimation file and two publications databases. The databases included items not in the UC data, like book reviews and editorials. We created our quality-weighted journals measures for supplementing the HRH salary equations just from articles meeting the standards applied to the UC data; there are about 15% fewer of these than of the items used by HRH.

(26.) This is analogous to Panel B of Table 7 for the UC data, except that here we include the quadratic in seniority so as to maintain comparability with the HRH specification.

Caption: FIGURE 1 Optimally Weighted Combinations of Citations and Quality-Adjusted Journal Output

TABLE 1
Variable Definitions, Means, and Standard Deviations

Variable                            Mean    Std Dev

Salary                            173.35      55.89
Number of articles                 32.57      24.80
Citations to those articles       827.47    1464.55
Citations to most-cited article   201.08      49.22
h-index                            11.33       6.81
[h.sub.(5,2)]-index                 2.53       1.21
Not-ISI                             0.12       0.12

Quality-, size-, and coauthor-adjusted journal output
(lifetime AER-sized pages) using journal weights from

  MSF                             174.47     136.14
  CLm                             129.41      98.33
  CLh                              85.35      71.88
  RePEc                            79.98      71.05
  Coupe                            55.89      49.10
  K&Y_all                          48.73      46.21
  K&Y econ                         45.15      42.78
  KMS                              47.05      41.35
  LP                               44.11      42.15
Control variables
  Experience (years)               22.66      10.86
  Male                              0.86       0.35
  Not standard pay scale            0.04       0.19

Variable                          Description

Salary                            Base salary in 2010 ($,000)
Number of articles                Career total articles
Citations to those articles       Total WoS citations to those
                                    articles to end of 2010
Citations to most-cited article   Total WoS citations to the
                                    most-cited article
h-index                           Economist has h articles with h
                                    or more citations
[h.sub.(5,2)]-index               Generalized fi-index; h articles
                                    with 5 [h.sup.2] citations
Not-ISI                           Share of journals not in WoS
                                    (so zero cites recorded)

Quality-, size-, and coauthor-adjusted journal output
(lifetime AER-sized pages) using journal weights from

  MSF                             Mason, Steagall and Fabritius
                                    reputational ranking
  CLm                             Combes-Linnemer medium-convexity
                                    weights
  CLh                             Combes-Linnemer high convexity
                                    weights
  RePEc                           RePEc Simple Impact Factor
  Coupe                           Average of 2-year impact factors
                                    for 1994-2000
  K&Y_all                         Kodrzycki and Yu eigenfactor
                                    ranks, cites from all journals
  K&Y econ                        Kodrzycki and Yu ranks, cites just
                                    from econ journals
  KMS                             Kalaitzidakis, Mamuneas and
                                    Stengos eigenfactor weights
  LP                              Laband and Piette eigenfactor
                                    weights
Control variables
  Experience (years)              Years since first appointment
  Male                            Person is male (=1) or female (=0)
  Not standard pay scale          Not on a standard, 9-month,
                                    academic year pay scale

Note: N = 167.

TABLE 2
Salary Regressions for Tenured UC Economists Using Lifetime
Quality-Adjusted Journal Output and WoS Citations

                                  Journal Weighting Scheme for
                                  Calculating Quality-Adjusted
                                  Journal Output over Lifetime
                                           Comes from

                                MSF            CLm            CLh

Quality-adjusted journals      0.132          0.148          0.127
                             (6.77) ***     (7.06) ***     (5.10) ***
Total citations                0.014          0.003          0.009
                               (0.89)         (0.16)         (0.53)
Experience (years)             0.015          0.012          0.017
                             (2.54) **      (2.06) **      (2.88) ***
Experience squared             -0.027         -0.021         -0.028
                             (2.36) **       (1.88) *      (2.50) **
Male                           0.026          0.012          0.020
                               (0.56)         (0.25)         (0.42)
Constant                       11.958         11.970         11.898
[R.sup.2]                   (142.02) ***   (140.09) ***   (137.07) ***
                               0.677          0.684          0.659
Vuong non-nested test (a)     3.12 ***       3.53 ***       2.90 ***

                                  Journal Weighting Scheme for
                                  Calculating Quality-Adjusted
                                  Journal Output over Lifetime
                                           Comes from

                               RePEc          Coupe         K&Y_all

Quality-adjusted journals      0.117          0.119          0.113
                             (5.94) ***     (5.83) ***     (6.08) ***
Total citations                0.020          0.025          0.025
                               (1.32)         (1.62)        (1.76) *
Experience (years)             0.016          0.018          0.019
                             (2.80) ***     (3.09) ***     (3.29) ***
Experience squared             -0.027         -0.032         -0.031
                             (2.40) **      (2.85) ***     (2.81) ***
Male                           0.022          0.035          0.025
                               (0.47)         (0.72)         (0.52)
Constant                       11.910         11.892         11.872
[R.sup.2]                   (139.32) ***   (141.34) ***   (139.50) ***
                               0.657          0.660          0.655
Vuong non-nested test (a)     2.66 ***        2.46"         2.64 ***

                                  Journal Weighting Scheme for
                                  Calculating Quality-Adjusted
                                  Journal Output over Lifetime
                                           Comes from

                              K&Y_econ         KMS             LP

Quality-adjusted journals      0.112          0.097          0.112
                             (5.91) ***     (4.81) ***     (4.20) ***
Total citations                0.026          0.030          0.013
                              (1.80) *      (2.02) **        (0.60)
Experience (years)             0.019          0.020          0.021
                             (3.24) ***     (3.33) ***     (3.53) ***
Experience squared             -0.031         -0.032         -0.036
                             (2.77) ***     (2.85) ***     (3.16) ***
Male                           0.026          0.024          0.032
                               (0.53)         (0.47)         (0.65)
Constant                       11.876         11.876         11.865
[R.sup.2]                   (139.45) ***   (132.73) ***   (135.59) ***
                               0.654          0.639          0.641
Vuong non-nested test (a)     2.62 **         1.92 *        2.22 **

Notes: Dependent variable is log of base salary for the
2010 academic year, as reported at: http://www.sacbee.com/statepay/
and journal output and the number of citations are standardized so
coefficients show the impact of a one standard deviation change.
Fixed effects for each UC campus (with UC Berkeley the omitted
category), an indicator for academics not on standard contracts,
and an indicator for the proportion of published articles not in
WoS journals (so citations appear zero) are not reported.
N = 167 associate and full professors, robust r-statistics
in parentheses.

(a) The Vuong non-nested test is for comparing a model using
journal output versus a model using citations (with both models
having all of the other control variables); it is distributed
as a z-statistic, with significant positive values favoring the
journals model over the citations model.

* Significant at 10%; ** significant at 5%;
*** significant at 1%.

TABLE 3
Sensitivity Analyses for Salary Regressions Using Lifetime
Quality-Adjusted Journal Output and WoS Citations

                                Journal Weighting Scheme for
                                Calculating Quality-Adjusted
                                Journal Output over Lifetime
                                         Comes from

                                MSF          CLm          CLh

A. Including the count of articles as a covariate

Quality-adjusted journals      0.108        0.123        0.096
                             (4.44) ***   (5.00) ***   (4.38) ***
Total citations                0.013        0.003        0.001
                               (0.86)       (0.18)       (0.03)
Number of articles             0.033        0.037        0.077
                               (1.50)      (1.80) *    (4.47) ***
[R.sup.2]                      0.679        0.688        0.682

B. Using log-transformed citations (following HRH 2015)

Quality-adjusted journals      0.095        0.109        0.083
                             (3.59) ***   (4.14) ***   (3.59) ***
ln(citations + 1)              0.079        0.054        0.056
                               (1.25)       (0.81)       (0.81)
Number of articles             0.029        0.034        0.069
[R.sup.2]                      (1.26)       (1.57)     (3.41) ***
                               0.683        0.690        0.685

C. Using prorated citations

Quality-adjusted journals      0.112        0.129        0.101
                             (4.64) ***   (5.25) ***   (4.58) ***
Total citations per author     0.005        -0.006       -0.008
                               (0.42)       (0.50)       (0.61)
Number of articles             0.034        0.036        0.077
[R.sup.2]                      (1.52)      (1.77) *    (4.60) ***
                               0.678        0.688        0.683

D. Including controls for the most frequent JEL letter
category used in published articles by each academic

Quality-adjusted journals      0.120        0.125        0.097
                             (4.14) ***   (4.50) ***   (3.91) ***
Total citations                0.008        -0.001       -0.002
                               (0.50)       (0.04)       (0.13)
Number of articles             0.030        0.037        0.077
                               (1.10)       (1.58)     (3.89) ***
[R.sup.2]                      0.704        0.708        0.702

E. Including diversity index for lifetime articles
(based on 137 JEL 2-digit categories)

Quality-adjusted journals      0.100        0.117        0.091
                             (4.11) ***   (4.66) ***   (4.13) ***
Total citations                0.016        0.005        0.003
                               (101)        (0.33)       (0.20)
Number of articles             0.034        0.035        0.072
                               (1.52)      (1.69) *    (4.17) ***
Diversity index                0.002        0.002        0.002
                               (1.49)      (1.76) *     (1.81) *
[R.sup.2]                      0.683        0.693        0.688

F. Omitting fixed effects for each campus

Quality-adjusted journals      0.182        0.212        0.184
                             (5.59) ***   (6.15) ***   (5.56) ***
Total citations                0.015        -0.009       -0.021
                               (0.77)       (0.50)       (1.11)
Number of articles             -0.022       -0.023       0.039
                               (0.68)       (0.78)       (1.63)
[R.sup.2]                      0.409        0.449        0.462

                                Journal Weighting Scheme for
                                Calculating Quality-Adjusted
                                Journal Output over Lifetime
                                         Comes from

                               RePEc        Coupe       K&Y_all

A. Including the count of articles as a covariate

Quality-adjusted journals      0.080        0.081        0.083
                             (4.27) ***   (3.67) ***   (4.79) ***
Total citations                0.015        0.021        0.014
                               (0.93)       (1.27)       (0.90)
Number of articles             0.068        0.060        0.077
                             (3.36) ***   (2.84) ***   (4.09) ***
[R.sup.2]                      0.673        0.670        0.678

B. Using log-transformed citations (following HRH 2015)

Quality-adjusted journals      0.068        0.068        0.072
                             (3.38) ***   (2.64) ***   (3.91) ***
ln(citations + 1)              0.087        0.094        0.077
                               (1.37)       (1.44)       (1.21)
Number of articles             0.060        0.054        0.069
[R.sup.2]                    (2.70) ***   (2.38) **    (3.15) ***
                               0.678        0.675        0.682

C. Using prorated citations

Quality-adjusted journals      0.083        0.083        0.085
                             (4.44) ***   (3.81) ***   (4.95) ***
Total citations per author     0.008        0.014        0.007
                               (0.70)       (1.18)       (0.64)
Number of articles             0.069        0.062        0.079
[R.sup.2]                    (3.44) ***   (2.95) ***   (4.23) ***
                               0.672        0.669        0.677

D. Including controls for the most frequent JEL letter
category used in published articles by each academic

Quality-adjusted journals      0.085        0.082        0.087
                             (3.64) ***   (3.33) ***   (4.18) ***
Total citations                0.011        0.017        0.009
                               (0.64)       (0.99)       (0.53)
Number of articles             0.069        0.065        0.078
                             (2.78) ***   (2.45) **    (3.48) ***
[R.sup.2]                      0.694        0.692        0.699

E. Including diversity index for lifetime articles
(based on 137 JEL 2-digit categories)

Quality-adjusted journals      0.075        0.074        0.077
                             (3.96) ***   (3.27) ***   (4.41) ***
Total citations                0.017        0.023        0.016
                               (109)        (1.41)       (1.06)
Number of articles             0.065        0.059        0.074
                             (3.18) ***   (2.76) ***   (3.91) ***
Diversity index                0.002        0.002        0.002
                              (1.89) *     (1.75) *      (1.62)
[R.sup.2]                      0.679        0.676        0.683

F. Omitting fixed effects for each campus

Quality-adjusted journals      0.164        0.164        0.157
                             (6.58) ***   (7.10) ***   (7.12) ***
Total citations                0.004        0.016        0.003
                               (0.25)       (0.94)       (0.20)
Number of articles             0.020        0.003        0.041
                               (0.79)       (0.11)      (1.84) *
[R.sup.2]                      0.430        0.421        0.446

                                Journal Weighting Scheme for
                                Calculating Quality-Adjusted
                                Journal Output over Lifetime
                                         Comes from

                              K&Y_econ       KMS           LP

A. Including the count of articles as a covariate

Quality-adjusted journals      0.082        0.070        0.098
                             (4.71) ***   (4.01) ***   (4.05) ***
Total citations                0.014        0.014        -0.010
                               (0.93)       (0.91)       (0.51)
Number of articles             0.077        0.089        0.098
                             (4.09) ***   (4.72) ***   (5.74) ***
[R.sup.2]                      0.677        0.671        0.684

B. Using log-transformed citations (following HRH 2015)

Quality-adjusted journals      0.071        0.057        0.080
                             (3.85) ***   (2.88) ***   (3.45) ***
ln(citations + 1)              0.078        0.080        0.052
                               (1.23)       (1.21)       (0.74)
Number of articles             0.069        0.079        0.087
[R.sup.2]                    (3.15) ***   (3.61) ***   (3.99) ***
                               0.681        0.675        0.686

C. Using prorated citations

Quality-adjusted journals      0.084        0.072        0.105
                             (4.86) ***   (4.17) ***   (4.28) ***
Total citations per author     0.008        0.007        -0.019
                               (0.68)       (0.59)       (1.22)
Number of articles             0.079        0.091        0.100
[R.sup.2]                    (4.23) ***   (4.90) ***   (6.02) ***
                               0.677        0.671        0.685

D. Including controls for the most frequent JEL letter
category used in published articles by each academic

Quality-adjusted journals      0.086        0.072        0.097
                             (4.12) ***   (3.43) ***   (3.41) ***
Total citations                0.010        0.010        -0.009
                               (0.58)       (0.63)       (0.43)
Number of articles             0.078         0.09        0.096
                             (3.48) ***   (4.09) ***   (4.87) ***
[R.sup.2]                      0.698        0.693        0.701

E. Including diversity index for lifetime articles
(based on 137 JEL 2-digit categories)

Quality-adjusted journals      0.076        0.063        0.097
                             (4.35) ***   (3.61) ***   (4.00) ***
Total citations                0.017        0.017        -0.008
                               (1.09)       (1.10)       (0.42)
Number of articles             0.074        0.085        0.092
                             (3.91) ***   (4.45) ***   (5.34) ***
Diversity index                0.002        0.002        0.003
                              (1.66) *     (1.68) *    (2.35) **
[R.sup.2]                      0.682        0.676        0.692

F. Omitting fixed effects for each campus

Quality-adjusted journals      0.156        0.143        0.196
                             (6.88) ***   (5.50) ***   (7.01) ***
Total citations                0.004        0.004        -0.046
                               (0.23)       (0.24)     (2.26) **
Number of articles             0.041        0.059        0.079
                              (1.84) *    (2.56) **    (3.95) ***
[R.sup.2]                      0.445        0.424        0.492

Notes: Each regression also includes all of the other
variables from Table 2.N= 167, robust t-statistics
in parentheses.

* Significant at 10%; ** significant at 5%;
*** significant at 1%.

TABLE 4
Citations Appear to Be a More Important Influence
on Salary in Lower Ranked Departments

                              Journal Weighting Scheme for
                              Calculating Quality-Adjusted
                              Journal Output over Lifetime
                                        Comes from

                               MSF          CLm          CLh

A. Using the subsample from the top four
departments (Berkeley, UCSD, UCLA, UC Davis)

Quality-adjusted journals     0.148        0.156        0.130
                            (6.19) ***   (5.98) ***   (4.59) ***
Total citations               0.003        -0.005       0.004
                              (0.18)       (0.28)       (0.24)
[R.sup.2]                     0.702        0.704        0.675

B. Using the sub-sample from economics
departments at the other (lower ranked) UC campuses

Quality-adjusted journals     0.089        0.101        0.093
                            (2.37) **    (2.63) **    (2.27) **
Total citations               0.083        0.077        0.083
                            (2.13) **     (1.91) *    (2.34) **
[R.sup.2]                     0.609        0.618        0.611

C. Interaction effects between Top 4 indicator
and quality-adjusted journals and citations
(i.e., test of A-B differences)

Quality-adjusted journals     0.037        0.027        0.020
                              (0.83)       (0.54)       (0.33)
Total citations               -0.230       -0.225       -0.224
                            (2.53) **    (2.40) **    (2.75) ***

                              Journal Weighting Scheme for
                              Calculating Quality-Adjusted
                              Journal Output over Lifetime
                                        Comes from

                              RePEc        Coupe       K&Y_all

A. Using the subsample from the top four
departments (Berkeley, UCSD, UCLA, UC Davis)

Quality-adjusted journals     0.130        0.138        0.120
                            (5.77) ***   (6.19) ***   (5.88) ***
Total citations               0.013        0.018        0.022
                              (0.89)       (1.14)       (1.40)
[R.sup.2]                     0.684        0.694        0.678

B. Using the sub-sample from economics
departments at the other (lower ranked) UC campuses

Quality-adjusted journals     0.065        0.066        0.067
                             (1.95) *      (1.51)      (1.89) *
Total citations               0.095        0.091        0.093
                            (2.89) ***   (2.24) **    (2.78) ***
[R.sup.2]                     0.586        0.582        0.586

C. Interaction effects between Top 4 indicator
and quality-adjusted journals and citations
(i.e., test of A-B differences)

Quality-adjusted journals     0.066        0.061        0.054
                             (1.72) *      (1.21)       (1.36)
Total citations               -0.109       -0.097       -0.096
                            (3.05) ***   (2.23) **    (2.65) ***

                              Journal Weighting Scheme for
                              Calculating Quality-Adjusted
                              Journal Output over Lifetime
                                        Comes from

                             K&Y_econ       KMS           LP

A. Using the subsample from the top four
departments (Berkeley, UCSD, UCLA, UC Davis)

Quality-adjusted journals     0.119        0.089        0.145
                            (5.72) ***   (3.79) ***   (4.41) ***
Total citations               0.022        0.031        -0.007
                              (1.43)      (1.92) *      (0.41)
[R.sup.2]                     0.677        0.644        0.673

B. Using the sub-sample from economics
departments at the other (lower ranked) UC campuses

Quality-adjusted journals     0.069        0.075        0.075
                             (1.98) *    (2.02) **     (1.90) *
Total citations               0.092        0.092        0.098
                            (2.72) ***   (3.02) ***   (3.70) ***
[R.sup.2]                     0.588        0.595        0.601

C. Interaction effects between Top 4 indicator
and quality-adjusted journals and citations
(i.e., test of A-B differences)

Quality-adjusted journals     0.051        0.018        0.042
                              (1.27)       (0.47)       (0.92)
Total citations               -0.094       -0.082       -0.118
                            (2.57) ***   (2.32) **    (3.59) ***

Notes: Each regression in Panel A and B also includes all
of the other variables from Table 2, while in Panel C the
specification from Table 2 is augmented with an indicator
for the top four departments and the interaction terms between
that indicator and quality-adjusted journals and total citations.
N = 104 in the top four and N = 63 in the other five departments.
Robust t-statistics in parentheses.

* Significant at 10%; ** significant at 5%;
*** significant at 1%.

TABLE 5

Salary Regressions Comparing Impact of Journal Articles
and WoS Citations for Articles of Different Vintages

                               Journal Weighting Scheme for
                               Calculating Quality-Adjusted
                               Journal Output over Lifetime
                                        Comes from

                               MSF          CLm          CLh

A. Results from Table 3A, including
articles published until end of 2010

Quality-adjusted journals     0.108        0.123        0.096
                            (4.44) ***   (5.00) ***   (4.38) ***
Total citations               0.013        0.003        0.001
                              (0.86)       (0.18)       (0.03)
Number of articles            0.033        0.037        0.077
[R.sup.2]                     (1.50)      (1.80) *    (4.47) ***
                              0.679        0.688        0.682

B. Omitting articles published in prior 3 years
(but citations for older articles counted to end of 2010)

Quality-adjusted journals     0.100        0.120        0.096
                            (4.13) ***   (4.87) ***   (4.39) ***
Total citations               0.017        0.005        0.001
                              (1.07)       (0.29)       (0.04)
Number of articles            0.043        0.044        0.082
                             (1.79) *    (1.99) **    (4.41) ***
[R.sup.2]                     0.676        0.686        0.682

C. Omitting articles published in prior 6 years
(but citations for older articles counted to end of 2010)

Quality-adjusted journals     0.102        0.117        0.090
                            (3.79) ***   (4.28) ***   (3.94) ***
Total citations               0.023        0.014        0.011
                              (1.52)       (0.91)       (0.71)
Number of articles            0.034        0.038        0.076
                              (1.24)       (1.54)     (3.59) ***
[R.sup.2]                     0.649        0.658        0.652

                               Journal Weighting Scheme for
                               Calculating Quality-Adjusted
                               Journal Output over Lifetime
                                        Comes from

                              RePEc        Coupe       K&Y_all

A. Results from Table 3A, including
articles published until end of 2010

Quality-adjusted journals     0.080        0.081        0.083
                            (4.27) ***   (3.67) ***   (4.79) ***
Total citations               0.015        0.021        0.014
                              (0.93)       (1.27)       (0.90)
Number of articles            0.068        0.060        0.077
[R.sup.2]                   (3.36) ***   (2.84) ***   (4.09) ***
                              0.673        0.670        0.678

B. Omitting articles published in prior 3 years
(but citations for older articles counted to end of 2010)

Quality-adjusted journals     0.071        0.068        0.076
                            (3.74) ***   (2.96) ***   (4.21) ***
Total citations               0.018        0.024        0.016
                              (1.12)       (1.42)       (1.02)
Number of articles            0.077        0.072        0.083
                            (3.49) ***   (3.07) ***   (3.97) ***
[R.sup.2]                     0.670        0.666        0.675

C. Omitting articles published in prior 6 years
(but citations for older articles counted to end of 2010)

Quality-adjusted journals     0.065        0.062        0.063
                            (3.22) ***   (2.60) **    (3.41) ***
Total citations               0.026        0.030        0.025
                              (1.61)      (1.83) *     (1.66) *
Number of articles            0.073        0.068        0.081
                            (2.96) ***   (2.62) ***   (3.40) ***
[R.sup.2]                     0.641        0.637        0.642

                               Journal Weighting Scheme for
                               Calculating Quality-Adjusted
                               Journal Output over Lifetime
                                        Comes from

                             K&Y_econ       KMS           LP

A. Results from Table 3A, including
articles published until end of 2010

Quality-adjusted journals     0.082        0.070        0.098
                            (4.71) ***   (4.01) ***   (4.05) ***
Total citations               0.014        0.014        -0.010
                              (0.93)       (0.91)       (0.51)
Number of articles            0.077        0.089        0.098
[R.sup.2]                   (4.09) ***   (4.72) ***   (5.74) ***
                              0.677        0.671        0.684

B. Omitting articles published in prior 3 years
(but citations for older articles counted to end of 2010)

Quality-adjusted journals     0.074        0.071        0.092
                            (4.14) ***   (4.17) ***   (3.72) ***
Total citations               0.017        0.014        -0.008
                              (1.08)       (0.96)       (0.38)
Number of articles            0.084        0.092        0.104
                            (3.98) ***   (4.54) ***   (5.49) ***
[R.sup.2]                     0.674        0.674        0.681

C. Omitting articles published in prior 6 years
(but citations for older articles counted to end of 2010)

Quality-adjusted journals     0.061        0.068        0.083
                            (3.36) ***   (3.64) ***   (3.38) ***
Total citations               0.026        0.021        0.006
                             (1.69) *      (1.43)       (0.33)
Number of articles            0.081        0.085        0.097
                            (3.41) ***   (3.76) ***   (4.50) ***
[R.sup.2]                     0.642        0.645        0.650

Notes: Each regression also includes all of the other
variables from Table 2. N = 167, robust r-statistics
in parentheses.

* Significant at 10%; ** significant at 5%;
*** significant at 1%.

TABLE 6
Salary Regression Sensitivity Analyses that Consider
the Distribution of Citations

                                   Journal Weighting Scheme for
                                   Calculating Quality-Adjusted
                                   Journal Output over Lifetime
                                             Comes from

                                    MSF          CLm          CLh

A. Including total citations and citations to the most-cited article

Quality-adjusted journals          0.120        0.135        0.107
                                 (4.58) ***   (5.20) ***   (4.60) ***
Number of articles                 0.041        0.047        0.090
                                  (1.85) *    (2.38) **    (5.28) ***
Total citations                    -0.063       -0.083       -0.081
                                   (1.42)      (1.89) *     (1.73) *
Citations to most-cited            0.068        0.076        0.072
                                  (1.66) *     (1.90) *     (1.73) *
[R.sup.2]                          0.682        0.692        0.686
Vuong non-nested test             3.09 ***     3.54 ***     3.41 ***

B. Including the h-indexfor citations

Quaiity-adjusted journals          0.097        0.112        0.085
                                 (3.78) ***   (4.56) ***   (3.88) ***
Number of articles                 0.019        0.027        0.061
                                   (0.79)       (1.17)     (2.88) ***
Citations h-index                  0.045        0.030        0.033
                                   (1.36)       (0.90)       (0.97)
[R.sup.2]                          0.681        0.689        0.684
Vuong non-nested test               1.47       2.07 **       1.79 *

C. Including the generalized [h.sub.(5,2)]
index for citations

Quality-adjusted journals          0.102        0.115        0.089
                                 (4.21) ***   (4.76) ***   (4.17) ***
Number of articles                 0.035        0.038        0.076
                                   (1.59)      (1.90) *    (4.56) ***
Citations [h.sub.(5,2)]-index      0.032        0.018        0.017
                                   (1.26)       (0.70)       (0.61)
[R.sup.2]                          0.682        0.689        0.683
Vuong non-nested test             2.84 ***     3.32 ***     3.21 ***

                                   Journal Weighting Scheme for
                                   Calculating Quality-Adjusted
                                   Journal Output over Lifetime
                                             Comes from

                                   RePEc        Coupe       K&Y_all

A. Including total citations and citations to the most-cited article

Quality-adjusted journals          0.089        0.093        0.095
                                 (4.43) ***   (4.04) ***   (4.84) ***
Number of articles                 0.078        0.068        0.090
                                 (3.66) ***   (3.12) ***   (4.69) ***
Total citations                    -0.053       -0.053       -0.073
                                   (1.13)       (1.16)       (1.59)
Citations to most-cited            0.061        0.066        0.078
                                   (1.44)       (1.58)      (1.85) *
[R.sup.2]                          0.675        0.673        0.682
Vuong non-nested test             2.94 ***     2.63 ***     3.28 ***

B. Including the h-indexfor citations

Quaiity-adjusted journals          0.070        0.069        0.075
                                 (3.75) ***   (2.90) ***   (4.26) ***
Number of articles                 0.046        0.043        0.059
                                  (1.89) *     (1.72) *    (2.53) **
Citations h-index                  0.054        0.054        0.044
                                   (1.65)       (1.56)       (1.36)
[R.sup.2]                          0.676        0.673        0.680
Vuong non-nested test               1.22         0.98         1.59

C. Including the generalized [h.sub.(5,2)]
index for citations

Quality-adjusted journals          0.074        0.074        0.078
                                 (4.03) ***   (3.17) ***   (4.50) ***
Number of articles                 0.070        0.064        0.079
                                 (3.55) ***   (3.08) ***   (4.32) ***
Citations [h.sub.(5,2)]-index      0.031        0.035        0.025
                                   (1.18)       (1.27)       (0.92)
[R.sup.2]                          0.675        0.672        0.679
Vuong non-nested test             2.78 ***     2.47 **      3.09 ***

                                   Journal Weighting Scheme for
                                   Calculating Quality-Adjusted
                                   Journal Output over Lifetime
                                             Comes from

                                  K&Y_econ       KMS           LP

A. Including total citations and citations to the most-cited article

Quality-adjusted journals          0.094        0.082        0.102
                                 (4.77) ***   (3.86) ***   (4.08) ***
Number of articles                 0.091        0.102        0.108
                                 (4.66) ***   (5.26) ***   (5.94) ***
Total citations                    -0.073       -0.071       -0.061
                                   (1.57)       (1.31)       (1.27)
Citations to most-cited            0.078        0.075        0.046
                                  (1.85) *      (1.60)       (1.10)
[R.sup.2]                          0.681        0.675        0.686
Vuong non-nested test             3.25 ***     2.92 ***     3.54 ***

B. Including the h-indexfor citations

Quaiity-adjusted journals          0.073        0.060        0.081
                                 (4.19) ***   (3.03) ***   (3.84) ***
Number of articles                 0.059        0.067        0.077
                                 (2.52) **    (2.84) ***   (3.35) ***
Citations h-index                  0.045        0.051        0.036
                                   (1.39)       (1.42)       (1.04)
[R.sup.2]                          0.680        0.674        0.686
Vuong non-nested test               1.56         1.10        1.90 *

C. Including the generalized [h.sub.(5,2)]
index for citations

Quality-adjusted journals          0.077        0.064        0.084
                                 (4.45) ***   (3.33) ***   (4.02) ***
Number of articles                 0.079        0.090        0.093
                                 (4.32) ***   (5.00) ***   (5.67) ***
Citations [h.sub.(5,2)]-index      0.025        0.024        0.020
                                   (0.96)       (0.84)       (0.75)
[R.sup.2]                          0.678        0.672        0.685
Vuong non-nested test             3.06 ***     2.76 ***     3.23 ***

Notes: Each regression has all of the other variables from Table 2.
N = 167, robust t-statistics in parentheses. The Vuong test is of
a model using citations variables versus a model using journals
variables (both having all control variables), with positive
values favoring the journals model.

* Significant at 10%; ** significant at 5%; *** significant at 1%.

TABLE 7
Salary Regressions using Lifetime Quality-Adjusted
Journal Output, Counts of Elite and Excellent
Journals, and Citations

                                 Journal Weighting Scheme for
                                 Calculating Quality-Adjusted
                                 Journal Output over Lifetime
                                          Comes from

                                 MSF          CLm          CLh

A. Using quality-, size- and coauthor-adjusted journal
pages, and journal tiers to measure journal output

Quality-adjusted journals       0.086        0.110        0.089
                              (2.71) ***   (2.98) ***   (2.38) **
Number of articles              0.033        0.032        0.058
                                (1.21)       (1.21)     (2.11) **
ln(total citations + 1)         0.094        0.088        0.096
                                (1.25)       (1.16)       (1.26)
ln(total citations + 1) (2)     -0.046       -0.049       -0.053
                                (1.41)       (1.55)       (1.64)
# Elite articles                0.007        0.002        -0.001
                                (1.28)       (0.27)       (0.09)
# Excellent articles            -0.001       -0.001       0.001
                                (0.20)       (0.20)       (0.17)
[R.sup.2]                       0.689        0.694        0.689
F-test: tier-counts = 0 (a)      1.04         0.11         0.96

B. Using quality-, size- and coauthor-adjusted
journal pages to measure journal output

Quality-adjusted journals       0.097        0.113        0.088
                              (3.69) ***   (4.30) ***   (3.78) ***
Number of articles              0.022        0.026        0.062
                                (0.93)       (1.16)     (2.91) ***
ln(total citations + 1)         0.115        0.092        0.095
                                (1.57)       (1.23)       (1.26)
ln(total citations + 1) (2)     -0.045       -0.049       -0.053
                                (1.38)       (1.55)      (1.68) *
[R.sup.2]                       0.687        0.694        0.689
F-test: citations                1.44         1.33         1.55
effect = 0 (a)

                                 Journal Weighting Scheme for
                                 Calculating Quality-Adjusted
                                 Journal Output over Lifetime
                                          Comes from

                                RePEc        Coupe      K&Y_all

A. Using quality-, size- and coauthor-adjusted journal
pages, and journal tiers to measure journal output

Quality-adjusted journals       0.047        0.042       0.056
                               (1.80) *     (1.49)     (2.25) **
Number of articles              0.046        0.042       0.055
                               (1.67) *     (1.53)     (2.00) **
ln(total citations + 1)         0.101        0.094       0.095
                                (1.36)      (1.21)       (1.27)
ln(total citations + 1) (2)     -0.043      -0.037       -0.041
                                (1.33)      (1.16)       (1.23)
# Elite articles                0.008        0.010       0.006
                                (1.35)     (1.95) *      (1.05)
# Excellent articles            0.002        0.002       0.002
                                (0.74)      (0.87)       (0.55)
[R.sup.2]                       0.684        0.683       0.686
F-test: tier-counts = 0 (a)      0.93        1.98         0.58

B. Using quality-, size- and coauthor-adjusted
journal pages to measure journal output

Quality-adjusted journals       0.070        0.066       0.073
                              (3.48) ***   (2.57) **   (3.95) ***
Number of articles              0.054        0.051       0.064
                              (2.31) **    (2.16) **   (2.80) ***
ln(total citations + 1)         0.121        0.123       0.109
                               (1.67) *     (1.60)       (1.48)
ln(total citations + 1) (2)     -0.044      -0.034       -0.040
                                (1.38)      (1.08)       (1.23)
[R.sup.2]                       0.681        0.677       0.685
F-test: citations                1.56        1.30         1.23
effect = 0 (a)

                                 Journal Weighting Scheme for
                                 Calculating Quality-Adjusted
                                 Journal Output over Lifetime
                                          Comes from

                               K&Y_econ       KMS           LP

A. Using quality-, size- and coauthor-adjusted journal
pages, and journal tiers to measure journal output

Quality-adjusted journals       0.054        0.027        0.090
                              (2.18) **      (0.82)     (2.63) ***
Number of articles              0.054        0.056        0.067
                              (1.98) **    (1.99) **    (2.30) **
ln(total citations + 1)         0.096        0.103        0.095
                                (1.28)       (1.36)       (1.26)
ln(total citations + 1) (2)     -0.041       -0.040       -0.071
                                (1.23)       (1.21)     (2.16) **
# Elite articles                0.006        0.009        0.001
                                (1.06)       (1.12)       (0.13)
# Excellent articles            0.002        0.003        0.002
                                (0.61)       (0.96)       (0.56)
[R.sup.2]                       0.686        0.681        0.695
F-test: tier-counts = 0 (a)      0.56         0.75         0.85

B. Using quality-, size- and coauthor-adjusted
journal pages to measure journal output

Quality-adjusted journals       0.072        0.058        0.094
                              (3.91) ***   (2.90) ***   (3.77) ***
Number of articles              0.064        0.074        0.078
                              (2.81) ***   (3.28) ***   (3.53) ***
ln(total citations + 1)         0.111        0.113        0.098
                                (1.51)       (1.48)       (1.32)
ln(total citations + 1) (2)     -0.040       -0.040       -0.074
                                (1.22)       (1.19)     (2.32) **
[R.sup.2]                       0.684        0.678        0.694
F-test: citations                1.26         1.22        2.76 *
effect = 0 (a)

Notes: The dependent variable is log of 2010 academic
year salary and each model also includes all of the other
variables, which are not measures of research productivity
and impact, which are included in the models in Table 2. N = 167.
Robust t- statistics in parentheses. The # Elite articles counts
articles published by each academic in the American Economic
Review, the Journal of Political Economy, the Quarterly Journal
of Economics, Econometrica, and the Review of Economics and
Statistics. The # Excellent articles counts articles published
in the other 31 journals of Scott and Mitias (1996). The
quadratic in log citations and the counts of articles in
elite and excellent journals follow the specification
used in Table 6B by HRH (2015).

(a) This is an F-test for coefficients
(in Panel A on # Elite articles and # Excellent articles,
and in Panel B on the polynomial in log citations)
being jointly zero.

* Significant at 10%; ** significant at 5%;
*** significant at 1%.

TABLE 8
Estimated Effects of Citations with Different
Approaches to Controlling for Quality-Adjusted
Journals in the HRH Sample

Approach to Measuring Journal Output Comes from

                                   HRH       MSF       CLm

Full sample (N = 1,009)

Incremental [R.sup.2] (a)        0.0375    0.0382    0.0320
% change from using                         -1.8%     17.2%
journal tiers (b)
Incremental salary (c)           $28,944   $27,763   $25,855
% change from using                         4.3%      11.9%
journal tiers (b)

Top-ranked departments NRC 2010 ranks 1-49 (N=485)

Incremental [R.sup.2] (a)        0.0281    0.0266    0.0188
% change from using                         5.6%      49.5%
journal tiers (b)
Incremental salary (c)           $29,065   $27,403   $23,402
% change from using                         6.1%      24.2%
journal tiers (b)

Lower ranked departments NRC 2010 ranks 50-120 (N = 524)

Incremental [R.sup.2] (a)        0.0552    0.0564    0.0560
% change from using                         -2.1%     -1.4%
journal tiers (b)
Incremental salary (c)           $21,847   $20,631   $20,709
% change from using                         5.9%      5.5%
journal tiers (b)

Approach to Measuring Journal Output Comes from

                                   CLh      RePEc     Coupe

Full sample (N = 1,009)

Incremental [R.sup.2] (a)        0.0284    0.0318    0.0336
% change from using               32.0%     17.9%     11.6%
journal tiers (b)
Incremental salary (c)           $24,749   $25,994   $26,794
% change from using               16.9%     11.3%     8.0%
journal tiers (b)

Top-ranked departments NRC 2010 ranks 1-49 (N=485)

Incremental [R.sup.2] (a)        0.0170    0.0206    0.0272
% change from using               65.3%     36.4%     3.3%
journal tiers (b)
Incremental salary (c)           $22,540   $24,484   $27,979
% change from using               28.9%     18.7%     3.9%
journal tiers (b)

Lower ranked departments NRC 2010 ranks 50-120 (N = 524)

Incremental [R.sup.2] (a)        0.0545    0.0558    0.0528
% change from using               1.3%      -1.1%     4.5%
journal tiers (b)
Incremental salary (c)           $20,634   $20,763   $20,568
% change from using               5.9%      5.2%      6.2%
journal tiers (b)

Approach to Measuring Journal Output Comes from

                                 K&Y_all   K&Y_eco     KMS       LP

Full sample (N = 1,009)

Incremental [R.sup.2] (a)        0.0302    0.0303    0.0318    0.0269
% change from using               24.2%     23.8%     17.9%     39.4%
journal tiers (b)
Incremental salary (c)           $25,458   $25,526   $26,196   $23,987
% change from using               13.7%     13.4%     10.5%     20.7%
journal tiers (b)

Top-ranked departments NRC 2010 ranks 1-49 (N=485)

Incremental [R.sup.2] (a)        0.0219    0.0216    0.0229    0.0177
% change from using               28.3%     30.1%     22.7%     58.8%
journal tiers (b)
Incremental salary (c)           $25,209   $25,146   $26,394   $22,417
% change from using               15.3%     15.6%     10.1%     29.7%
journal tiers (b)

Lower ranked departments NRC 2010 ranks 50-120 (N = 524)

Incremental [R.sup.2] (a)        0.0541    0.0545    0.0543    0.0529
% change from using               2.0%      1.3%      1.7%      4.3%
journal tiers (b)
Incremental salary (c)           $20,693   $20,736   $20,523   $20,501
% change from using               5.6%      5.4%      6.4%      6.6%
journal tiers (b)

Notes: The base model is that reported in column (5),
Table 6B of HRH (2015). That model for log salary of
1,009 economists from 53 different schools has as explanatory
variables: quadratics in experience, in the logarithm of lifetime
citations (plus one), and in seniority, dummy variables for males
and for each school, and counts of the number of articles published
in Elite, Excellent, and Other journals, where Elite and Excellent
are defined in Table 7. Models in the columns headed MSF though LP
control for quality-adjusted journal output of each economist by
using the number of size-, quality-, and coauthor adjusted journal
pages published in the career to the end of 2006 (using the type
of journal weight indicated by the column heading), and the count of
these articles, which is the same approach used in Tables 2 to 7 of
the current study.

(a) The increase in [R.sup.2] for each model when the logarithm
and squared logarithm of lifetime citations (plus one) are added
as explanatory variables.

(b) The percentage increase in either the incremental [R.sup.2]
or the incremental salary when the HRH approach to measuring
quality-adjusted journal output is used rather than using the
adjusted journal pages approach (based on the type of journal
weights indicated by the column heading).

(c) The predicted increase in salary from a standard deviation
increase in citations. Duan's (1983) smearing estimator is used,
given the log dependent variable.

TABLE 9
Comparison of Salary Determinants for Top-Ranked
Schools and Other Schools in the HRH Sample

                                   Journal Weighting Scheme
                               for Calculating Quality-Adjusted
                                Journal Output over Lifetime
                                          Comes from

                                 MSF          CLm          CLh

A. Top-ranked departments NRC 2010 ranks 1-49 (N = 485)

Quality-adjusted journals       0.066        0.082        0.070
                              (3.18) ***   (3.96) ***   (3.59) ***
Number of articles              0.022        0.025        0.045
                                (0.99)       (1.17)     (2.17) **
ln(total citations + 1)         0.023        0.023        0.024
                                (1.36)       (1.37)       (1.41)
ln(total citations + 1) (2)     0.005        0.004        0.004
                              (2.09) **      (1.61)       (1.43)
Experience                      0.022        0.022        0.023
                              (3.60) ***   (3.63) ***   (3.82) ***
Experience (2)                  -0.000       -0.000       -0.000
                              (3.23) ***   (3.27) ***   (3.49) ***
Seniority                       -0.019       -0.018       -0.018
                              (4.05) ***   (3.81) ***   (3.80) ***
Seniority (2)                   0.000        0.000        0.000
                              (2.70) ***   (2.59) ***   (2.57) **
Male                            0.050        0.043        0.042
                               (1.95) *     (1.74) *     (1.70) *
[R.sup.2]                       0.645        0.651        0.650

B. Lower ranked departments NRC 2010 ranks 50-120 (N = 524)

Quality-adjusted journals       -0.013       -0.008       -0.004
                                (0.70)       (0.45)       (0.21)
Number of articles              0.076        0.071        0.068
                              (3.85) ***   (3.85) ***   (3.98) ***
ln(total citations + 1)         0.009        0.009        0.009
                                (0.63)       (0.62)       (0.63)
ln(total citations + 1) (2)     0.008        0.008        0.008
                              (3.23) ***   (3.18) ***   (3.09) ***
Experience                      0.010        0.010        0.010
                              (2.22) **    (2.20) **    (2.16) **
Experience (2)                  -0.000       -0.000       -0.000
                                (0.70)       (0.70)       (0.67)
Seniority                       -0.019       -0.019       -0.019
                              (4.26) ***   (4.24) ***   (4.20) ***
Seniority (2)                   0.000        0.000        0.000
                              (2.97) ***   (2.95) ***   (2.92) ***
Male                            -0.010       -0.010       -0.010
                                (0.53)       (0.52)       (0.52)
[R.sup.2]                       0.662        0.662        0.662
F-test for pooling             3.27 ***     3.10 ***     2.71 ***
samples (a)

                                   Journal Weighting Scheme
                              for Calculating Quality-Adjusted
                                 Journal Output over Lifetime
                                           Comes from

                                RePEc        Coupe       K&Y_all

A. Top-ranked departments NRC 2010 ranks 1-49 (N = 485)

Quality-adjusted journals       0.064        0.052        0.055
                              (3.39) ***   (2.49) **    (3.17) ***
Number of articles              0.039        0.032        0.047
                               (1.87) *      (1.39)     (2.24) **
ln(total citations + 1)         0.026        0.025        0.027
                                (1.50)       (1.43)       (1.53)
ln(total citations + 1) (2)     0.004        0.005        0.004
                                (1.57)     (2.04) **      (1.62)
Experience                      0.023        0.023        0.023
                              (3.60) ***   (3.67) ***   (3.72) ***
Experience (2)                  -0.000       -0.000       -0.000
                              (3.15) ***   (3.26) ***   (3.32) ***
Seniority                       -0.019       -0.020       -0.019
                              (3.88) ***   (4.10) ***   (3.98) ***
Seniority (2)                   0.000        0.000        0.000
                              (2.54) **    (2.71) ***   (2.66) ***
Male                            0.046        0.052        0.046
                               (1.84) *    (2.05) **     (1.86) *
[R.sup.2]                       0.647        0.642        0.645

B. Lower ranked departments NRC 2010 ranks 50-120 (N = 524)

Quality-adjusted journals       -0.008       -0.002       -0.004
                                (0.49)       (0.11)       (0.25)
Number of articles              0.070        0.067        0.068
                              (4.07) ***   (3.70) ***   (4.00) ***
ln(total citations + 1)         0.009        0.010        0.009
                                (0.60)       (0.64)       (0.61)
ln(total citations + 1) (2)     0.008        0.008        0.008
                              (3.19) ***   (3.02) ***   (3.07) ***
Experience                      0.010        0.010        0.010
                              (2.19) **    (2.15) **    (2.16) **
Experience (2)                  -0.000       -0.000       -0.000
                                (0.69)       (0.68)       (0.67)
Seniority                       -0.019       -0.019       -0.019
                              (4.23) ***   (4.18) ***   (4.21) ***
Seniority (2)                   0.000        0.000        0.000
                              (2.93) ***   (2.91) ***   (2.93) ***
Male                            -0.010       -0.010       -0.010
                                (0.52)       (0.53)       (0.52)
[R.sup.2]                       0.662        0.662        0.662
F-test for pooling             2 92 ***     2.71 ***     2.61 ***
samples (a)

                                  Journal Weighting Scheme
                              for Calculating Quality-Adjusted
                                 Journal Output over Lifetime
                                          Comes from

                               K&Y_econ       KMS           LP

A. Top-ranked departments NRC 2010 ranks 1-49 (N = 485)

Quality-adjusted journals       0.055        0.047        0.065
                              (3.13) ***   (2.63) ***   (3.63) ***
Number of articles              0.047        0.050        0.054
                              (2.25) **    (2.39) **    (2.60) ***
ln(total citations + 1)         0.027        0.022        0.029
                                (1.53)       (1.29)      (1.67) *
ln(total citations + 1) (2)     0.004        0.005        0.003
                                (1.63)      (1.96) *      (1.20)
Experience                      0.023        0.024        0.023
                              (3.71) ***   (3.87) ***   (3.74) ***
Experience (2)                  -0.000       -0.000       -0.000
                              (3.31) ***   (3.54) ***   (3.41) ***
Seniority                       -0.019       -0.019       -0.019
                              (3.97) ***   (4.00) ***   (3.98) ***
Seniority (2)                   0.000        0.000        0.000
                              (2.65) ***   (2.68) ***   (2.71) ***
Male                            0.046        0.046        0.041
                               (1.85) *     (1.89) *     (1.66) *
[R.sup.2]                       0.645        0.644        0.649

B. Lower ranked departments NRC 2010 ranks 50-120 (N = 524)

Quality-adjusted journals       -0.005       -0.001       -0.001
                                (0.31)       (0.09)       (0.05)
Number of articles              0.068        0.067        0.066
                              (4.03) ***   (3.92) ***   (3.85) ***
ln(total citations + 1)         0.009        0.010        0.010
                                (0.60)       (0.65)       (0.65)
ln(total citations + 1) (2)     0.008        0.008        0.008
                              (3.10) ***   (3.08) ***   (3.02) ***
Experience                      0.010        0.010        0.010
                              (2.17) **    (2.14) **    (2.14) **
Experience (2)                  -0.000       -0.000       -0.000
                                (0.67)       (0.67)       (0.67)
Seniority                       -0.019       -0.019       -0.019
                              (4.21) ***   (4.18) ***   (4.18) ***
Seniority (2)                   0.000        0.000        0.000
                              (2.93) ***   (2.90) ***   (2.89) ***
Male                            -0.010       -0.010       -0.010
                                (0.52)       (0.52)       (0.53)
[R.sup.2]                       0.662        0.662        0.662
F-test for pooling             2.38 ***     2.52 ***     2.56 ***
samples (a)

Notes: The base model is that reported in column (5),
Table 6B of HRH (2015) for the log salary of 1,009 economists
in 53 schools. The journal quality measures and article counts
are used instead of the counts of articles in "elite,"
"excellent," and "other" journals used by HRH. The models
also include unreported fixed effects for each school.
Robust t-statistics in parentheses.

(a) This is an F-test for interaction terms on all
coefficients being jointly zero, with 9, and
938 degrees of freedom.

* Significant at 10%; ** significant at 5%;
*** significant at 1%.