Examining the average citation index of education in Rural Australia (now the Australian and International Journal of Rural Education).

Drummond, Aaron ; Halsey, R. John

INTRODUCTION

Since its inception in 1991, the Society for Provision of Education in Rural Australia's (SPERA's) flagship journal, Education in Rural Australia (1), has been attempting to raise the profile of rural education and rural education research in Australia. The diversity of SPERA's interests in rural education can be demonstrated by reaching for the most recent edition of the journal at the time of this article being written--the second issue of the 2011 edition. Chronicled within its pages are descriptions of pre-service teachers' beliefs about Education for Sustainability (Boon, 2011), the perceived importance of university presence within rural Australia (Drummond, Halsey & van Breda, 2011), and an examination of whether Vocational Education is being used as a substitute for university education in rural contexts (Curtis, 2011), amongst others. The issue prior this contains discussions of investment in rural social space for the promotion of sustainability (Lock, Reid & White, 2011), and the views of rural residents on rural sustainability (Halsey, 2011), to name a few. Clearly, rural educational research covers a diverse and important range of topics.

The editors, and the authors the journal has chronicled, have clearly attempted to bring quality research to bear upon rural education issues. In the present day however, there is an increasing push for research to be quantified in terms of its quality. The Excellence in Research Australia (ERA, not to be confused with Education in Rural Australia), attempted to categorise journals into quality bands, during which time Education in Rural Australia was classified as a B journal. Journals that similarly received a B ranking included the Australian Journal of Education, and the British Journal of Psychology.

Following the revoking of the bands, a renewed push to recognise citation indices as a proxy for quality has begun. Be it right or wrong, to be competitive within academic publishing, a journal must be able to quantify its quality in some manner. One method for examining the quality has been to use citation indices such as Thomson-Reuters' Impact Factor[tm]. Here, we seek to quantify the quality of Education in Rural Australia in a similar manner, by adopting an Average Citation Index (ACI) to examine the amount of citations the journal receives per article published.

METHOD

To determine the number of citations the articles within the journal received, we searched individually for each published article on Google Scholar, and then followed their "cited by" links. As in conventional citation measures, only citations from indexed journals were used for calculating the ACI--books, reports and non-peer reviewed journals were not included in the citation counts. To calculate the ACI for each two year period, we examined the number of citations a journal received during the ACI year to articles published in the two years prior to the ACI year. We then divided the number of citations received by the number of articles published by the journal in the two years prior to the ACI year. The formula can be described mathematically as:

ACI = [a + b]/[c + a]

Where:

a = the number of citations to items published two years prior to the ACI year

b = the number of citations to items published a year prior to the ACI year

c = the number of items published two years prior to the ACI year; and

d = the number of items published a year prior to the ACI year

Note that only citations during the ACI year count toward that year's ACI. So if a 2008 article was cited in 2011, the 2010 ACI would not change. As an example of a calculation of an ACI, for 2011's ACI, we divided the number of citations in 2011 to articles published in the journal during 2009 and 2010 by the total number of articles the journal published in 2009 and 2010. Mathematically:

2011ACI = [2009cites + 2010cites]/[2009articles + 2010articles]

An ACI over 5 years was also calculated by dividing the number of 2011 citations to articles published in the journal from 2006-2010 by the total number of articles the journal published in these years. Mathematically, the 5 yearly ACI for 2011 is expressed as:

5yearACI = [2006cites + 2007cites + 2008cites + 2009cites + 2010cites]/[2006articles + 2007articles + 2008articles + 2009articles + 2010articles]

As for the 2-year ACI, the 5-year ACI includes only citations within 2011 toward the ACI for that year. This means that for the journal to maintain its ACI, articles must continue to be cited each year.

RESULTS

Only scholarly articles were included in the ACI calculations--Editorial articles and messages to members were not included as citable items for the purposes of calculating ACI. One article in 2007 and two articles in 2006 were not indexed on Google Scholar. These articles were excluded from both citations and total article counts for any ACI calculations. Table 1. displays the 2-yearly ACI's for the last four years of the journal and the five-yearly ACI.

To determine the citation practices of the journal, the mean citation index was broken down between external citations, and citations made within Education in Rural Australia to articles within the journal (self-cites). The citation against self-citation indices are displayed in Figure 1.

[FIGURE 1 OMITTED]

DISCUSSION

The present results show that SPERA's journal, Education in Rural Australia (1) has a solid Average Citation Index, which has risen considerably in 2011 when compared to previous years. The ACI rise is characterised by a marked increase in self-citation however, and while it is important for a journal to respect and use its own work, it is vital that the journal's articles are cited outside of the journal also. Nonetheless, the citation index is higher than comparable Australian Education journals, for example, the Australian Journal of Education, which holds a 2011 Thomson-Reuters' Impact Factor[tm] of about 0.4. The 2011 impact factor would also rank the journal as 115th out of 204 education journals by Field of Research (FoR) code. The increasing number of articles with an international component (e.g., Barbour, 2011; Stevens, 2010) is also a good indicator of a wider community of research emerging within the journal's pages.

Fluctuation in the Journal's ACI in the last four years is concerning, as it appears from the data that the journal has 'on-years' with higher citation measures, and 'off-years' with lower ones. This may be indicative of the slow turn-around time of the journal itself--with research tending to be cited all at once during the on-years. This is supported by the fact that not a single publication in the journal was cited within the year of publication for any indexed year. These facts illustrate an essential enhancement to the journal--online-first publication. If online-first publication was available, the journal may receive citations to its articles more frequently, and the work could be used (and cited) immediately upon article acceptance.

The move to an online version of the SPERA journal is one that is characterised by a wide variety of advantages. Online publication would result in an increased ability to reach potential readers, an increase in the speed of publication (and hence citation) and a reduction in production costs for online-based subscriptions. Since presently, the journal is converted to a PDF format before printing, little additional work or technical expertise is required to facilitate the transition. Conversely, difficulties with internet access and inflated internet costs in rural and remote areas are indicative of the need to retain a print-based option for the journal also. These arguments are support for both an online and print-based publication model. The 2011 five-yearly ACI closely matches the 2-year ACI for 2011. Five year citation indices are more indicative of the long-term citation practices of a journal, and tend to be a more stable indicator of a journals use. That the 5-year and 2-year ACI's for 2011 are relatively close is a good indicator of these indices' reliability.

It should be noted that the method used to determine ACI in the present article is less than perfect. A recent attempt by a researcher to determine citation rates for the Public Library of Science One (PLoS ONE) used Scopus (2) to calculate the index. When the official citation index was released for the journal, the researcher found he had underestimated the journal's impact by about 1.5 (Beltrao, 2009). While doubtful that the present data underestimates the ACI by quite as much as PloS ONE's was (primarily due to the lower total volume of publications), it is likely that some underestimation has occurred. We believe, based on Beltrao's attempts, and the incomplete nature of cited by links on Google Scholar, that the journals ACI is likely to be a little higher than the 0.65 calculated.

In addition, a number of government reports, books and informal papers cite articles from the journal. These are not included in the ACI due to an attempt to keep the ACI as close to conventional citation measures as possible. The fact that articles in the journal are receiving citations from these kinds of papers is a good indication that the journal is having community impact, which is particularly important in the field of education, in which many users are likely to be practitioners.

One possible direction for the SPERA journal is to adopt an open access policy to publishing in an attempt to increase its readership. Doing this would have one primary advantage and disadvantage. While readership would almost certainly increase, income from journal subscriptions would likely fall. One option to remedy this is to create open access versions of articles only if all authors are current SPERA members. This would allow full dues-paying members to have open-access articles, and may increase membership numbers as well as the breadth of the journal's readership, without sacrificing subscriptions.

One advantage of calculating the ACI manually is that the recent name change of the journal should not affect future ACI calculations. Typically, journals indexed by traditional citation indices are dramatically affected by changing their name for the two years following the name change. This occurs because the citation indices are calculated for the journal against the previous years that journal's title has been indexed verbatim. It is therefore recommended that a manual calculation of ACI be used for 2012 and 2013, before an official citation index is sought in 2014. This will ensure that the 2012-2013 citation indices do not underestimate the journals true use. Overall the present results are indicative of a respectable Average Citation Index for the SPERA Journal, which has risen substantially during 2011. A concerted effort by educational researchers to stay abreast of current research directions and utilise the research should see the journals quantitative indices of use continue to rise.

Acknowledgements This research was funded by the Sidney Myer Foundation, The Myer Fund, and the Sidney Myer Chair of Rural Education and Communities. We are indebted to M. van Breda for comments on an earlier version of this paper.

REFERENCES

Barbour, M. K. (2011). The promise and the reality: exploring virtual schooling in rural jurisdictions. Education in Rural Australia, 21, 1-20.

Beltrao, P. (2009). Guestimating PloS ONE impact factor (Update). Public Rambling Blog, retrieved 21st June, 2012 from http://pbeltrao.blogspot.com.au/2009/04/guestimating-plos-one-impact-factor.html

Boon, H. J. (2011) Beliefs and education for sustainability in rural and regional South Australia. Education in Rural Australia, 21, 37-54.

Curtis, D. D. (2011) Tertiary education provision in rural Australia: Is VET a substitute for, or a pathway into, Higher Education? Education in Rural Australia, 21, 19-36.

Drummond, Halsey & van Breda, (2011). The perceived importance of university presence in rural Australia. Education in Rural Australia, 21, 1-18.

Halsey, R. J. (2011). Farm fair voices, space, history, the middle ground and 'The Future' of rural communities. Education in Rural Australia, 21, 39-64.

Lock, G., Reid, J. & White, S. (2011). Investing in sustainable and resilient rural social space: lessons for teacher education. Education in Rural Australia, 21, 67-78.

Stevens, K. (2010) From closed to open classes--repositioning schools to sustain rural communities. Education in Rural Australia, 20, 3-16.

Footnotes

(1) Now the Australian and International Journal of Rural Education

(2) We attempted to use Scopus as well, but found that Scopus does not currently index articles in the SPERA journal. This is an issue that requires rectifying since Scopus is becoming more common as a means of examining the citation quantities of individual articles, and the publication record of individual researchers.

Aaron Drummond * ([dagger]) & R. John Halsey ([dagger]) (^)

([dagger]) School of Education, Flinders University, Adelaide, Australia.

(^) Sidney Myer Chair of Rural Education and Communities.

* To whom correspondence should be addressed: C/-School of Education, Flinders University, GPO Box 2100, Adelaide, SA, 5001, Australia. Phone: +61 8 8201 3425. Fax: +618 8201 3184. Email address: Email address: aaron.drummond@flinders.edu.au

Table 1. Average Citation Index (ACI) across
the last four years, and a five-yearly ACI for
2011.

Year            ACI

2008            0.05
2009            0.40
2010            0.09
2011            0.65
2011 (5-year)   0.54