Periodical usefulness: The U.S. logistics educator perspective

Logistics professionals face a challenge today - how to best allocate precious budgetary resources and reading time to a growing array of logistics-related publications. Time-challenged members of the logistics community would benefit from the knowledge of what publications are widely used and found to be of value. This article generates insight into these issues and provides guidance for individuals in search of valuable information tools.

Unlike traditional studies of this nature, which base evaluations on the academic prestige or the research impact of periodicals (e.g., Heischmidt and Gordon 1993; Howard and Nikolai 1983), this study focuses on functional usefulness. Certainly, educators will have a direct interest in their peers' ratings of the periodicals in terms of research, outreach, and teaching merits. Additionally, logistics managers can use the rating results to help identify appropriate information resources for analogous industry activities (e.g., employee training and personal professional development).

The analysis of periodical usefulness is presented in four major sections. The first two sections lay the foundation for the study, discussing the ideal structure of a periodical ranking study and how the current study was designed to attain an accurate assessment of logistics periodical usefulness from the target population. The third section reveals results of the study, including periodical ratings, group comparisons, a bias analysis, and usefulness index scores. The final section presents insights gleaned from the study, its potential uses by key constituents, and its caveats.

BACKGROUND

The evaluation of periodicals is neither a trivial topic nor one of exclusive interest to educators. Studies providing a benchmark of specialty periodicals can be used for a variety of purposes, in addition to those described above. The evaluation results can be used by practitioners to become more aware of valuable resources and information outlets (Fawcett, Vellenga, and Truitt 1995), university administrators to assess faculty research performance (Hull and Wright 1990), and periodical publishers to keep their editorial objectives and content in sync with reader needs (Zivney and Reichenstein 1994).

Survey-based studies have been widely used across business disciplines to perform evaluations of relevant periodicals. In many of these studies, experts in the particular discipline evaluated periodicals using individual Likert scale assessments of quality, prestige, impact, relevance, timeliness, and/or readability (Coe and Weinstock 1983; Heischmidt and Gordon 1993; Hull and Wright 1990; Malouin and Outreville 1987). The most recent logistics periodical study combined multiple variables to arrive at an overall quality and prestige perception rating for each periodical (Fawcett, Vellenga, and Truitt 1995). Also, a limited number of studies have factored usage, readership, or popularity into their analyses (Browne and Becker 1991; Hult, Neese, and Bashaw 1997; Luke and Doke 1987). It is believed that these more expansive studies provide stronger and more reliable evaluations of periodical importance (Hult, Neese, and Bashaw 1997).

Previous studies targeted university faculty and administrators in the relevant discipline as the survey population (Heischmidt and Gordon 1993; Howard and Nikolai 1983). These individuals are viewed as having the greatest familiarity with and expertise regarding the periodicals, and thus constitute the most appropriate population for evaluation studies. For example, the most recent studies of logistics periodicals targeted United States college and university professors in the logistics and transportation field (Fawcett, Vellenga, and Truitt 1995; Ferguson 1975).

Traditionally, periodical evaluation studies focused narrowly on the research value of peer reviewed academic journals (Benjamin and Brenner 1974; Browne and Becker 1991; Heischmidt and Gordon 1993; Hult, Neese, and Bashaw 1997; Malouin and Outreville 1987). More recent studies have widened the scope of evaluation studies, citing one of two main reasons. First, academic- and practitioner-focused publications both benefit educators and warrant evaluation. Also, many publications are broad based and should not be judged exclusively on their research value (Hull and Wright 1990; Hult, Neese, and Bashaw 1997; Mason and Steagall 1997).

RESEARCH DESIGN

Based on the research goals and the knowledge gleaned from the background research, an expert opinion survey was developed. Key activities included: identifying relevant periodicals, designing the survey instrument, and defining the survey population.

First, an integrated list of 47 research journals and practitioner-oriented publications was compiled from past logistics journal ranking study results (Fawcett, Vellenga, and Truitt 1995), input from 22 logistics educators, and a review of serial holdings from three university libraries. Additional information regarding these periodicals (official title, publisher name and location, and ISSN) was assembled. An alphabetized list of the periodicals and related information was created for distribution with each survey.

Next, a four-page survey instrument was developed, tested, and revised. The questionnaire instructed respondents to identify up to ten periodicals that they use most frequently for their logistics research activities. Then, the respondents were asked to assess the merit of these periodicals using a five-point scale ( = low and 5 = high). The factors used in this assessment were the periodical's quality of articles, its value to their logistics research activities, and its impact on the discipline. The same process and similar factors were used to collect information regarding the respondents' use and perceptions of periodicals for logistics outreach activities and logistics teaching activities, respectively. Additionally, the respondents were asked to assess the statements "I am very familiar with this periodical" and "I regularly read this periodical" for 47 publications using a five-point scale (1 = strongly disagree and 5 = strongly agree).

The survey population - U.S. logistics faculty at four-year institutions - was chosen for their familiarity with pertinent periodicals and their direct involvement in logistics research, outreach, and teaching. The database was developed using multiple conference attendee lists and professional organization directories. Journal articles listing authors of logistics studies (Gentry, Allen, and Vellenga 1995; Miyazaki, Phillips, and Phillips 1999), a published listing of universities with relevant logistics coursework (Council of Logistics Management 1999), and university websites were also used to ensure that a diverse group of educators was included in the population. In total, 171 potential respondents were identified.

The questionnaire, explanatory cover letter, and periodical list were mailed to the survey population. Three reminders were sent to non-respondents every four weeks after the initial mailing. Two questionnaires were returned as undeliverable because of wrong or changed addresses. The effective sample size, then, was 169.

One hundred two surveys were completed and returned. The 60.4% response rate was comparable to the most recent logistics periodical evaluation study (Fawcett, Vellenga, and Truitt 1995). Experts from 58 institutions participated in the study. Table 1 provides a breakdown of the respondents by various criteria.

RESULTS AND DISCUSSION

The respondents identified and evaluated their ten most widely used periodicals in each education-related activity. In total, 290 top ten usage lists were created - 102 for research, 90 for outreach, and 98 for teaching. These lists included 42 periodicals from the periodical information sheet distributed with the survey and 39 others identified by individual respondents.

In many instances, a limited number of respondents identified and evaluated a particular publication. To promote effective statistical analysis, avoid over-interpretation of the data, and provide a more balanced periodical ranking, only the 22 publications identified in greater than 10% of the 290 top ten usage lists are included in this examination. Summary information for these periodicals is provided in Table 2. Additional details for all periodicals involved in the research can be found in Appendix A at the end of this article.

A key influence on the respondents' top ten lists appeared to be the periodicals' subject matter. Periodicals with broad coverage of logistics and supply chain issues dominated the rankings. These titles also received high and consistent mean merit ratings across research, teaching, and outreach.

In contrast, periodicals focusing on a single aspect of logistics or a related field received fewer respondent top ten rankings. They also tended to receive more moderate merit ratings across the three activities. Still, these related field periodicals (e.g., Decision Sciences, Journal of Operations Management, and Journal of Marketing) warranted inclusion in Table 2, unlike some periodicals (e.g., Transportation Law Journal and Transportation Science) that had received high ratings in previous studies (Fawcett, Vellenga, and Truitt 1995). Logistics educators have broadened their resource base to include periodicals from other functional areas that complement logistics research, outreach, and teaching.

Demographic Group Comparisons

One objective of the research was to analyze the influence of key groups on the periodical merit ratings from Table 2. T-tests were performed to examine differences in mean research, outreach, and teaching merit ratings for each periodical between the following groups:

* Tenured faculty versus non-tenured faculty

* Research institution faculty versus teaching/balanced institution faculty

It was believed that these demographic pairings would display the greatest ratings contrasts, if any.

T-tests revealed significant differences in mean merit ratings for only seven periodicals, although analyses were conducted across the groups for each logistics activity (research, outreach, and teaching). In general, there is consensus between the different demographic groups regarding the merit of the most frequently used periodicals. Table 3 highlights the noteworthy t-test results.

Involvement Bias Analysis

Another goal of the research was to investigate merit ratings bias in the survey responses. Survey-based periodical rating studies are often criticized for their lack of objectivity (Mason and Steagall 1997). In some studies, journal ratings varied by the raters' special interests (Peery and Adams 1981), familiarity with the periodicals (Levin and Kratochwill 1976), or the publication records of the respondents (Cudd and Morris 1988).

In the current study, the respondents' involvement with a periodical serves as the basis of bias analysis. T-tests were performed to examine differences in mean merit ratings between respondents who have worked with a particular periodical as an author, reviewer, or editor, and respondents not directly involved with the periodical.

None of the t-tests revealed significant differences in mean merit ratings between the two groups. Unlike studies in other fields (Cudd and Morris 1988), our research respondents involved with particular periodicals did not artificially inflate the ratings and bias the survey results. The general agreement between the two groups (along with the previous t-test results) suggests that a general consensus exists among logistics educators regarding the value of these 22 periodicals.

Periodical Usefulness Index Development

The final objective of the research was to develop an overall assessment of each periodical's usefulness across the three key activities. This assessment is based on the respondents' periodical readership activity and their related top ten merit ratings. The sizable data set with similar ratings by demographic groups and lack of involvement bias provides reasonable support for taking this summarization step in the analysis.

Periodical usefulness is characterized as a combination of its merit across the three key educational activities and its usage. Readership activity level was used as the indicator of usage in lieu of familiarity, as the latter does not confirm actual use. The usefulness index score for each periodical was developed using the data contained in the last four columns of Table 2. Each category mean was converted to a 25-point scale item in a 100-point usefulness index as follows:

Usefulness Index Score = Usage Score + Research Merit Score + Outreach Merit Score + Teaching Merit Score

Where: Usage Score = mean readership activity level x 5

Research Merit Score = mean research merit rating x 5

Outreach Merit Score = mean outreach merit rating x 5

Teaching Merit Score = mean teaching merit rating x 5

For example, the usefulness index score of 77.70 for Transportation Journal was calculated as follows: (3.36 x 5) + (4.19 x 5) + (4.01 x 5) + (3.98 x 5). The results of usefulness index score calculations are provided in Table 4.

Table 4 provides a number of noteworthy results and interesting insights into the perceptions of the survey respondents. First and foremost is the usefulness index score attained by the Journal ofBusiness Logistics. It is the most widely read periodical by the audience and deemed the most useful in terms of research merit. It also attained strong outreach and teaching merit scores.

Another striking result is the respondents' strong perceptions and extensive use of publications that are not traditional peer-reviewed journals. Unlike other disciplines where few if any non-academic journals achieve high rankings in these types of studies, logistics educators rate such publications highly. In fact, five of the top ten publications found in Table 4 rely upon invited articles, editorially reviewed articles, and articles by professional journalists. Harvard Business Review and Supply Chain Management Review stand out among these publications, achieving almost identical usefulness index scores. Both are deemed very useful for outreach and teaching purposes.

Finally, the usefulness index scores highlight the respondents' propensity to draw upon a diverse set of resources. Journals focusing on international logistics issues are widely used and/ or highly rated by the respondents, as are publications oriented to specific supply chain issues (e.g., Production and Inventory Management Journal), and broader business topics (e.g., Harvard Business Review). The expanding scope of logistics research, outreach, and teaching activities enhances the usefulness of such periodicals to logistics educators.

IMPLICATIONS AND CONCLUSIONS

The research presented in this article represents a departure from earlier logistics periodical evaluation studies. Three primary differences - the combination of peer-reviewed journals and practitioner publications in a common ranking set, the open-ended nature by which respondents identified their top ten lists for rating purposes, and the inclusion of three independent activity ratings plus a usage factor in the development of an overall usefulness index - produced interesting and unique findings that have already been discussed.

User Implications

Although the study population consisted exclusively of U.S. logistics educators, the usefulness index scores and related rankings provide insights that should be of interest to multiple stakeholders. Key interest groups include educators, practitioners, university administrators, and periodical administrators. Critical implications for each group are discussed below.

Logistics educators - The diverse group of highly rated publications suggests that it is possible for logistics educators to find relevant information for their logistics research, outreach, and teaching activities in numerous and varied publications. Logistics educators are not necessarily constrained to a narrow group of topics or publication outlets in their activities. The results suggest that it is possible to make a significant contribution through multiple types of publication outlets. Finally, the results could be used as another tool for benchmarking performance and developing a targeted list of outlets for future publications.

Logistics practitioners - While most managers do not actively conduct research, they are often involved in training, conference presentations, collaborative initiatives, and other industry-related roles. Thus, the periodical scores and rankings, especially those related to teaching and outreach activities, can be of value to this group. Knowing which periodicals their full-time education counterparts find most useful will help managers streamline their information searches and identify pertinent instructional and outreach materials.

In addition, the research results can serve two other purposes for practitioners. First, Appendix A provides managers with an extensive list of relevant publications. This list can be consulted when making subscription purchase decisions. Second, the results provide insight regarding the research, outreach, and teaching focus of logistics educators. This should be of interest to managers since they rely upon educators to prepare future managers for careers in logistics and provide insight into the key trends and issues in the field.

University administrators - In many institutions, department chairpersons and other individuals responsible for the evaluation of logistics faculty performance are unfamiliar with the field. The study results provide an external source of information regarding the tri-fold usefulness of peerreviewed logistics journals. These usefulness ratings could be used as one of multiple inputs in the creation or revision of a target list of research publication outlets for their faculty. However, administrators are cautioned against using these results exclusively as many prestigious niche journals simply do not have broad appeal across the three activity areas and did not receive a usefulness index score. Appendix A reveals that some of these journals have strengths in a particular activity that should be recognized accordingly.

Administrators should also take note of the types of publications logistics educators find to be most useful. The results suggest that practitioner-focused periodicals and related field journals are important to the tri-fold responsibilities of most logistics faculty. Additionally, publication of articles in such outlets is valuable in terms of applied scholarship according to the primary accrediting agency for university business programs (Association to Advance Collegiate Schools of Business International 2001). Administrators should weight this combination of peer value and intellectual contribution accordingly when evaluating faculty participation in these non-traditional outlets.

Periodical administrators - The dearth of narrowly focused academic journals and practitioner publications in the rankings serves as a warning signal to publishers and editors. The influence of broader logistics and supply chain issues on the respondents' activities may portend a shift in subscriptions and article submissions. Specialty topic periodical editors may find it necessary to adapt to these changes in order to remain relevant and financially viable. This may involve changing a periodical's strategic focus, devoting editions to special topics, or achieving economies through joint publication with other periodicals (e.g., similar to what the Journal of the Transportation Research Forum and Transportation Quarterly have already done).

Research Limitations

In addition to these implications it is also important to address the study's limitations. While appropriate steps were taken to ensure that the results are reliable, valid, and unbiased, concerns regarding the sampling frame, the range of periodicals analyzed, access to periodicals, and the ability to generalize the research results are acknowledged.

First, the sampling of logistics educators could be viewed as too narrow. However, given the specific goal of the research - to evaluate publications germane to the logistics discipline logistics educators were the logical survey population. Given the response rates and lack of bias among respondent groups, the authors believe that the results adequately depict U.S. logistics educators' perceptions of journal usefulness for conducting logistics research, outreach, and teaching activities.

Second, the inclusion of some non-logistics journals in the study could be viewed as problematic. Considering the survey population, it was expected that widely distributed logistics periodicals would receive higher usefulness index scores than non-logistics periodicals due to their popularity and direct applications. Thus, no head-to-head comparisons of journal usefulness index scores were made and no statements were made to the effect that logistics periodicals are "better" than the marketing or operations management journals that appeared in the rankings. In fact, the appearance of these related field journals among the top 22 was highlighted as a positive finding in the results.

Third, the gratis distribution of some periodicals (e.g., Journal ofBusiness Logistics, Inbound Logistics, CLM Annual Conference Proceedings) to the survey population could be seen as having an unfairly high impact on readership levels and the corresponding usage index score. However, it would be imprudent to assume that access solely influences use which suggests that educators indiscriminately rely upon whatever arrives in their mailbox. Also, access to most of the other 22 publications is not a problem as Appendix B highlights. Widespread circulation, publication websites, Internet accessible databases of full-text articles, university library serial holdings, and interlibrary loan programs put even the most expensive publications within the reach of logistics educators.

Finally, given these three limitations and the original intent of the research, the results of the study should not be broadly generalized. The rankings may not reflect the periodical usefulness perceptions of educators that work primarily in another field but occasionally participate in logistics activities. Their perceptions of logistics and interest areas may lead these educators to use a vastly different group of periodicals. Also, the usefulness index scores apply specifically to logistics education activities and are not transferable to related disciplines. The scores reported in this study do not necessarily reflect the value of the journals for marketing, operations, or supply chain management education activities.

Future Directions

While these limitations do not detract from the value of the current study, they highlight some opportunities for future research into periodical usefulness. The expansion of the survey population to include logistics educators from different countries and logistics practitioners would provide additional perspectives on the current topic. Given the convergence of multiple disciplines on supply chain management issues, it would also be valuable to compare the usefulness of logistics, operations, and marketing periodicals for supply chain related education activities. Finally, it will be valuable to repeat the current study in another five to seven years to identify trends and track changes in logistics periodical usefulness. By then, it is likely that new periodicals and Internet-based information outlets will emerge and play an important role in logistics educators' activities.

In sum, the current research revisits the topic of periodical usefulness which is important given the emergence of new periodicals, the expansion of logistics education, and the increasing attention being focused on logistics-related issues. It provides a timely and unique analysis of logistics educators' perceptions of periodical usefulness, highlights important implications for key stakeholders, and identifies opportunities for extending the research.

NOTES

Association to Advance Collegiate Schools of Business (2001), "Standards for Business Accreditation," retrieved February 1, 2002 from: http://www.aacsb.edu/accreditation/business/ BusinessStandards2000.pdf.

Benjamin, James and Vincent Brenner (1974), "Perceptions of Journal Quality," The Accounting Review, Vol. 49, No. 2, pp. 360-362.

Browne, William G. and Boris W. Becker (1991), "A Longitudinal Study of Marketing Journal Familiarity and Quality," AMA Educators' Proceedings: Enhancing Knowledge Development in Marketing, Chicago: American Marketing Association, pp. 702-710.

Coe, Robert C. and Irwin Weinstock (1983), "Evaluating Finance Journals: The Department Chairperson's Perspective," Journal of Finance Research, Vol. 6, No. 4, pp. 345-349.

Council of Logistics Management, "Logistics Program Coursework," retrieved December 1, 1999 from: http://www.clmi.org/education/coursework.asp.

Cudd, Mike and Joe Morris (1988), "Bias in Journal Ratings," The Financial Review, Vol. 23, No. 1, pp. 117-125.

Fawcett, Stanley E., David B. Vellenga, and Lawrence J. Truitt (1995), "An Evaluation of Logistics and Transportation Professional Organizations, Programs, and Publications," Journal of Business Logistics, Vol. 16, No. 1, pp. 299-314.

Ferguson, Wade (1975), "An Evaluation of Journals that Publish Business Logistics Articles," Transportation Journal, Vol. 22, No. 4, pp. 69-72.

Gentry, Julie J., Benjamin J. Allen, and David B. Vellenga (1995), "Affiliation of Authors in Transportation and Logistics Academic Journals - Revisited," Transportation Journal, Vol. 35, No. 3, pp. 54-62.

Heischmidt, Kenneth A. and Peter Gordon (1993), "Rating Marketing Publications: Impact of Accreditation and Publication History," Journal of Education for Business, Vol. 68, No. 3, pp. 152-158. Howard, Thomas and Loren Nikolai (1983), "Attitude Measurement and Perceptions of Accounting Faculty Publication Outlets," The Accounting Review, Vol. 58, No. 4, pp. 765-776.

Hull, Rita P. and Gail B. Wright (1990), "Faculty Perceptions of Journal Quality: An Update," Accounting Horizons, Vol. 4, No. 1, pp. 77-101.

Hult, G. Tomas M., William T. Neese, and R. Edward Bashaw (1997), "Faculty Perceptions of Marketing Journals," Journal ofMarketing Education, Vol. 19, No. 1, pp. 37-53.

Levin, Joel R. and Thomas R. Kratochwill (1976), "The Name of the Journal Fame Game: Quality or Familiarity?" American Psychologist, Vol. 31, No. 9, pp. 673-674.

Luke, Robert H. and E. Reed Doke (1987), "Marketing Journal Hierarchies: Faculty Perceptions, 1986-1987," Journal of the Academy ofMarketing Science, Vol. 15, No. 1, pp. 74-78.

Malouin, Jean-Louis and J. Francois Outreville (1987), "The Relative Impact of Economics Journals: ACross-Country Survey and Comparison," Journal of Economics andBusiness, Vol. 39, No. 3, 267-277.

Mason, Paul M. and Jeffery W. Steagall (1997), "Economics Journal Rankings By Type of School: Perceptions Versus Citations," Quarterly Journal of Business and Economics, Vol. 36, No. 1, pp. 69-81

Miyazaki, Anthony D., Jason K. Phillips, and Diane M. Phillips (1999), "Twenty Years of JBL: An Analysis of Published Research," Journal ofBusiness Logistics, Vol. 20, No. 2, pp. 1- 19.

Peery, J. Craig and Gerald R. Adams (1981), "Qualitative Ratings of Human Development Journals," Human Development, Vol. 24, No. 5, pp. 312-319.

Zivney, Terry L. and William Reichenstein (1994), "The Pecking Order in Finance Journals," Finance Practice and Education, Vol. 4, No. 2, pp. 77-88.

Brian J. Gibson

Joe B. Hanna

Auburn University

ABOUT THE AUTHORS

Brian J. Gibson is an Associate Professor of Logistics in the Department of Aviation Management and Logistics at Auburn University. Dr. Gibson holds a Ph.D. from the University of Tennessee. He is active in executive education and distance learning, supply chain research, and numerous professional organizations.

Joe B. Hanna is an Associate Professor of Logistics in the Department of Aviation Management and Logistics at Auburn University. Dr. Hanna holds a Ph.D. from New Mexico State University. He has published logistics articles in a variety of outlets, is active in continuing and distance education programs, and holds a membership in several professional organizations.

Copyright Council of Logistics Management 2003
Provided by ProQuest Information and Learning Company. All rights Reserved