Relevance of the hierarchical linear model to TIMSS data analyses

Multilevel international data have been released from the Third International Mathematics and Science Study (TIMSS), which provides an opportunity to apply multilevel modeling techniques in educational research. In this article, TIMSS factors are classified in fixed and random categories according to the project design. Relevant methods for the multilevel data analyses have been discussed in light of the fundamental relationship between the traditional dummy variable regression and the hierarchical linear modeling.

With more than 40 nations participating and five grade levels assessed in two subject areas, the Third International Mathematics and Science Study (TIMSS) is the largest and most comprehensive project in comparative education (Martin & Kelly, 1996). Secondary analyses of the TIMSS data may enrich information on the U.S. condition of education in a cross-nation context (Peak, 1996). Because TIMSS data have a hierarchical structure with students nested in schools and schools nested in countries, some researchers attempted to adopt the hierarchical linear model (HLM) for international comparisons. Based on the eighth grade science data, researchers of the TIMSS International Center reported, "An unconditional, three level analysis of science achievement indicates that the variance can be approportioned 26% within school, 35% between schools, [and] 39% between countries" (Gregory & Shen, 1999, p. 8). Meanwhile, Gregory and Shen (1999) cautioned that the lack of randomization at the country level may not need the variance partitioning in HLM. Since the choice of research methodology inevitably affects results of TIMSS data analyses, the purpose of this study is to examine features of the TIMSS design, and specify proper fixed and random factors within the hierarchical data structure. This investigation may help clarify relevance of HLM for TIMSS investigations.

Fixed and Random Factors in Hierarchical Modeling

Multilevel data can be analyzed by several software packages, such as FILM, M1wiN, and SAS (Wang, 1997). Accordingly, the terminology varies across the different computer software applications. Bryk and Raudenbush (1992) acknowledged in their book for HLM that,

The models discussed in this book appear in diverse literatures under a variety of titles.In sociological research, they are often referred to as multilevel linear models [cf. Goldstein, 1987; Mason et al., 1983]. In biometric applications, the terms mixed-effects models and randomeffects models are common [cf. Elston & Grizzle, 1962; Laird & Ware, 1982]. (p. 3)

At any levels of the hierarchy, factors can be either fixed or random, depending on the research design. Casella and Berger (1990) defined:

A factor is a fixed factor if all the values of interest are included in the experiment.

A factor is a random factor if the values of interest are not included in the experiment and those that are can be considered to be randomly chosen from the values of interest. (p. 529)

Milliken and Johnson (1984) added that "a model is called a mixed or mixed effects model if some of the factors in the treatment structure are fixed effects and some are random effects" (p. 213).

In TIMSS, research interest is confined in around 40 participating nations. Thus, country can be treated as a fixed factor, and the research findings are not designed for generalization over all countries in the world. On the other hand, schools and students are sampled randomly within each nation. Factors at these levels can be treated as random factors, and the sample statistics will be used to represent the condition of education in each nation. Therefore, the hierarchical model contains both fixed and random factors. The HLM computing, as a method of mixed-effects modeling, should be relevant to the TIMSS data analyses.

Dummy Factor Regression

Thus, statistical null hypotheses may differ across the levels of hierarchical structure, depending on a proper specification of the fixed and random effects. Whether random sampling has been incorporated at the national level does not affect the relevancy of HLM to the TIMSS data analyses. Instead, dummy variable regressions can be adopted along with the random coefficient modeling to specify the mixed structure of fixed and random effects in TIMSS. The clarification of research methodology may facilitate analyses of TIMSS data in a multilevel context.

References

Bryk, A. S. & Raudenbush, S. W. (1992). Hierarchical linear models. Newbury Park, CA: Sage.

Casella, G and Berger, R. L. (1990). Statistical inference. Pacific Grove, CA: Brooks/Cole Publishing.

Gregory, K. & Shen, C. (1999, April). Effective schools: An international perspective using

TIMSS science achievement-Grade 8. Paper presented at the Annual Meeting of the American Educational Research Association, Montreal, Canada.

Martin, M., & Kelly, D. (1996). Third international mathematics and science study: Technical report. Chestnut Hill, MA: Boston College.

Milliken, G. A. & Johnson, D. E. (1992). Analysis of messy data. Volume 1: Designed Experiments. New York, NY: Chapman and Hall.

Peak, L. (1996). Pursuing excellence (NCES No. 97-198). Washington, DC: U.S. Government Printing Office.

Wang, J. (1997). Using SAS PROC MIXED to demystify hierarchical linear model. Journal of Experimental Education, 66 (1), 84-93.

JIANJUN WANG

Department of Teacher Education School of Education California State University 9001 Stockdale Highway Bakersfield, CA 93311-1099

Copyright Project Innovation Summer 2000
Provided by ProQuest Information and Learning Company. All rights Reserved