Testing the construct validity of the productivity environmental preference survey learning style inventory instrument.

Englander, Fred ; Terregrossa, Ralph A. ; Wang, Zhaobo 等

INTRODUCTION

Teaching effectiveness at the college level depends on the instructor's knowledge of his or her discipline and on the instructor's ability to convey that knowledge. The principles of education hypothesize that the most effective way to teach is to utilize a teaching method that most closely matches the way that the individual student learns. The objective of matching the instructor's teaching method with the student's learning style to maximize teaching effectiveness and student achievement is the cornerstone of the DDLSM (Rundle & Dunn, 2009b).

Particularly in introductory courses where class size tends to be relatively large, it is impractical for an instructor to design an optimal teaching method for each student. Recognizing the impracticality of formulating and implementing a tailor-made teaching method for each student, the DDLSM recommends that the instructor "identify individual and group patterns among students' learning style preferences and develop teaching style strategies to respond to those patterns" (Dunn, 2000, p. x).

This study examines the results from two groups of students who have completed the Building Excellence (BE) (Rundle and Dunn, 2009a) self-report learning style instrument which reflects the Dunn and Dunn Learning Style Model (DDLSM) (Rundle & Dunn, 2009b) in order to evaluate the construct validity of certain key components of the Productivity Environmental Preference Survey (PEPS) survey (Dunn, Dunn, & Price, 2006). Although there are substantial similarities between the two survey instruments, there are some differences in the manner in which the data from the respective instruments are collected. These differences allow a test of certain construct validity related assumptions in the PEPS. It is argued that to the extent that the construct validity of the PEPS is supported or rejected by the results of this analysis, it would be reasonable to make somewhat broader inferences regarding the internal validity of the overall DDLSM.

LITERATURE REVIEW

The ongoing growth of the academic literature on the use of learning style methods as an educational tool has been prodigious. A Google Scholar search of the term learning styles elicits nearly 1.6 million articles. In this vast literature, of course, there have been many studies undertaken which have been both favorable and critical of the general approach of using broadly defined learning style approaches (e.g., Rohrer & Pashler, 2012; Wilson, 2012) and specific learning style theories in order to accomplish improvements in student performance for students at all levels of education. One of the most widely cited articles on the Google Scholar list is the critical evaluation of learning style theory undertaken by Coffield, et al. (2004). They identify seventy-one separate learning style theories and then proceed to evaluate what they regard as the thirteen 'major' theories among the seventy-one, based on the quality of the research supporting the theory and the theory's impact on the "professional or academic audience." (Coffield, et al. 2004)

The Dunn and Dunn approach to learning styles is among what Coffield, et. al. (2004) consider to be the major learning style theories. In fact, the evaluation of the relative significance of various learning style theories undertaken by Desmedt and Valcke (2004), based on the academic citations of the lead authors of learning style related articles, places the DDLSM as second in importance among learning style theories, following the lead position held by David Kolb.

On issues relating to the question of the validity of the DDLSM (where 'validity' is a broader criterion for evaluation than the 'construct validity' criteria which is the focus of the present paper), as Coffield et al. (2004, p.27) report, Rita Dunn has offered robust claims regarding the validity of the DDLSM:

   Research on the Dunn and Dunn model of the learning style is more
   extensive and far more thorough than the research on any other
   educational movement, bar none. As of 1989, it had been conducted
   at more than 60 institutions of higher education, at multiple grade
   levels ... and with every level of academic proficiency, including
   gifted, average, underachieving, at risk, drop-out, special
   education and vocational/industrial arts populations.

Coffield et al., (2004) offer a much different view. They argue that the studies of validity that Dunn references in the above quote have been undertaken by Professor Dunn, herself, graduate students under her tutelage or by colleagues of Professor Dunn. Therefore, "there appears to be little independent evaluation of their model ... Data presented by Curry (1987) as evidence of good validity only confirmed predictive validity and not construct or face validity." (Coffield et al., 2004, p.28) Kavale and Forness (1990) also criticize the DDLSM for a lack of independent evaluations of validity. In reference to a meta-analysis of learning style research that Kavale and Forness (1987) had performed, the authors explained that it was not possible to include the majority of the studies of the DDLSM in their study because the research undertaken by analysts who were colleagues and graduate students of Professor Dunn was such that, "when even a cursory examination revealed a study to be so inadequate that its data were essentially meaningless, it was eliminated from consideration." (Kavale & Forness, 1990, p.358).

Coffield et al. (2004) review existing research and further criticize Dunn and Dunn for what Coffield et al. (2004) claim to be the inadequate evidence that is provided supporting the validity of the PEPS survey, the instrument utilized to ascertain the learning style preferences of students and others seventeen years old and older. The problems are said to relate to "missing data and the quality of Dunn et al.'s citations, referencing and interpretations of statistics." (Coffield, et al., 2004, p.30)

Focusing on the narrower issue of the construct validity of the DDLSM, Coffield et al. (2004) criticize the dichotomy that the DDLSM makes regarding the existence of global versus analytic learning style preferences. They suggest that it is unrealistic to portray these differences as "polar extremes" and indicate that the expert opinion of most cognitive psychologists and neuropsychologists is that learners don't have rigid preferences for one polar position on the global/analytic spectrum or the other. This criticism has been echoed by Ivie (2009).

THE TWO UTILIZED SURVEY INSTRUMENTS: PEPS AND BE

Two instruments that have been used in recent years to help measure a college student's learning style preferences, each consistent with the DDLSM, are the PEPS survey (Dunn, Dunn & Price, 2006) and the BE survey (Rundle & Dunn, 2009a). The PEPS (Dunn, Dunn and Price 2006) instrument is used to identify student preferences for the twenty elements comprising five categories of the learning style model. The PEPS, designed to identify how college students and other adults learn and perform in their academic and occupational pursuits, is a self-report survey composed of 100 questions that can be completed in approximately twenty minutes. Each question is intended to identify an individual's preference regarding each of the environmental, emotional, sociological, physiological, and psychological elements. For example, to determine their preference regarding sound, an environmental element, students are asked to answer whether they strongly disagree, disagree, are uncertain, agree, or strongly agree to a series of statements, such as:

1. I can block out noise or sound when I work.

2. I prefer to work with music playing.

3. Noise or extraneous sound usually keeps me from concentrating.

4. I can block out most sound when I work.

In a similar manner, preferences regarding all environmental, emotional, sociological, physiological and psychological elements are identified.

The BE (Rundle & Dunn, 2009a), is also a learning style survey instrument, but is based on a revised version of the DDLSM. It is composed of six learning style categories. Each category contains several related elements that describe, respectively, the environmental, emotional, sociological, perceptual, physiological and psychological dimensions of student learning style preferences. The psychological category identifies the way that students absorb and process new and difficult information, including a preference for global/analytic methods.

An important hypothesis of the DDLSM is that analytic and global learners have different environmental, emotional, sociological and physiological preferences. The model hypothesizes that there are particular preferences for noise, light, design, persistence, and intake that distinguish analytic learners from global learners (Dunn, 2000). The five discriminating learning style elements are listed below in Table 1 as well as the hypothesized sign of the correlation of each element with the analytic and global learning styles.

Analytic learners learn best in a quiet, brightly lighted and formal (e.g., sitting at a desk) environment. They like to work alone, tend to be persistent (e.g., prefer to start and finish one project at a time), and do not snack while learning. They also learn more easily when details are presented in a sequential, step-by-step manner that builds toward a conceptual understanding of the idea to be learned.

Global learners learn more easily when they understand the total concept first then subsequently focus on the underlying details. They learn best with background sound, soft light in a relaxed environment (e.g., sitting on a couch or in a coffee shop). They prefer to work with others, tend not to be persistent (e.g., work simultaneously on several projects), take frequent breaks, enjoy snacks when learning, and prefer to be taught with the use of illustrations and symbols. Global learners prefer new information to be presented anecdotally, especially in a humorous way that explains how the concept relates to them.

The BE learning style instrument measures the analytic and global dichotomous element independently of the measures of the five discriminating elements. That is, the BE utilizes a set of survey questions to directly identify the analytic and global learning style element that is distinct from an alternative set of questions that is used to identify the discriminating learning style elements of noise, light, design, persistence and intake. The procedure used to measure a student's preference for analytic/global learning methods via the PEPS instrument is more indirect. A student's learning style preference for analytic/global learning is inferred from that student's preferences of the five, discriminating learning style variables which are hypothesized to determine that individual's analytic/global preference profile.

The central focus of this study is to empirically test this important hypothesis of the DDLSM: whether and to what extent the variable measuring the analytic versus global leaning style dichotomous element is correlated with measures of students' preferences for noise, light, design, persistence and intake. These five discriminating elements have been used in the DDLSM for survey instruments preceding the development of the BE survey, such as the PEPS survey, to determine the extent to which a student's learning style preferences could be identified as 'global' or 'analytic'.

Testing the statistical association between a student's status as 'global' or 'analytic' (as directly measured by the BE instrument) and the five discriminating elements constitutes a test of the construct validity of the PEPS which is an instrument for carrying out the DDLSM. The test consists of estimating a regression model whereby the dichotomous analytic/global learning style element is the dependent variable and the five discriminating learning style preference elements of noise, light, design, persistence and intake are the explanatory variables. If the sample of students is characterized either by an analytic or global learning style, then the parameters of the discriminating learning style preference variables should have the appropriate signs as summarized above in Table 1 and the five explanatory variables that were hypothesized in the DDLSM to underlie the broader analytic/global learning style element should explain a substantial proportion of the student to student variation in the dependent variable.

To the extent that the parameters of the explanatory variables have the expected sign and are statistically significant, the test described above allows an objective and independent evaluation of the construct validity of the PEPS learning style instrument--the analytic and global dichotomous learning style element could be reasonably well inferred from the five discriminating elements identified by the PEPS. It would also provide indirect support for the overall DDLSM.

LEARNING STYLES

The Dunn and Dunn (2000) learning styles model (Rundle & Dunn, 2009b) theorizes that an individual's learning style is composed of a combination of interrelated perceptual, environmental, physiological, emotional, sociological, and psychological categories. The perceptual category includes preferences for alternative perceptual modalities, including auditory, or learning by listening; visual-picture, or learning by seeing images, illustrations or pictures; visual-word, or learning by reading; tactile and/or kinesthetic, or learning through hands-on experience and by doing; and, verbal-kinesthetic, or learning by verbalizing.

The environmental category includes preferences for background sound versus silence, bright or soft light, cool or warm temperature and formal versus informal seating. The physiological category reflects the student's ability to remain energized, focused and alert. This category includes preferences for intake of snacks or drinks while learning, the time of day when the student does his or her best work, and whether the student needs to be moving while learning.

The emotional category includes preferences for internal versus external motivation, persistence, or starting and finishing one project at a time, conformity to societal norms, and structure, or a preference for internal or external direction. The sociological category reflects whether students prefer to work alone, with a partner or with a group of peers, and whether students prefer to learn with an authoritative versus collegial adult. This category also reflects whether students prefer to learn using a variety of methods or by using an established routine.

The psychological category includes the preference for either a reflective or compulsive approach to making decisions and solving problems. This category also identifies the students' thought processing method, hypothesized to include analytic or global processing methods. The integrated learners have both analytic and global characteristics and utilize the alternative styles depending on the nature of, and interest in, the material to be learned.

METHODOLOGY

Sample and Data Collection

Data for students' preferences for the discriminating learning style elements of sound, light, design, persistence and intake, and for the analytic and global cognitive processing method were collected from the BE survey that was administered to over twenty six hundred (2600) entering freshmen in the fall semester of 2004. The BE is an online survey containing 118 self-reflective questions that are answered on a five-point Likert scale. The BE survey identifies all twenty-six learning style elements contained in the six categories that comprise the DDLSM (Rundle & Dunn, 2009b). The sample analyzed in this study includes a total of sixty one students, twenty seven in economics and thirty four in accounting. Students in both courses were taught in a college of business, accredited by the AACSB, at a university located in the New York City. The macroeconomics course was taught by an associate professor of economics, and the accounting course was taught by an associate professor of accounting.

Method and Data Analysis

The least squares regression model was fitted to the aggregated data for both disciplines. Students' cognitive processing method, or the analytic and global learning style, served as the dependent variable and students' preferences for the five discriminating elements are included as explanatory variables. The direction of causality from the five discriminating elements to the analytic and global learning style is consistent with the Dunn and Dunn learning style model. The OLS regression model is summarized in equation (1):

[AG.sub.i] = [a.sub.0] + [5.summation over (b=1)][a.sub.b] LS[V.sub.bi+] [e.sub.i] (1)

where [AG.sub.i] is the ith student's analytic (A) or global (G) learning style, i goes from 1 to 61; [a.sub.0] is the intercept; [LSV.sub.bi] is the bth discriminating learning style element for ith student; and [e.sub.ij] is the stochastic error term. The standardized version of the regression model also was estimated to determine the relative importance of the alternative discriminating learning style preferences. Both regression results are reported in Table 2.

The high adjusted R-squared indicates that the students' preferences for the five discriminating learning style elements of persistence, noise, light, design and intake, listed in order of importance, explain fifty eight percent of the variation of students' analytic and global cognitive processing style. The statistically significant F statistic indicates that overall the regression model provides a good fit of the data which measures the student's relative preference for analytic or global learning approaches.

Three of the five discriminating learning style preferences, persistence, noise and light, are statistically significant. The magnitude of the negative coefficient for persistence indicates that this cohort of students has a strong preference to work on several tasks simultaneously rather than start and finish one task at a time. The magnitude of the positive coefficient for the variable noise indicates that there is a strong preference for background noise while learning, and the magnitude of the negative coefficient for the variable light indicates a moderate preference for low light while learning for this group of students. According to the DDLSM, these overall results indicate that this group of students is characterized by the global learning style. Although the coefficients for the design and intake elements were not significant, all of the coefficients had hypothesized signs consistent with the global thought processing style.

LIMITATIONS OF THE STUDY

The construct validity of a research task is only one part of the broader set of concerns that comprise internal validity. Therefore, only part of the potential criticisms of the DDLSM raised by such researchers as Kavale and Froness (1990) and Coffield et al. (2004) are addressed in this paper. A second limitation of the present research is that it analyzes the learning style preferences of only 61 student subjects at a single university. A larger sample drawn from a greater number of colleges and universities would allow a greater confidence in the generalizability of these findings. Also, the sample utilized in this study is based on the responses of prospective economics and accounting majors. It would be instructive to examine whether the conclusions that flow from an analysis of that sample apply to students from a wider spectrum of disciplines.

RECOMMENDATIONS FOR FUTURE RESEARCH

The DDLSM hypothesizes that the most effective way to teach is to utilize pedagogical methods that match students' learning styles. However, there is very little information as to how students across a broad spectrum of disciples learn. That is, how are the learning style preferences of students in many disciplines different from one another? It would be very useful to find a much larger dataset, including students from a much broader variety of disciplines, and analyze their learning styles in order to identify possible differences and therefore tailor teaching approaches accordingly.

CONCLUDING REMARKS

Both the Productivity Environmental Preference Survey (PEPS) and the Building Excellence Survey (BE) instruments are designed to identify college students' learning styles in accordance with the Dunn and Dunn learning style model Dunn (2000). The model hypothesizes that a student's thought processing style can be identified as either analytic or global, an element associated with the psychological learning style category. The model also hypothesizes that analytic and global learners can be distinguished by their environmental preferences for sound, light and seating, their emotional preference for persistence and their physiological preference for intake. This hypothesis was particularly relevant to the manner in which a PEPS survey respondent was identified as being an analytic or global learner.

Analytic learners have strong preferences for learning persistently in a step-by-step, sequential manner, in a formal or traditional seating arrangement, in a quiet and brightly lighted environment and with no snacks or drinks. Alternatively, global learners have strong preferences for learning through broad concepts with supporting graphic illustrations, take intermittent breaks and require snacks or drinks while learning, prefer a casual or informal seating arrangement and prefer low light and background noise. The Dunn and Dunn model (Dunn, 2000) further hypothesizes that to be identified as an analytic or global learner, the student must have a strong preference for at least three of the five discriminating learning style elements, and those three elements must have the appropriate, hypothesized signs.

Unlike the BE, the PEPS survey instrument does not utilize an independent set of survey questions to identify the analytic/global learning style element. Rather, the PEPS indirectly infers a respondent's analytic/global learning style preferences from the respondents' five discriminating learning style elements of noise, light, design, persistence and intake. Since the PEPS identifies students' preferences for the discriminating learning style elements but does not directly measure the analytic or global element, whether a student is an analytic or global learner therefore can only be determined from his or her preferences for the five discriminating learning style elements.

Under such circumstances, the ordinary least squares regression method may be utilized to test the construct validity of the PEPS instrument using students' learning style preferences identified by the BE instrument. In this study, the analytic/global learning style element was regressed against the five discriminating elements plus an intercept term. The dependent and explanatory learning style variables were identified by the BE instrument. The coefficients of three of the five discriminating elements, persistence, noise and light, were statistically significant and of sufficient magnitude to indicate a strong preference. All five elements had signs consistent with the global learning style. Thus, the criteria hypothesized by the DDLSM to identify a student's analytic/global processing style were met.

These results indicate that the PEPS instrument can be used to distinguish a college student's analytic or global thought processing style from his or her preferences for noise, light, seating, persistence and intake. In addition, the results provide indirect evidence which supports the construct validity of the BE instrument and the internal validity of the DDLSM. Such evidence can be seen to at least partially address the criticisms relating to a lack of independent testing of the internal validity of the DDLSM advanced by Kavale and Forness 1990) and Coffield et al. (2004).

REFERENCES

Coffield, F., Moseley, D., Hall, E., & K. Ecclestone, K. (2004). Learning Styles and Pedagogy in Post-16 learning: A Systematic and Critical Review. London: The Learning and Skills Research Center, www.lsda.org.uk/files/ PDF/1543.pdf.

Curry, L. (1987). Integrating concepts of cognitive learning styles: A review with attention to psychometric standards. Ottawa: Canadian College of Health Services Executives.

Desmedt, E. ,& Valcke, M. (2004). Mapping the learning styles "jungle": an overview of the literature based on citation analysis. Educational Psychology, 24(4), 445-464.

Dunn, R. (2000). Capitalizing on college students' learning styles: theory, practice, and research. In R. Dunn, R. & S. Griggs (Eds), Practical approaches to using learning styles in higher education (pp.3-18). Westport, CT: Bergin & Garvey.

Dunn, R., Dunn, K., & Price, G. E. (2006). Productivity environmental preference survey. Lawrence KS: Price Systems, Inc.

Ivie, S. (2009). Learning styles: Humpty Dumpty revisited. McGill Journal of Education. 44(2) , 177-192.

Kavale, K., & Forness, S. (1987). Substance over style: assessing the efficacy of modality testing and teaching. Exceptional Children, 54(3), 228-239.

Kavale, K., & Forness SR (1990). Substance over style: a rejoinder to Dunn's animadversions. Exceptional Children, 56(4), 357-361.

Rohrer, D., & Pashler, H. (2012). Learning styles: where's the evidence? Medical Education, 46, 630-635.

Rundle, S., & Dunn, R. (1996-2009a). Building Excellence[R] (BE) Survey. www.learningstyles.net.

Rundle, S., & Dunn, R. (1996-2009b). Building Excellence (BE) Survey 2000 Research Manual. http://www.asb.dk/fileexplorer/fetchfile.aspx?file=7783.

Wilson, R. (2012). The emperor's new clothes: learning styles and multiple intelligences. Colleagues, 8(2), http://scholarworks.gvsu.edu/colleagues/vol8/iss2/7.

Fred Englander

Fairleigh Dickinson University

Ralph A. Terregrossa

St. John's University

Zhaobo Wang

Fairleigh Dickinson University

Fred Englander is a Professor of Economics at Fairleigh Dickinson University in Madison, New Jersey. He received his Ph.D. in economics from Rutgers University. Dr. Englander has published articles in the Southern Economics Journal, Business Ethics Quarterly, Science and Engineering Ethics and the Journal of Education for Business.

Ralph A. Terregrossa is an Associate Professor of Economics at St. John's University in New York City. He holds a Ph.D. in economics from Binghamton University. Dr. Terregrossa has published articles in The Quarterly Review of Economics and Finance, International Advances in Economic Research and Educational Review.

Zhaobo Wang is an Associate Professor of Production and Operations Management at Fairleigh Dickinson University, Madison, New Jersey. Dr. Wang received a Ph.D. in operations research from Rutgers University. He has published articles in the Journal of Educational and Behavioral Statistics and the Journal for Economic Educators.

Table 1
The Discriminating (Analytic/Global) Elements and the
Hypothesized Sign for Each

Discriminating Element   Analytic Learners   Global Learners

Noise                        Negative           Positive
Light                        Positive           Negative
Design                       Positive           Negative
Persistence                  Positive           Negative
Intake                       Negative           Positive

Table 2
Least Squares and Standardized Regression Results
for the Analytic and Global Cognition Styles

                                                        Standardized
Variable      Coefficient   t Statistic   P Statistic   Coefficient

Intercept     -20.6288      -5.40         .0001         0
Persistence   -0.7654       -5.83         .0001         -052612
Noise         0.34292       3.70          .0005         0.32527
Light         -.26977       -2.66         0.01          -0.24018
Design        -0.1553       -1.16         0.2499        -0.10172
Intake        0.03067       .33           0.7457        0.02804

Notes: Adjusted R-squared = 0.58; F Statistic = 17.21 and
Pr > F = .0001.