Differential effects of student characteristics on performance: online vis-a-vis offline accounting courses.

Huh, Sungkyoo ; Jin, Jongdae ; Lee, Kyung Joo 等

INTRODUCTION

A considerable body of research on distance learning suggests that there is no significant difference in achievement levels between online learners and offline learners (E.G., The Institute for Higher Education Policy (1999), Chamberlin (2001) and Yin et. al. (2002)). However, online learners and offline learners may perform differently due to differences in student perception, available learning tools, use of the learning tools, and other technical issues. (See Barker (2002), Beard et. al. (2002), Dunbar (2004), Kendall (2001), Lightner et. al. (2001), Perreault et. al. (2002), Schulman et. al. (1999), Schwartzman et. al. (2002), and Woods (2002)) Furthermore, many previous studies suggest that student performances can be affected by student characteristics such as gender, age, educational experience, and motivation. (E.G., Sullivan (2001), Younger (1999)) Thus, the purpose of this study is to examine whether there exist any systematic differences in the effects of student characteristics on student performances as measured by test scores between online courses and offline courses.

The remainder of the paper is organized as follows: first, sample data descriptions are discussed the next section, which is followed by discussions on data analyses and their results. Concluding remarks are made in the final section.

SAMPLE DESCRIPTIONS

Sample data are collected from students who took undergraduate accounting courses offered through online as well as offline at California State University-San Bernardino during the three years from fall 2003 to spring 2005. Both online and offline classes were taught by the same instructor who used Blackboard as a web-based learning assistance tool. The same textbook was used and the same lecture notes for each chapter developed by the instructor were provided to students in both classes. Exams for on line and off line classes are developed by the instructor in such a way that exams for on line classes are equivalent to those for off line classes. All exams were proctored and graded by the same instructor.

Student performance data such as test scores and GPA are collected from the course instructor or the university database, while student demographic data such as gender, age, commuting distance and working hours are from survey questionnaires to the student sample. After deleting students with insufficient data, the final data of 91 students (54 online learners and 37 offline learners) are analyzed in this study.

The descriptive statistics for the characteristics variables are presented in Table 1. There are no significant differences in gender compositions, marital status, GPA, and performance (test scores) between on line learners and their matching off line learners. On the other hand, significant differences exist in age, commuting distance, and working hours between on line learners and off line learners. As expected, students taking online courses are older, commute longer distance and work more hours than those taking offline courses.

Table 2 shows the correlations among student characteristics. Online sample exhibits significant positive correlation between commuting distance and working hours. For offline sample, distance has significantly positive correlations with marital status as well as working hours, which is also positively correlated with marital status.

ANALYSIS AND RESULTS

Univariate Analyses: Mean Difference Comparisons

In order to conduct unvariate comparison analyses, all sample students are divided into two subgroups for each characteristics variable: i.e., Low GPA and High GPA. For each sample (online or offline), students with higher GPA than the sample median GPA belong to High GPA group, while students with lower GPA than the sample median GPA to Low GPA group. Same procedure was applied to other variables. The potential effects of student characteristics on performances were then examined by comparing test scores between these two groups.

Table 3 presents the results of comparing performances between two subgroups and corresponding Wilcoxon z-statistics for both online sample and offline sample. GPA is the only factor affecting performance for both online and offline students. For example, online (offline) students with high GPA have average test score of 75.9 (80.5), while those with low GPA show 67.5 (68.6). These differences (about 10 points) are statistically significant (a<0.05). Other than GPA, gender has significant impact on performance for offline sample. Specifically, male students are doing better than female students (80.5 versus 70.9).

Regression Analyses

Results from univariate analyses in preceding section show that GPA is a factor affecting performances for both online learners and offline learners, while gender is a factor for offline learners. As an attempt to investigate if these results hold after controlling for other student characteristics, we estimate the following regression model:

Score= [[alpha].sub.0] + [[alpha].sub.1] GPA + [[alpha].sub.2] Age + [[alpha].sub.3] Distance + [[alpha].sub.4] Hour + [[alpha].sub.5] Gender + [[alpha].sub.6] Marital + [epsilon] (1)

Where,

Scores = final test scores, including midterm test results,

GPA = grade point average,

Age= age of the student,

Distance = the distance from a student's residence to the campus (miles),

Hour = the number of working hours per week,

Gender= 1 if female; 0 if male.

Marital= marital status, 1 if married; 0 if single.

[[alpha].sub.i] = the partial regression coefficients of variable 'i',

[member of] = the error term.

The significantly positive correlation between Distance and Hour (see Table 2) may cause the multicolinearity problem. To avoid this potential problem, we estimate the regression model (1) without Hours (Model 2) or Distance (Model 3) along with the full model (Model 1). Results from estimating the regression model (1) are presented in Table 4. The regression coefficients of GPA are all positive and statistically significant (a<0.01) across different models for both online and offline samples. This result indicates that GPA is a factor affecting student performances. The coefficient estimates for Gender are consistently negative, indicating that male students perform better than female students. However, this gender difference is significant (a<0.01) only for offline sample. Overall, these results suggest that GPA is a factor affecting performances for both online learners and offline learners, while gender is a factor for offline learners even after controlling for other characteristics variables.

Given the significant effects of GPA and Gender on performances, we employed the following regression model to examine whether there exist systematic differences in these effects between online and offline students:

Scores = [[alpha].sub.0] + [[alpha].sub.1] GPA + [[alpha].sub.2] Gender + [[alpha].sub.3] Maturity + [[alpha].sub.4] GPA * On-Off + [[alpha].sub.5] Gender * On-Off + [[alpha].sub.6] Maturity * On-Off + [epsilon] (2)

Where,

Scores = final test scores, including midterm test results,

GPA = grade point average,

Gender= 1 if female; 0 if male.

Maturity= metric to measure the level of student's maturity,

On-Off=1 if the student is offline learner; 0 if online learner.

[[alpha].sub.i] = the partial regression coefficients of variable 'i',

[member of] = the error term.

We construct and include a new variable Maturity in lieu of the variables such as Age, Distance, Hour, and Marital. The reasons for this are as follows: First, these variables may represent the common characteristics, i.e., the maturity. Second, although each of theses variables does not affect the performance, they may collectively have significant effect. Finally, we can estimate parsimonious model by including less variables while maintaining essentially the same range of student characteristics variables. To obtain this metric, sample was first classified into two subgroups based on Age, Distance (or Hour) and Marital variable, respectively. Next, each student was assigned 1 if older (Age), farther/longer (Distance/Hour), or married (Marital), and 0 if otherwise. Finally, assigned values of three variables were added up to get the measure of Maturity. Hence, the value of Maturity has the range between 0(least mature) and 3(most mature).

We estimate the regression model (2) separately for online sample and offline sample, as well as for the total sample. Table 5 shows the estimation results. First, regression coefficients of GPA are positive and statistically significant (a<0.01) for both online and offline samples. However, the coefficient for offline sample (19.414) is larger than that for online sample (14.747). This difference can be seen in the positive value of the regression coefficient (GPA*On-Off). More importantly, the coefficient is statistically significant (a<0.01), indicating that GPA has more effect on performance for offline students than for online students. Second, regression coefficients of Gender are negative but statistically significant (a<0.01) only for offline sample. Furthermore, the coefficient for offline sample (-10.177) is smaller than that for online sample (0.314), and this difference is statistically significant (a<0.01) as shown in the coefficient (Gender*On-Off). This result indicates that male students perform better in offline courses than in online courses. Third, regression coefficients of Maturity are positive but statistically insignificant for both online and offline samples. While offline sample has larger coefficient (5.037) than online counterpart (1.455), this difference is not significant.

Overall, these results suggest that GPA and gender have significant effects on student performances, and theses effects are larger for offline students than for online students.

CONCLUSIONS

The purposes of this study are twofold. The first is to examine the potential effects of student characteristics on performances as measured by test scores. Second purpose is to investigate if there is any systematic difference in those effects between online courses and offline courses. Academic and demographic data of 91 students who took undergraduate accounting courses offered through online as well as offline at California State University-San Bernardino during a three-year period extending from fall 2003 to spring 2005 are examined using univariate analyses as well as regression models.

The empirical results can be summarized as follows: First, There is no significant difference in student performances (test scores) between online learners and offline learners. Second, while no significant differences exist in gender compositions, marital status, and GPA, students taking online courses are older, commute longer distance and work more hours than those taking offline courses. Third, GPA is a factor affecting performances for both online learners and offline learners, while gender is a factor for offline learners even after controlling for other characteristics variables. Finally, the effects of GPA and gender variables on performances are larger for offline students than for online students. These results are robust across different testing methodologies.

REFERENCES

Barker, Phillip (2002). On being online tutor. Innovations in Education and Teaching International, Vol 39 (1), pp. 3- 13.

Beard, L. A. and C. Harper (2002). Student perception of online versus campus instruction. Education, Vol. 122 (4), pp. 658-664.

Chamberlin, W. S. (2001). Face to face vs. cyberspace: finding the middle ground. Syllabus, Vol. 15, 11.

Cuellar, N. (2002). The Transition from Classroom to Online Teaching. Nursing Forum. July/Sep. pp. 5-13.

Dunbar, Amy E. (2004). Genesis of an Online Course. Issues in Accounting Education. Vol 19, No.3, pp 321-343.

The Institute for Higher Education Policy. (1999). What's the difference?: A review of contemporary research on the effectiveness of distance learning in higher education.

Kendall, Margaret (2001). Teaching online to campus-based students: The experience of using WebCT for the community information module at Manchester Metropolitan University. Education for Information, Vol 19, pp. 325-346.

Lightner, S. and C. O. Houston (2001). Offering a globally-linked international accounting course in real time: a sharing of experiences and lessons learned. Journal of Accounting Education, Vol 19 (4), pp. 247-263.

Orde, Barbara J., J. Andrews, A. Awad, S. Fitzpatrick, C. Klay, C. Liu, D. Maloney, M. Meny, A. Patrick, S. Welsh, and J. Whitney (2001). Online course development: summative reflections. International Journal of Instructional Media, Vol. 2 (4), pp. 397-403.

Perreault, H., Waldman L., and Zhao, M. (2002). Overcoming Barriers to Successful Delivery of Distance-Learning Courses. Journal of Education for Business. July/August. pp. 313-318.

Schulman, A. and Sims, R. (1999). Learning in an Online Format versus an In-class Format: An Experimental Study. Journal Online.

Schwartzman, R. and H. Tuttle (2002). What can online course components teach about improving instruction and learning?. Journal of Instructional Psychology, Vol. 29, No. , pp. 29-38.

Sullivan, Patrick. (2001). Gender differences and the online classroom: male and /female college students evaluate their experiences. Community College Journal of Research and Practice. Vol. 25, pp. 805-818.

Woods Jr., Robert H (2002). How much communication is enough in online courses?--exploring the relationship between frequency of instructor-initiated personal email and learner's perceptions of and participation in online learning. International Journal of Instructional Media, Vol. 29(4), pp. 377-394.

Yin, L. Roger, L. E. Urven, R. M. Schramm, and S. J. Friedman (2002). Assessing the consequences of on-line learning: issues, problems, and opportunities at the University of Wisconsin-Whitewater. Assessment Update, Vol 14, No. 2, pp. 4-13.

Younger, Michael, M. Warrington, and J. Williams (1999). The Gender Gap and Classroom Interactions: reality and rhetoric?. British Journal of Sociology of Education; Vol. 20. No., pp 325-341.

Sungkyoo Huh, California State University-San Bernardino

Jongdae Jin, California State University, San Bernardino

Kyung Joo Lee, University of Maryland-Eastern Shore

Sehwan Yoo, University of Advancing Technology

Table 1: Descriptive Statistics for Student Characteristics
Variables

 Online sample (N=54)

 Std.
Variables Mean Dev Median

Score 70.009 12.944 72.250
GPA 3.106 0.539 3.249
Age 30.204 8.381 27.000
Distance 45.685 29.740 38.500
Hour 32.963 14.193 40.000

Gender Female: 40 (74.07%)
Marital Married: 21 (38.89%)

 Offline sample (N=37)

 Std
Variables Mean Dev Median

Score 74.784 12.937 77.500
GPA 3.195 0.425 3.244
Age 26.622 6.958 25.000
Distance 17.784 13.105 20.000
Hour 22.703 15.028 25.000

Gender Female: 22 (59.46%)
Marital Married: 13 (35.14%)

 Wilcoxon z-statistics

Variables (p-value)

Score 1.592 (0.111)
GPA 0.388 (0.698)
Age 2.203 (0.028) **
Distance 4.624 (0.001) ***
Hour 3.366 (0.001) ***

Gender 2.160 (0.142)
Marital 0.132 (0.716)

1) Score = final test scores, including midterm test results.

GPA = the student's previous grade point average.

Age = age of the student.

Distance = distance from student's residence to the
campus (miles).

Hour = working hours per week.

Gender = female (1) and male (0).

Marital = marital status; married (1) and single (0).

2) For the variables Gender and Marital, tests on
differences in frequencies between online and offline
samples are based on chi-square statistics.

***: Significant at a<0.01; **: significant at
a<0.05; *: significant at a<0.10

Table 2: Correlations Among Student Characteristics Variables

Panel A: Online Sample

 GPA Age Distance

GPA 1.000 0.098 0.200
Age 1.000 -0.028
Distance 1.000
Hour
Gender
Marital

 Hour Gender Marital

GPA 0.032 -0.162 -0.174
Age -0.082 0.045 0.053
Distance 0.264 * -0.062 0.018
Hour 1.000 0.035 -0.060
Gender 1.000 0.039
Marital 1.000

Panel B: Offline Sample

 GPA Age Distance

GPA 1.000 0.133 -0.231
Age 1.000 -0.059
Distance 1.000
Hour
Gender
Marital

 Hour Gender Marital

GPA -0.107 0.026 -0.218
Age 0.090 -0.270 -0.166
Distance 0.493 *** 0.033 0.459 ***
Hour 1.000 0.095 0.343 **
Gender 1.000 0.146
Marital 1.000

1) GPA = the student's previous grade point average.

Age = age of the student.

Distance = distance from student's residence to
the campus (miles).

Hour = working hours per week.

Gender = 1 if female; 0 if male.

Marital = 1 if married; 0 if single.

2) Pearson correlations are reported.

***: Significant at a<0.01; **: significant at
a<0.05; *: significant at a<0.10

Table 3: Effects of Student Characteristics on Performance

 Univariate Analyses

 Online sample (n=54)
Variables/
Group Mean Wilcoxon z-stat
 SCORE (p-value)

GPA:
 High 75.922 2.103 (0.035) **
 Low 67.520
Age:
 Old 69.250 0.087 (0.931)
 Young 70.769
Distance:
 Far 69.422 0.564 (0.573)
 Near 70.864
Hour:
 Long 70.905 0.643 (0.520)
 Short 68.059
Gender:
 Female 69.238 0.849 (0.396)
 Male 72.214
Marital:
 Married 69.821 0.036 (0.972)
 Single 70.129

 Univariate Analyses

 Offline sample (n=37)
Variables/
Group Mean Wilcoxon z-stat
 SCORE (p-value)

GPA:
 High 80.579 2.766 (0.006) ***
 Low 68.667
Age:
 Old 76.950 0.991 (0.321)
 Young 72.235
Distance:
 Far 74.434 0.000 (1.000)
 Near 75.153
Hour:
 Long 74.039 0.152 (0.879)
 Short 75.569
Gender:
 Female 70.898 1.980 (0.048) **
 Male 80.483
Marital:
 Married 74.019 0.493 (0.622)
 Single 75.198

1) For each variable, sample was classified into two groups based on
median value of the variable. For example, High if GPA>=median;
Low if GPA<median.

** Significant at a<0.01; ** significant at a<0.05;
* significant at a<0.10

Table 4: Effects of Student Characteristics on Performance

Regression Analyses

Score=[[alpha].sub.0] + [[alpha].sub.1] GPA + [[alpha].sub.2]
Age + [[alpha].sub.3] Distance + [[alpha].sub.4] Hour +
[[alpha].sub.5] Gender + [[alpha].sub.6] Marital + [epsilon]

 Online sample (n=54)

 Model 1 Model 2 Model 3

Intercept 20.058 23.348 20.242
 (1.80) * (2.21) ** (1.82) *
GPA 15.517 15.467 14.896
 (5.40) *** (5.39) *** (5.29) ***
Age 0.002 -0.011 0.008
 (0.01) (0.06) (0.04)
Distance -0.056 -0.043
 (1.07) (0.85)
Hours 0.104 0.073
 (0.96) (0.70)
Gender -0.394 -0.214 -0.242
 (0.12) (0.06) (0.07)
Marital 2.904 2.706 2.662
 (0.96) (0.89) (0.88)
Adj. R2(%) 32.24 32.33 32.02

 Offline sample (n=37)

 Model 1 Model 2 Model 3

Intercept 11.461 11.418 12.212
 (0.79) (0.80) (0.88)
GPA 18.146 18.141 18.022
 (4.61) *** (4.65) *** (4.70) ***
Age 0.387 0.355 0.384
 (1.57) (1.47) (1.58)
Distance 0.034 -0.008
 (0.23) (0.06)
Hours -0.087 -0.076
 (0.69) (0.67)
Gender -9.024 -9.295 -9.075
 (2.66) ** (2.78) *** (2.72) **
Marital 5.082 4.646 5.365
 (1.31) (1.23) (1.49)
Adj. R2(%) 44.92 45.85 46.6

1) Estimates and t-statistics (parenthesis) from the
regression are shown.

*** Significant at a<0.01; **: significant at a<0.05;
*: significant at a<0.10

Table 5: Differential Effects of Student Characteristics on Performance

Online vis-a-vis Offline

Scores = [[alpha].sub.0] + [[alpha].sub.1] GPA + [[alpha].sub.2]
Gender + [[alpha].sub.3] Maturity + [[alpha].sub.4] GPA*On-Off +
[[alpha].sub.5] Gender*On-Off + [[alpha].sub.6] Maturity*On-Off +
[member of]

 Online Offline

Intercept 23.731 (2.48) ** 16.365 (1.27)
GPA 14.747 (5.36) *** 19.414 (5.08) ***
Gender -0.314 (0.09) -10.177 (3.23) ***
Maturity 1.455 (0.50) 5.037 (1.57)
GPA*On-Off
Gender*On-Off
Maturity*On-Off
Adj. R2 (%) 33.42 4721

 Total

Intercept 21.482 (2.82) ***
GPA 15.361 (6.88) ***
Gender -0.025 (0.01)
Maturity 1.618 (0.58)
GPA*On-Off 2.571 (2.10) **
Gender*On-Off -10.286 (2.26) **
Maturity*On-Off 2.954 (0.70)
Adj. R2 (%) 40.79

1) Maturity= metric to measure the level of student's maturity. To
obtain this metric, sample was first classified into two groups
based on Age, Distance (or Hour) and Marital variable,
respectively. Next, Each student was assigned 1 if older
(Age),farther (Distance), or married (Marital); 0 if otherwise.
Finally, assigned values of three variables were added up to get
the measure of Maturity. Hence, the value of Maturity has the range
between 0(least mature) and 3(most mature). On-Off=1 if the student
is offline learner; 0 if online learner.

2) Estimates and t-statistics (parenthesis) from the regression
are shown.

***: Significant at a<0.01; ** significant at a<0.05;
* significant at a<0.10