Differential effects of student characteristics on performance: online vis-a-vis offline accounting courses.
Huh, Sungkyoo ; Jin, John Jongdae ; Lee, Kyung Joo 等
INTRODUCTION
A considerable body of research on distance learning suggests that
there is no significant difference in achievement levels between online
learners and offline learners (E.G., The Institute for Higher Education
Policy (1999), Chamberlin (2001) and Yin et. al. (2002)). However,
online learners and offline learners may perform differently due to
differences in student perception, available learning tools, use of the
learning tools, and other technical issues. (See Barker (2002), Beard
et. al. (2002), Dunbar (2004), Kendall (2001), Lightner et. al. (2001),
Perreault et. al. (2002), Schulman et. al. (1999), Schwartzman et. al.
(2002), and Woods (2002)) Furthermore, many previous studies suggest
that student performances can be affected by student characteristics
such as gender, age, educational experience, and motivation. (E.G.,
Sullivan (2001), Younger (1999)) Thus, the purpose of this study is to
examine whether there exist any systematic differences in the effects of
student characteristics on student performances as measured by test
scores between online courses and offline courses.
The remainder of the paper is organized as follows: first, sample
data descriptions are discussed the next section, which is followed by
discussions on data analyses and their results. Concluding remarks are
made in the final section.
SAMPLE DESCRIPTIONS
Sample data are collected from students who took undergraduate
accounting courses offered through online as well as offline at
California State University-San Bernardino during the three years from
fall 2003 to spring 2005. Both online and offline classes were taught by
the same instructor who used Blackboard as a web-based learning
assistance tool. The same textbook was used and the same lecture notes
for each chapter developed by the instructor were provided to students
in both classes. Exams for on line and off line classes are developed by
the instructor in such a way that exams for on line classes are
equivalent to those for off line classes. All exams were proctored and
graded by the same instructor.
Student performance data such as test scores and GPA are collected
from the course instructor or the university database, while student
demographic data such as gender, age, commuting distance and working
hours are from survey questionnaires to the student sample. After
deleting students with insufficient data, the final data of 91 students
(54 online learners and 37 offline learners) are analyzed in this study.
The descriptive statistics for the characteristics variables are
presented in Table 1. There are no significant differences in gender
compositions, marital status, GPA, and performance (test scores) between
on line learners and their matching off line learners. On the other
hand, significant differences exist in age, commuting distance, and
working hours between on line learners and off line learners. As
expected, students taking online courses are older, commute longer
distance and work more hours than those taking offline courses.
Table 2 shows the correlations among student characteristics.
Online sample exhibits significant positive correlation between
commuting distance and working hours. For offline sample, distance has
significantly positive correlations with marital status as well as
working hours, which is also positively correlated with marital status.
ANALYSIS AND RESULTS
Univariate Analyses: Mean Difference Comparisons
In order to conduct unvariate comparison analyses, all sample
students are divided into two subgroups for each characteristics
variable: i.e., Low GPA and High GPA. For each sample (online or
offline), students with higher GPA than the sample median GPA belong to
High GPA group, while students with lower GPA than the sample median GPA
to Low GPA group. Same procedure was applied to other variables. The
potential effects of student characteristics on performances were then
examined by comparing test scores between these two groups.
Table 3 presents the results of comparing performances between two
subgroups and corresponding Wilcoxon z-statistics for both online sample
and offline sample. GPA is the only factor affecting performance for
both online and offline students. For example, online (offline) students
with high GPA have average test score of 75.9 (80.5), while those with
low GPA show 67.5 (68.6). These differences (about 10 points) are
statistically significant (a<0.05). Other than GPA, gender has
significant impact on performance for offline sample. Specifically, male
students are doing better than female students (80.5 versus 70.9).
Regression Analyses
Results from univariate analyses in preceding section show that GPA
is a factor affecting performances for both online learners and offline
learners, while gender is a factor for offline learners. As an attempt
to investigate if these results hold after controlling for other student
characteristics, we estimate the following regression model:
Score = [[alpha].sub.0] + [[alpha].sub.1] GPA + [[alpha].sub.2] Age
+ [[alpha].sub.3] Distance + [[alpha].sub.4] Hour + [[alpha].sub.5]
Gender + [[alpha].sub.6] Marital + [epsilon] (1)
Where,
Scores = final test scores, including midterm test results,
GPA = grade point average,
Age= age of the student,
Distance = the distance from a student's residence to the
campus (miles),
Hour = the number of working hours per week,
Gender= 1 if female; 0 if male.
Marital= marital status, 1 if married; 0 if single.
[[alpha].sub.i] = the partial regression coefficients of variable
'i',
[epsilon] = the error term.
The significantly positive correlation between Distance and Hour
(see Table 2) may cause the multicolinearity problem. To avoid this
potential problem, we estimate the regression model (1) without Hours
(Model 2) or Distance (Model 3) along with the full model (Model 1).
Results from estimating the regression model (1) are presented in Table
4. The regression coefficients of GPA are all positive and statistically
significant (a<0.01) across different models for both online and
offline samples. This result indicates that GPA is a factor affecting
student performances. The coefficient estimates for Gender are
consistently negative, indicating that male students perform better than
female students. However, this gender difference is significant
(a<0.01) only for offline sample. Overall, these results suggest that
GPA is a factor affecting performances for both online learners and
offline learners, while gender is a factor for offline learners even
after controlling for other characteristics variables.
Given the significant effects of GPA and Gender on performances, we
employed the following Regression model to examine whether there exist
systematic differences in these effects between online and offline
students:
Scores = [[alpha].sub.0] + [[alpha].sub.1] GPA + [[alpha].sub.2]
Gender + [[alpha].sub.3] Maturity + [[alpha].sub.4] GPA*On-Off +
[[alpha].sub.5] Gender*On-Off + [[alpha].sub.6] Maturity*On-Off +
[epsilon] (2)
Where,
Scores = final test scores, including midterm test results,
GPA = grade point average,
Gender = 1 if female, 0 if male.
Maturity = metric to measure the level of student's maturity,
On- Off = 1 if the student is offline learner; 0 if online learner.
[[alpha].sub.1] = the partial regression coefficients of variable
'i',
[epsilon] = the error term.
We construct and include a new variable Maturity in lieu of the
variables such as Age, Distance, Hour, and Marital. The reasons for this
are as follows: First, these variables may represent the common
characteristics, i.e., the maturity. Second, although each of theses
variables does not affect the performance, they may collectively have
significant effect. Finally, we can estimate parsimonious model by
including less variables while maintaining essentially the same range of
student characteristics variables. To obtain this metric, sample was
first classified into two subgroups based on Age, Distance (or Hour) and
Marital variable, respectively. Next, each student was assigned 1 if
older (Age), farther/longer (Distance/Hour), or married (Marital), and 0
if otherwise. Finally, assigned values of three variables were added up
to get the measure of Maturity. Hence, the value of Maturity has the
range between 0(least mature) and 3(most mature).
We estimate the regression model (2) separately for online sample
and offline sample, as well as for the total sample. Table 5 shows the
estimation results. First, regression coefficients of GPA are positive
and statistically significant (a<0.01) for both online and offline
samples. However, the coefficient for offline sample (19.414) is larger
than that for online sample (14.747). This difference can be seen in the
positive value of the regression coefficient (GPA*On-Off). More
importantly, the coefficient is statistically significant (a<0.01),
indicating that GPA has more effect on performance for offline students
than for online students. Second, regression coefficients of Gender are
negative but statistically significant (a<0.01) only for offline
sample. Furthermore, the coefficient for offline sample (-10.177) is
smaller than that for online sample (-0.314), and this difference is
statistically significant (a<0.01) as shown in the coefficient
(Gender*On-Off). This result indicates that male students perform better
in offline courses than in online courses. Third, regression
coefficients of Maturity are positive but statistically insignificant
for both online and offline samples. While offline sample has larger
coefficient (5.037) than online counterpart (1.455), this difference is
not significant.
Overall, these results suggest that GPA and gender have significant
effects on student performances, and theses effects are larger for
offline students than for online students.
CONCLUSIONS
The purposes of this study are twofold. The first is to examine the
potential effects of student characteristics on performances as measured
by test scores. Second purpose is to investigate if there is any
systematic difference in those effects between online courses and
offline courses. Academic and demographic data of 91 students who took
undergraduate accounting courses offered through online as well as
offline at California State University-San Bernardino during a
three-year period extending from fall 2003 to spring 2005 are examined
using univariate analyses as well as regression models.
The empirical results can be summarized as follows: First, There is
no significant difference in student performances (test scores) between
online learners and offline learners. Second, while no significant
differences exist in gender compositions, marital status, and GPA,
students taking online courses are older, commute longer distance and
work more hours than those taking offline courses. Third, GPA is a
factor affecting performances for both online learners and offline
learners, while gender is a factor for offline learners even after
controlling for other characteristics variables. Finally, the effects of
GPA and gender variables on performances are larger for offline students
than for online students. These results are robust across different
testing methodologies.
REFERENCES
Barker, P. (2002). On being online tutor. Innovations in Education
and Teaching International, 9 (1), pp. 3-13.
Beard, L. & C. Harper (2002). Student perception of online
versus campus instruction. Education, 122 (4), pp. 658-664.
Chamberlin, W. S. (2001). Face to face vs. cyberspace: finding the
middle ground. Syllabus, Vol. 15, 11.
Cuellar, N. (2002). The Transition from Classroom to Online
Teaching. Nursing Forum. July/Sep. pp. 5-13.
Dunbar, Amy E. (2004). Genesis of an Online Course. Issues in
Accounting Education. Vol 19, No.3, pp 321-343.
The Institute for Higher Education Policy. (1999). What's the
difference?: A review of contemporary research on the effectiveness of
distance learning in higher education.
Kendall, Margaret (2001). Teaching online to campus-based students:
The experience of using WebCT for the community information module at
Manchester Metropolitan University. Education for Information, Vol 19,
pp. 325-346.
Lightner, S. and C. O. Houston (2001). Offering a globally-linked
international accounting course in real time: a sharing of experiences
and lessons learned. Journal of Accounting Education, Vol 19 (4), pp.
247-263.
Orde, Barbara J., J. Andrews, A. Awad, S. Fitzpatrick, C. Klay, C.
Liu, D. Maloney, M. Meny, A. Patrick, S. Welsh, and J. Whitney (2001).
Online course development: Summative reflections. International Journal
of Instructional Media, Vol. 2 (4), pp. 397-403.
Perreault, H., Waldman L., and Zhao, M. (2002). Overcoming Barriers
to Successful Delivery of Distance-Learning Courses. Journal of
Education for Business. July/August. pp. 313-318.
Schulman, A. & Sims, R. (1999). Learning in an Online Format
versus an In-class format: An experimental study. Journal Online.
Schwartzman, R. and H. Tuttle (2002). What can online course
components teach about improving instruction and learning?.
Journal of Instructional Psychology, Vol. 29, No., pp.29-38.
Sullivan, Patrick. (2001). Gender differences and the online
classroom: male and /female college students evaluate their experiences.
Community College Journal of Research and Practice. Vol. 25, pp.
805-818.
Woods Jr., Robert H (2002). How much communication is enough in
online courses?--exploring the relationship between frequency of
instructor-initiated personal email and learner's perceptions of
and participation in online learning. International Journal of
Instructional Media, Vol. 29(4), pp.377-394.
Yin, L. Roger, L. E. Urven, R. M. Schramm, and S. J. Friedman
(2002). Assessing the consequences of on-line learning: issues,
problems, and opportunities at the University of Wisconsin-Whitewater.
Assessment Update, Vol 14, No. 2, pp. 4-13.
Younger, Michael, M. Warrington, and J. Williams (1999). The Gender
Gap and Classroom Interactions: reality and rhetoric?.
British Journal of Sociology of Education; Vol. 20. No., pp
325-341.
Sungkyoo Huh, California State University-San Bernardino
John Jongdae Jin, California State University- San Bernardino
Kyung Joo Lee, University of Maryland-Eastern Shore
Sehwan Yoo, University of Maryland Eastern Shore
Table 1: Descriptive Statistics for Student Characteristics
Variables
Variables Online sample (N=54)
Mean Std. Dev Median
Score 70.009 12.944 72.250
GPA 3.106 0.539 3.249
Age 30.204 8.381 27.000
Distance 45.685 29.740 38.500
Hour 32.963 14.193 40.000
Gender Female: 40 (74.07%)
Marital Married: 21 (38.89%)
Variables Offline sample (N=37)
Mean Std Dev Median
Score 74.784 12.937 77.500
GPA 3.195 0.425 3.244
Age 26.622 6.958 25.000
Distance 17.784 13.105 20.000
Hour 22.703 15.028 25.000
Gender Female: 22 (59.46%)
Marital Married: 13 (35.14%)
Wilcoxon z-statistics
Variables (p-value)
Score 1.592 (0.111)
GPA 0.388 (0.698)
Age 2.203 (0.028) **
Distance 4.624 (0.001) ***
Hour 3.366 (0.001) ***
Gender 2.160 (0.142)
Marital 0.132 (0.716)
1) Score = final test scores, including midterm test results.
GPA = the student's previous grade point average.
Age = age of the student.
Distance = distance from student's residence to the campus (miles).
Hour = working hours per week.
Gender = female (1) and male (0).
Marital = marital status; married (1) and single (0).
2) For the variables Gender and Marital, tests on differences in
frequencies between online and offline samples are based on
chi-square statistics.
***: Significant at a<0.01; **: significant at a<0.05; *:
significant at a<0.10;
Table 2: Correlations Among Student Characteristics Variables
Panel A: Online Sample
GPA Age Distance
GPA 1.000 0.098 0.200
Age 1.000 -0.028
Distance 1.000
Hour
Gender
Marital
Hour Gender Marital
GPA 0.032 -0.162 -0.174
Age -0.082 0.045 0.053
Distance 0.264 * -0.062 0.018
Hour 1.000 0.035 -0.060
Gender 1.000 0.039
Marital 1.000
Panel B: Offline Sample
GPA Age Distance
GPA 1.000 0.133 -0.231
Age 1.000 -0.059
Distance 1.000
Hour
Gender
Marital
Hour Gender Marital
GPA -0.107 0.026 -0.218
Age 0.090 -0.270 -0.166
Distance 0.493 *** 0.033 0.459 ***
Hour 1.000 0.095 0.343 **
Gender 1.000 0.146
Marital 1.000
1) GPA = the student's previous grade point average.
Age = age of the student.
Distance = distance from student's residence to the campus (miles).
Hour = working hours per week.
Gender = 1 if female; 0 if male.
Marital = 1 if married; 0 if single.
2) Pearson correlations are reported.
***: Significant at a<0.01; **: significant at a<0.05; *:
significant at a<0.10;
Table 3: Effects of Student Characteristics on Performance:
Univariate Analyses
Online sample (n = 54)
Variables/
Group Mean Wilcoxon z-stat
SCORE (p-value)
GPA:
High 75.922 2.103 (0.035) **
Low 67.520
Age:
Old 69.250 0.087 (0.931)
Young 70.769
Distance:
Far 69.422 0.564 (0.573)
Near 70.864
Hour:
Long 70.905 0.643 (0.520)
Short 68.059
Gender:
Female 69.238 0.849 (0.396)
Male 72.214
Marital:
Married 69.821 0.036 (0.972)
Single 70.129
Offline sample (n = 37)
Variables/
Group Mean Wilcoxon z-stat
SCORE (p-value)
GPA:
High 80.579 2.766 (0.006) ***
Low 68.667
Age:
Old 76.950 0.991 (0.321)
Young 72.235
Distance:
Far 74.434 0.000 (1.000)
Near 75.153
Hour:
Long 74.039 0.152 (0.879)
Short 75.569
Gender:
Female 70.898 1.980 (0.048) **
Male 80.483
Marital:
Married 74.019 0.493 (0.622)
Single 75.198
1) For each variable, sample was classified into two groups based on
median value of the variable. For example, High if GPA> = median;
Low if GPA<median.
Significant at a<0.01; **: significant at a<0.05; *: significant at
a<0.10;
Table 4: Effects of Student Characteristics on Performance:
Regression Analyses
Score = [[alpha].sub.0] + [[alpha].sub.1] GPA + [[alpha].sub.2] Age
+ [[alpha].sub.3] Distance + [[alpha].sub.4] Hour + [[alpha].sub.5]
Gender + [[alpha].sub.6] Marital + [epsilon]
Online sample (n=54)
Model 1 Model 2 Model 3
Intercept 20.058 23.348 20.242
(1.80) * (2.21) ** (1.82) *
GPA 15.517 15.467 14.896
(5.40) *** (5.39) *** (5.29) ***
Age 0.002 -0.011 0.008
(0.01) (0.06) (0.04)
Distance -0.056 -0.043
(1.07) (0.85)
Hours 0.104 0.073
(0.96) (0.70)
Gender -0.394 -0.214 -0.242
(0.12) (0.06) (0.07)
Marital 2.904 2.706 2.662
(0.96) (0.89) (0.88)
Adj. [R.sup.2] (%) 32.24 32.33 32.02
Offline sample (n=37)
Model 1 Model 2 Model 3
Intercept 11.461 11.418 12.212
(0.79) (0.80) (0.88)
GPA 18.146 18.141 18.022
(4.61) *** (4.65) *** (4.70) ***
Age 0.387 0.355 0.384
(1.57) (1.47) (1.58)
Distance 0.034 -0.008
(0.23) (0.06)
Hours -0.087 -0.076
(0.69) (0.67)
Gender -9.024 -9.295 -9.075
(2.66) ** (2.78) *** (2.72) **
Marital 5.082 4.646 5.365
(1.31) (1.23) (1.49)
Adj. [R.sup.2] (%) 44.92 45.85 46.60
1) Estimates and t-statistics (parenthesis) from the regression are
shown.
***: Significant at a<0.01; **: significant at a<0.05; *:
significant at a<0.10;
Table 5: Differential Effects of Student Characteristics on
Performance: Online vis-a-vis Offline
Score = [[alpha].sub.0] + [[alpha].sub.1] GPA + [[alpha].sub.2]
Gender + [[alpha].sub.3] Maturity + [[alpha].sub.4] GPA *On-Off +
[[alpha].sub.5] Gender *On-Off + [[alpha].sub.6] Maturity + [epsilon]
Online Offline
Intercept 23.731 (2.48) ** 16.365 (1.27)
GPA 14.747 (5.36) *** 19.414 (5.08) ***
Gender -0.314 (0.09) -10.177 (3.23) ***
Maturity 1.455 (0.50) 5.037 (1.57)
GPA*On-Off
Gender*On-Off
Maturity*On-Off
Adj. R2 (%) 33.42 47.21
Total
Intercept 21.482 (2.82) ***
GPA 15.361 (6.88) ***
Gender -0.025 (0.01)
Maturity 1.618 (0.58)
GPA*On-Off 2.571 (2.10) **
Gender*On-Off -10.286(2.26) **
Maturity*On-Off 2.954 (0.70)
Adj. R2 (%) 40.79
1) Maturity = metric to measure the level of student's maturity. To
obtain this metric, sample was first classified into two groups
based on Age, Distance (or Hour) and Marital variable, respectively.
Next, Each student was assigned 1 if older (Age) farther (Distance),
or married (Marital); 0 if otherwise. Finally, assigned values of
three variables were added up to get the measure of Maturity. Hence,
the value of Maturity has the range between 0(least mature) and
3(most mature). On-Off = 1 if the student is offline learner; 0 if
online learner.
2) Estimates and t-statistics (parenthesis) from the regression are
shown.
***: Significant at a<0.01; **: significant at a<0.05; *:
significant at a<0.10;
