Distance learning in a core business class: determinants of success in learning outcomes and post-course performance.
Ruth, Derek ; Conners, Susan E.
INTRODUCTION
One of the most rapidly growing areas in college education is the
offering of distance learning courses (Pina, 2008). For example, in the
ten year period leading up to 2007, enrollment of students in some form
of online learning coursework had increase some 4.4 times, and
online-only students had increased an incredible 11.5 times (State
University System of Florida, 2008), and Although there exist today
numerous online colleges, an increasing number of traditional college
institutions are introducing and expanding distance learning courses and
programs for their students in addition to offering courses in a
traditional classroom setting. In these dual mode institutions, many
students take both traditional and online courses. Although recent
research, particularly in the area of education, has compared distance
learning courses with their traditional counterparts in terms of
learning outcomes from those target courses (Spooner et al, 1999) , far
less is known about how students fare once they have moved beyond those
target courses. Distance learning courses offer benefits to both the
students who take the course and the institutions offering them. For
students, it allows for asynchronous learning and frees them from the
need to be in a specific location when learning. At the same time, some
have expressed concerns about the quality of education that students
receive in distance learning classes relative to their peers taking
similar courses in the traditional classroom setting (Carr, 2000 &
Schoech, 2000).
The purpose of this paper is to explore and compare the
determinants of student learning outcomes in distance learning courses
with traditional classroom-based courses. The major contribution of this
paper is that we not only compared learning outcomes of traditional
versus online students, but we also followed their academic performance
in the semester after they left the course of interest to see if either
teaching method had any lasting effect on learning outcomes
COURSE DELIVERY
The target course for this study was Management 101, an
introductory course that is required for all management students, but
also taken widely by students outside of the school of management. Many
of these students cannot physically attend class so instruction is
delivered online asynchronously as well as in a traditional classroom
setting. The online courses are delivered using the Blackboard software
system with traditional instructional design methods to construct the
course. The course content is identical when delivered online as on
campus. The asynchronous delivery accommodates the various time zones
and geographical challenges for students. The software allows for
electronic posting of content to supplement the textbook, electronic
submission of assignments, discussions, and online testing.
ISSUES OF DISTANCE LEARNING EDUCATION
From a student standpoint, there are at least two advantages to
being able to take courses online. In most cases, distance learning
courses are asynchronous, allowing students to fulfill course
requirements at a scheduled pace, rather than having to attend class at
specific times, such as is the case for traditional classroom-based
courses. Such flexibility of scheduling improves students'
abilities to schedule coursework around family, work, and other
obligations. In addition to more flexible scheduling, distance-learning
courses typically offer students flexibility of location. In being able
to take a course via distance learning, students are no longer confined
to living near to the campus from which the course is taught.
While there are still conflicting opinions on distance learning,
now a research base supports the value of online distance education.
According to Prestera & Moller (2001), "Today's
computer-mediated communication tools are used to create rich learning
environments where many-to-many relationships can flourish. At the same
time that technological advancements are improving our capacity to
deliver instruction at a distance, two forces are reshaping education
and workplace learning: the reexamination of what learning means and the
willingness to reconsider instructional formats". Another advantage
is that students are introduced to and use the technologies used in
business (Leidner & Jarvenpaa 1993).
The benefits that accrue to students in distance learning classes
end up being advantageous to colleges as well. In eliminating the
traditional classroom setting, distance-learning courses mean a reduced
burden on university facilities and reduce overhead costs overall on a
per student basis. Since distance-learning courses can be offered at
locations independent of the main campus, this also allows the
university to attract students from outside of a school's
traditional recruiting areas. Distance-learning is becoming an important
option within education because it facilitates the sharing of costs,
information and expertise in multiple locations (Webster & Hackley
1997). This can help schools increase overall enrollment and/or increase
the quality of the average student at the school.
Despite the benefits associated with distance-learning, some have
expressed concern about the quality of distance-learning courses
relative to those based in a traditional classroom. Much of this concern
is rooted in the lack of face-to-face interaction between professor and
student in many distance-learning courses. College administrators
interviewed in The Chronicle of Higher Education (Carr, 2000a) agreed
that course completion rates in distance education courses are often
lower than in traditional classes. Distance-learning courses also tend
to have more flexible schedules than those of their distance-learning
counterparts.
Distance-learning courses often have minimal face-to-face
interaction between student and instructor. In order to compensate for
this, distance-learning courses often build in other interactive
activities such as discussion forums and chat rooms. Whereas traditional
lectures typically take place one or more times per week at the same
time and in the same place, distance-learning classes often have lessons
and activities available on demand. A process based assessment model is
used in these classes and shifts the emphasis to student based learning
and discovery (Bergstrom, 2010). Regardless of the advantages and
disadvantages of each method of course delivery, the fact remains that
the two methods differ in their approach to teaching. As such, the
purpose of this paper is to begin to explore the factors that affect
students' enrollment and ultimate success in distance-learning
courses.
The remainder of this paper is divided into four sections. In
Theory and Hypotheses, we develop a series of hypotheses that will be
tested later in the paper. In Data and Methods, we detail the data
collection and statistical methods used to test the hypothesis. In
Results, we discuss the outcomes of our data analyses and results of our
hypothesis testing. In Discussion, we explore the meaning of the results
as well as their implications for both researchers and educators.
THEORY AND HYPOTHESES
As students progress in their college programs, demands on their
time often increase. As students age, they are more likely to be married
and/or have children that are demanding of their time. As their time in
school progresses, many if not most students have to deal with climbing
student debt loads and may need to work in order to meet those financial
obligations. We believe that these pressures may encourage students to
gravitate toward the flexibility of distance learning courses
(Pierrakeas et al, 2004).
As well, students will become more accustomed with the obligations
and expectations of coursework at their school of choice, increasing the
likelihood that a student will feel comfortable to enroll in a distance
learning course (Tuckman, 2007). Taken together, we expect to find that:
H1: Students will be more likely to take distance learning courses
later in their academic programs.
As discussed previously, there are both advantages and
disadvantages associated with the provision of courses via distance
learning. In terms of the net effect of distance learning on student
performance, we have no reason to believe that students'
performance will differ significantly from those of their peers who are
taking the course in a traditional setting. Prior research has shown
that learning outcomes need not be deficient in distance learning
courses, so long as the teaching methods are also effective (Schultz et
al, 2008; Kan & Cheung, 2007). Because our third hypothesis looks at
post-Management 101 performance of students, we must first test the
following hypothesis:
H2: There will be no significant difference in the performance of
students enrolled in a distance learning course in comparison with
students enrolled in a traditional course.
Although previous studies have considered a question similar to
that of Hypothesis 2, there is very little research that has tracked
student performance once those students have moved beyond a target
course to see whether or not there are lasting effects of taking the
given target course in a distance learning versus a traditional setting.
This is of particular concern in so-called dual-mode schools that offer
both online and traditional courses to the same group of students.
Ultimately, it is important to understand whether or not a former
distance learning student is able to perform equally well when they
return to the traditional classroom. In addition to expecting no
performance differences between distance learning and traditional
students, we also do not expect to find performance differences as
students move on to other coursework in their program:
H3: There will be no significant difference in post-course
performance between distance learning and traditional students.
DATA AND METHODS
Data was collected from 118 students enrolled in five different
Management 101 Introduction to Business courses offered in the Spring
and Summer semesters of 2009. As shown in Table 1, three of the courses
were distance learning (47 students total) and two were taught in a
traditional classroom setting (71 students total).
Variable Definitions
In order to identify the learning format of each course, a dummy
variable, Dummy if Distance Learning, was created, which took the value
of 1 if students took Management 101 via Distance Learning and 0 for the
Traditional format course. Semester of Program was coded as a number
from 1 to 8. This number took integer values and represents increments
of fifteen hours' worth of courses that the students has completed
(0-15 hours=1; 16-30 hours=2 and so on). To control for the possibility
that differences might be due to differential student aptitudes, we
controlled for each student's GPA. In results not reported, we also
looked at the influence of such variables as gender and race but the
results were robust in terms of sign and significance. In order to
compare the performance of students inside and outside of the school of
management, we created a dummy variable (School of Management) that took
a value of 1 if the student was enrolled in a management major and zero
otherwise (please see Table 2 for a breakdown of student enrollments).
Grade in Next Management course was used to assess the performance of
students after they had taken Management 101. This course was designated
as the first management course taken after Management 101.
Analyses
Hypothesis 1 was tested using a one-way ANOVA analysis with an
F-test to test for significant difference between the stage of program
in which students enrolled in the traditional versus distance learning
course format (results are presented in Table1). Hypotheses 2 and 3 were
tested using multiple regression and missing data was deleted list-wise
(results are presented in Tables 3 and 4). Because we were concerned
that significant results may be driven by differing performance among
students from different colleges around campus, we also tested all
hypotheses twice: once using all students enrolled in Management 101,
and again using only those students who were enrolled in the School of
Management. The two hypotheses were tested using the full sample of
students, and again using only students enrolled in the School of
Management. Tables 1, 3 and 4 outline the results of the study,
discussed below, and Table 2 shows the breakdown of students in the
study by college of enrollment.
RESULTS
Results of our data analyses provided strong support for Hypothesis
1, indicating that students enroll in distance learning courses
significantly later in their programs than do their peers who attend
classes in a traditional setting. On average, students enrolled in the
distance learning version of Management 101 were 1.34 semesters further
along in their programs than their peers who took the course in a
traditional classroom setting. Although the plan of study for for
management students recommends that Management 101 be taken during the
first year of study, the typical student waited until roughly halfway
through their program before taking the Distance Learning version of the
course. As the p-values in Table 1 indicate, the differences between the
means were highly significant. In results not reported, we also tested
all hypotheses in the paper using credit hours completed in the program
as an alternate measure of students' stage in the program. As with
semester of program, credit hours completed showed a sizable and
significant difference between students in distance learning and
traditional classroom students in terms of stage of program.
Hypothesis 2 predicted that there would be no significant
difference between distance learning and traditional students in terms
of their performance in Management 101. Table 3 below shows the results
of our hypothesis testing. Models 2a and 2b represent tests on the
entire sample of students. In Model 2b, we controlled for the school in
which a given student was enrolled to help rule out the possibility that
results were driven by differential skill sets possessed by the students
from different schools. In Model 2c, we tested the hypothesis using only
Management students. In all three models, there was no significant
difference in students' performance in Management 101 (with alpha
set at 0.05), supporting Hypothesis 2.
Hypothesis 3 considers the performance of students in the sample
who went on to take another course in Management. This is the key
hypothesis of the paper because undetected differences in format and
grading styles among the two course formats might be able to explain
grades in Management 101, but any difference in actual learning that
takes place in Management 101 should be reflected in students'
performance as they progress in their respective programs. Models 3a and
3b test the hypothesis with the full sample of students, whereas Model
3c uses only Management students. Note that the sample size drops
appreciably in comparison with the test of Hypothesis 2 because not all
students took a subsequent management course during the sampling window
of the study. In all three samples, there was no significant difference
between students who had taken the Management 101 course via distance
learning. Surprisingly, and although there was no significant
difference, the large and positive coefficients in all three models hint
that students who had taken Management 101 via distance learning may in
fact outperform their peers who took the course in a traditional
setting.
DISCUSSION
This study offers three key findings, the last of which is arguably
the most important and least explored. Given a choice, students seem to
take distance learning courses later in their programs. To the extent
that this is true, schools may want to consider maintaining traditional
classroom-based courses for those courses that fall earlier in a given
program's curriculum in addition to distance learning classes.
Instead, schools may want to focus on making online courses available in
the upper levels of the program first. In this study, the students had
the opportunity to take the target class either in a traditional format
or distance learning.
The second finding is that there seems to be no difference in
performance between those students who take distance learning courses
and those who take courses in a traditional classroom setting. More
important is our last finding, which followed student performance after
they had taken Management 101, to see whether future student performance
remained unaffected by the format in which they took the introductory
course. It is our hope that this study allays some of the concerns that
faculty, students and administrators have regarding the quality of
education provided in a distance learning environment with a
foundational course.
REFERENCES
Bergstrom, P. (2010). Process-based assessment for professional
learning in higher education: perspectives on the student-teacher
relationship. International Review of Research in Open and Distance
Learning. Vol. 11, No. 2.
Carr, S. (2000a, February 11). As distance education comes of age,
the challenge is keeping the students. The Chronicle of Higher
Education, 46(23) pp. A39-41.
Carr, S. (2000b, March 10) Online psychology instruction is
effective, but not satisfying, study finds. The Chronicle of Higher
Education, 46(27), pp. 2-5.
Kan, A.C.N.; Cheung, L.L.M. (2007). Relative effects of distance
versus traditional course delivery on student performance in Hong Kong.
International Journal of Management, 24(4), 763-773.
Leidner, D. & Jarvenpaa, S. (1993). The information age
confronts education: case studies on electronic classrooms. Information
Systems Research, Vol. 4, No. 1, pp. 24-54.
Pierrakeas, C., Xenos, M., Panagiotakopoulos, C. & Vergidis, D.
(2004). A comparative study of dropout rates and causes for two
different distance education courses. International Review of Research
in Open and Distance Learning, 5(2), 15 pp.
Pina, A.A. (2008). Factors influencing the institutionalization of
distance learning in higher education. Quarterly Review of Distance
Education, 9(4), 427-438.
Prestera, G. & Moller, L.(2001). Facilitating asynchronous
distance learning: exploiting opportunities for knowledge building in
asynchronous distance learning environments. Proceedings of Mid-South
Instructional Technology Conference.
Schoech, D. (2000). Teaching over the Internet: Results of one
doctoral course. Research on Social Work Practice, 10, 467-487.
Schultz, M.C., Schultz, J.T., Round, G. (2008). Management of
academic quality: A comparison of online versus lecture course. Journal
of College Teaching & Learning, 5(10), 23-28.
Spooner, F., Jordan, L., Algozzine, B. & Spooner, M. (1999).
Student ratings of instruction in distance learning and on-campus
classes. Journal of Educational Research, 92, 132-141.
State University System of Florida. (2008). What is the role of
distance learning in the State University System? Information Brief.
Board of Governors. Board of Governors, State University System of
Florida. Volume 6, Issue2. 11 pp.
Tuckman, B. (2007). The effect of motivational scaffolding on
procrastinators' distance learning outcomes. Computers &
Education, 49(2), 414-422.
Webster, J. & Hackley, P. (1997). Teaching effectiveness in
technology-mediated distance learning. The Academy of Management
Journal, Vol. 40, No. 6, pp. 1282-1309.
Derek Ruth, Purdue University Calumet
Susan E. Conners, Purdue University Calumet
Table 1--Descriptive Statistics and one-way ANOVA results
comparing Distance Learning and Traditional students based on
their Semester of Program (ranging from 1 to 8)
Course Type N Mean Std. Dev'n
Traditional 47 2.83 1.79
Distance Learning 71 4.17 2.48
Total 118 3.64 2.32
F = 28.650, p-value = <0.001
Table 2--Breakdown of students by school of enrollment.
College of Enrollment Frequency Percent
Management 35 29.4
All Others 83 70.6
TOTAL 118 100.0
Table 3--Multiple regression results for the effect of various factors
on students' grade in Management 101 (p-values in parentheses).
Dependent: Grade in Management 101 Model 2a Model 2b Model 2c
Dummy if Distance Learning -0.150 -0.109 0.397
(0.386) (0.521) (0.203)
Semester of Program 0.064 0.076 0.038
(0.100) (0.048) (0.643)
GPA 1.014 0.992 0.823
(<0.001) (<0.001) (<0.001)
Dummy if School of Management 0.419
(0.019)
Intercept -0.400 -0.538 0.185
(0.118) (0.037) (0.664)
F-Test 47.585 38.698 12.672
(<0.001) (<0.001) (<0.001)
Sample Size 108 108 31
R-Squared 0.576 0.598 0.576
Table 4--Multiple regression results for the effect of various factors
on students' grade in their next management class after Management 101
(p-values in parentheses).
Dependent: Grade in Next Model 3a Model 3b Model 3c
Management Course
Dummy if Distance Learning 0.428 0.424 0.482
(Management 101) (0.145) (0.154) (0.237)
Semester of Program -0.097 -0.095 -0.052
(0.134) (0.155) (0.612)
GPA 1.236 1.232 1.073
(<0.001) (<0.001) (<0.001)
Dummy if School of Management 0.071
(0.796)
Intercept -0.651 -0.679 -0.382
(0.200) (0.196) (0.538)
F-Test 17.012 12.528 9.429
(<0.001) (<0.001) (0.001)
Sample Size 51 51 21
R-Squared 0.515 0.516 0.611
