Comparisons of performances between online learners and offline learners across different types of tests.

Huh, Sungkyoo ; Yoo, Sehwan ; Jin, Jongdae 等

INTRODUCTION

A considerable body of research on distance learning suggests that there is no significant difference in achievement levels between online learners and offline learners (E.G., The Institute for Higher Education Policy (1999), Chamberlin (2001) and Yin et. al. (2002)). However, most of these previous studies examined the course grade but not the components of the course grade such as multiple choice questions, assignments, problems, etc. Besides that, online learners may perform differently than offline learners due to differences in student perception, available learning tools, use of the learning tools, and other technical issues. (See Barker (2002), Beard et. al. (2002), Dunbar (2004), Kendall (2001), Lightner et. al. (2001), Perreault et. al. (2002), Schulman et. al. (1999), Schwartzman et. al. (2002), and Woods (2002)) Thus, the purpose of this study is to examine student performances in those course grade components (multiple choice and non-multiple choice questions, in particular) to see if there are any differences in their performances between on-line learners and off line learners.

The remainder of the paper is organized as follows: first, sample data descriptions are discussed the next section, which is followed by discussions on data analyses and their results. Concluding remarks are made in the final section.

SAMPLE DESCRIPTIONS

Sample data are collected from students who took undergraduate accounting courses offered through online as well as offline at California State University-San Bernardino during the three years from fall 2003 to spring 2005. Both online and offline classes were taught by the same instructor who used Blackboard as a web-based learning assistance tool. The same textbook was used and the same lecture notes for each chapter developed by the instructor were provided to students in both classes. Exams for on line and off line classes are developed by the instructor in such a way that exams for on line classes are equivalent to those for off line classes. All exams were proctored and graded by the same instructor.

Student performance data such as test scores and GPA are collected from the course instructor or the university database, while student demographic data such as gender, age, and working hours are from survey questionnaires to the student sample. After deleting students with insufficient data, the final data of 119 students are analyzed in this study.

The sample descriptions are presented in Table 1. There are no significant difference in gender compositions, marital status, GPA, the number of courses taking, and class standing between on line learners and their matching off line learners. On the other hand, significant differences exist in age, commuting distance, and working hours between on line learners and off line learners. Thus, it is necessary to control for the effect of these differential factors between the two learner groups on student performances to examine the net difference in student performances between on line learners and off line learners in this study.

ANALYSIS AND RESULTS

Preliminary comparisons between online learners and offline learners in their performances in multiple-choice questions and non-multiple choice questions are made and their results are presented in Table 2. There are significant differences in total scores and multiple choice scores but not in non-multiple choice scores between online learners and offline learners. Since multiple choice scores and non-multiple choice scores are two major determinants of total scores, the significant difference in total scores may be due to the significant difference in multiple choice scores. (1)

As suggested in many previous studies, student performances can be affected by student characteristics such as gender, age, educational experience, and motivation. (E.G., Sullivan (2001), Younger (1999)) Thus, effect of these characteristics on student performances should be controlled for to see the online versus offline difference in the performance. For this, the following comparative static analyses are conducted and their results are presented in Tables 3 through 6.

In order to control for the effect of GPA on student performances, all sample students are divided into two subgroups: i.e., LOW GPA and HIGH GPA. Students with higher GPA than the sample mean GPA of 3.144 belong to HIGH GPA, while students with lower GPA than the sample mean GPA to LOW GPA. As shown in Panel A of Table 3, there are significant differences in total scores between online learners and offline learners in LOW GPA group, while no significant differences between online learners and offline learners in HIGH GPA group. Offline learners with low GPA do significantly better than online learners with low GPA by on average of 9.461 points, which is statistically significant at 1%.

The similar results are found for multiple choice scores shown in Panel B of Table 3. Offline learners in both LOW GPA and HIGH GPA groups earn higher points in multiple choices than online learners by on average 5.583 points in LOW GPA and 2.207 points in HIGH GPA, which are statistically significant at 1% and 10 %, respectively. This different performance between on line learners and off line learners may not be due to the difference in question type, because both on line class and its matching off line class were taught by the same instructor using the same textbook and supplementary learning materials. Besides that, the instructor used and graded the same student learning assessment rubrics including questions in both on line class and its matching off line class.

If students with low GPA have poorer studying habits than those with high GPA, it is intuitively appealing that students with low GPA perform better in a more controlled learning environment (Off line course) then in a self driving learning environment (On line course). However, there are no significant differences in non-multiple choice scores between online and offline learners.

To control for the effect of gender on performances, sample students are divided into female group and male group. As shown in Table 4, there are no significant differences in total scores, multiple choice scores, and non-multiple choice scores between female online learners and male offline learners. Similar results are observed from male learners.

Results from comparisons between online learners and offline learners after controlling for the age effect are presented in Table 5. Sample students are classified as young if their ages are lower than the sample mean age, or classified as old. Old offline learners earn higher total scores, multiple choice scores, and non-multiple choice scores than old online learners by on average of 10.5972 points, 4.618 points, and 5.4525 points, respectively, all of which are statistically significant at 10%. However, there are no significant differences in any scores between young online learners and young offline learners.

Results from comparisons between online learners and offline learners after controlling for the effect of working hours are presented in Table 6. Sample students are classified as short working if they work less than the sample mean working hours, or classified as long working. There are no significant differences in any scores between online learners and offline learners in both short working and long working groups.

Regression Analyses

Coefficients of correlations between influencing factors on student performances are computed to control for the interaction effect of those related factors. As shown in Table 7, there is a significant positive correlation between working hours and commuting distance. Age, commuting distance, and working hours have significant positive correlations with online-offline identifier, indicating that online learners are older, live further away from the campus, and work longer hours than off line learners. Thus, product terms of these interrelated factors are included in the following regression model to control for their interaction effects on student performances. (2)

Scores = [[alpha].sub.0] + [[alpha].sub.1] Gender +[[alpha].sub.2] Age + [[alpha].sub.3] Distance + [[alpha].sub.4] Hour + [[alpha].sub.5] On-Off + [[alpha].sub.6] Distance * Hour + [[alpha].sub.7] On-Off * Age + [[alpha].sub.8] On-Off * Distance + [[alpha].sub.9] On-Off * Hour + [epsilon] (1)

Where Scores = total score, multiple choice scores, or non-multiple choice scores, Distance = the distance from a student's residence to the campus, Hour = the number of working hours, On-Off = 0 if offline or 1, [[alpha].sub.1] = the partial regression coefficients of variable 'i', [epsilon] = the error term.

Results from the multiple regression model (1) are presented in Table 8. The regression coefficients of On-Off are -0.616, -0.508, and -0.639 for total scores, multiple-choice scores, and non-multiple choice scores, respectively, all of which are not statistically significant. These results indicate that there are no significant differences in total scores, multiple scores, and non-multiple scores between online learners and offline learners.

Another way to measure a net effect of On-Off on Scores after controlling for the effects of all the other influencing variables is to run a two- step regression in which the following regression model is estimated in the first step,

Scores = [[alpha].sub.0] + [[alpha].sub.1] Gender + [[alpha].sub.2] Age + [[alpha].sub.3] Distance + [[alpha].sub.4] Hour + [[alpha].sub.6] Distance * Hour + [[alpha].sub.7] On-Off * Age + [[alpha].sub.8] On-Off * Distance + [epsilon] (2)

In the second step, the error term from the first step ([epsilon]) is regressed over On-Off variable using the following model,

[epsilon] = [[alpha].sub.0] + [[alpha].sub.1] On-Off + [epsilon] (3)

Results from this two-step regression analyses are presented in Table 9. The regression coefficients of On-Off from the model (3) are -1.002, -0.779, and -0.714 for total scores, multiple choice scores, and non-multiple choice scores, respectively, all of which are not statistically significant. These results are consistent with those from a multiple regression (1) reported in Table 9.

Mann-Whitney Test

To mitigate the problem of skewness and outliers in Scores, a non-parametric method called Mann-Whitney test is conducted for the performance difference between online learners and offline learners. As presented in Table 10, Z-values are -0.881, -1.343, and -0.332 for total scores, multiple scores, and non-multiple scores, respectively, all of which are not statistically significant at 10%. This confirms that there are no significant differences in total scores, multiple scores, and non-multiple scores between online learners and offline learners, again.

In sum, from comparative static analyses we found that students with low GPA perform better in off line courses than in on line courses. Old students also do better in off line course that in on line courses. From regression analyses and Mann-Whitney test we could not find any significant difference in student performance between on line learners and off line learners, which is robust across different performance measures and testing methodologies.

CONCLUSIONS

Student performances in multiple choice and non-multiple choice questions are examined to see if there is any difference in the performance between on line learners and off line learners in this study. Academic and demographic data of 119 students who took undergraduate accounting courses offered through online as well as offline at California State University-San Bernardino during a three-year period extending from fall 2003 to spring 2005 are examined.

A couple of interesting findings are that students with low GPA perform better in off line courses than in on line courses. Old students also do better in off line course that in on line courses. These findings may have an important implication for student admission decisions to on line classes. In general, results other than the above mentioned two suggest that there are no significantly different student performances between on line learners and off line learners, which is robust across different performance measures and testing methodologies.

Appendix: A Sample Exam. Exam II (ACCT 347)

Name: -- Date: --

Multiple Choice (20 x 3 = 60 points)

1. Hartley, Inc. has one product with a selling price per unit of $200, the unit variable cost is $75, and the total monthly fixed costs are $300,000. How much is Hartley's contribution margin ratio?

A) 62.5%.

B) 37.5%.

C) 150%.

D) 266.6%.

2. Which statement describes a fixed cost?

A) It varies in total at every level of activity.

B) The amount per unit varies depending on the activity level.

C) Its total varies proportionally to the level of activity.

D) It remains the same per unit regardless of activity level.

3. Which statement below describes a variable cost?

A) It varies in total with changes in the level of activity.

B) It remains constant in total over different levels of activity.

C) It varies inversely in total with changes in the level of activity.

D) It varies proportionately per unit with changes in the level of activity.

4. Which one of the following is most likely a variable cost?

A) Direct materials

B) Depreciation

C) Rent expense

D) Property taxes

5. If a company identifies it has a mixed cost, which one of the following is a reasonable option?

A) It should break it into a variable cost element and a fixed cost element.

B) It should consider the cost to be a fixed cost.

C) It should consider the cost to be a variable cost.

D) It should omit the cost from the analysis.

6. Which one of the following computes the margin of safety ratio?

A) actual sales--break-even sales

B) (actual sales--break-even sales) actual sales

C) (actual sales--break-even sales) break-even sales

D) (actual sales--expected sales) break-even sales

7. Wasp, Inc. produced 200 items and had the following costs: Hourly labor, $5,000, depreciation, $2,000; materials, $2,000; and rent, $3,000. How much is the variable cost per unit?

A) $60

B) $50

C) $25

D) $35

8. Select the correct statement concerning the cost volume-profit graph that follows

[GRAPHIC OMITTED]

A) The point identified by 'B' is the breakeven point.

B) Line F is the break even line.

C) Line F is the variable cost line.

D) Line E is the total cost line.

9. Which cost is not charged to the product under absorption costing?

A) direct materials.

B) direct labor.

C) variable manufacturing overhead.

D) fixed administrative expenses.

10. Variable costing

A) is used for external reporting purposes.

B) is required under GAAP.

C) treats fixed manufacturing overhead as a period cost.

D) is also known as full costing.

11. In income statements prepared under absorption costing and variable costing, where would you find the terms contribution margin and gross profit?

Contribution margin Gross profit Gross profit

A) In absorption costing In variable costing income statement
 income statement

B) In absorption costing In both income statements
 income statement

C) In variable costing In absorption costing income
 income statement statement

D) In both income statements In variable costing income statement

12. When units produced exceeds units sold,

A) net income under absorption costing is higher than net income under variable costing.

B) net income under absorption costing is lower than net income under variable costing.

C) net income under absorption costing equals net income under variable costing.

D) the relationship between net income under absorption costing and net income under variable costing cannot be predicted.

13. If a division manager's compensation is based upon the division's net income, the manager may decide to meet the net income targets by increasing production

A) when using variable costing, in order to increase net income.

B) when using variable costing, in order to decrease net income.

C) when using absorption costing, in order to increase net income.

D) when using absorption costing, in order to decrease net income.

14. Manuel Company's degree of operating leverage is 2.0. Techno Corporation's degree of operating leverage is 6.0. Techno's earnings would go up (or down) by -- as much as Manual's with an equal increase (or decrease) in sales.

A) 1/3

B) 2 times

C) 3 times

D) 6 times

15. In cost-plus pricing, the target selling price is computed as

A) variable cost per unit + desired ROI per unit.

B) fixed cost per unit + desired ROI per unit.

C) total unit cost + desired ROI per unit.

D) variable cost per unit + fixed manufacturing cost per unit + desired ROI per unit.

16. In cost-plus pricing, the markup percentage is computed by dividing the desired ROI per unit by the

A) fixed cost per unit.

B) total cost per unit.

C) total manufacturing cost per unit.

D) variable cost per unit.

17. The cost-plus pricing approach's major advantage is

A) it considers customer demand.

B) that sales volume has no effect on per unit costs.

C) it is simple to compute.

D) it can be used to determine a product's target cost.

18. The following per unit information is available for a new product of Blue Ribbon Company:

Desired ROI $48
Fixed cost 80
Variable cost 120
Total cost 200
Selling price 248

Blue Ribbon Company's markup percentage would be

A) 19%.

B) 24%.

C) 40%.

D) 60%.

19. Bryson Company has just developed a new product. The following data is available for this product:

Desired ROI per unit $36
Fixed cost per unit 60
Variable cost per unit 90
Total cost per unit 150

The target selling price for this product is

A) $186.

B) $150.

C) $126.

D) $96.

20. In time and material pricing, the charge for a particular job is the sum of the labor charge and the

A) materials charge.

B) material loading charge.

C) materials charge + desired profit.

D) materials charge + the material loading charge.

21. Ripple Company bottles and distributes Ripple Fizz, a flavored wine beverage. The beverage is sold for $1 per 8-ounce bottle to retailers. Management estimates the following revenues and costs at 100% of capacity.(10 points)

Net sales $2,100,000 Selling expenses-variable $90,000
Direct materials 500,000 Selling expenses-fixed 70,000
Direct labor 300,000 Administrative
 expenses-variable 20,000

Manufacturing 350,000 Administrative
 expenses-fixed 50,000
overhead-variable

Manufacturing 275,000
overhead-fixed

Instructions

A. How much is net income for the year using the CVP approach?

B. Compute the break-even point units and dollars.

C. How much is the contribution margin ratio?

22. Determine whether each of the following would be a product cost or a period cost under an absorption or a variable system for Carson Company (10 points).

 Absorption Variable

 Product Period Product Period

a. Direct Materials -- -- -- --

b. Direct Labor -- -- -- --

c. Factory Utilities (variable) -- -- -- --

d. Factory Rent -- -- -- --

e. Indirect Labor -- -- -- --

f. Factory Supervisory Salaries -- -- -- --

g. Factory Maintenance (variable) -- -- -- --

h. Factory Depreciation -- -- -- --

i. Sales salaries -- -- -- --

j. Sales commissions -- -- -- --

23. Momentum Bikes manufactures a basic road bicycle. Production and sales data for the most recent year are as follows (no beginning inventory): (10 points)

Variable production costs $90 per bike
Fixed production costs $450,000
Variable selling & administrative costs $22 per bike
Fixed selling & administrative costs $500,000
Selling price $200 per bike
Production 20,000 bikes
Sales 17,000 bikes

Instructions

(a) Prepare a brief income statement using variable costing.

(b) Compute the amount to be reported for inventory in the year end variable costing balance sheet.

24. Trout Company is considering introducing a new line of pagers targeting the preteen population. Trout believes that if the pagers can be priced competitively at $45, approximately 500,000 units can be sold. The controller has determined that an investment in new equipment totaling $4,000,000 will be required. Trout requires a return of 14% on all investments. (10 points)

Instructions

Compute the target cost per unit of the pager.

REFERENCES

Barker, Phillip (2002). On being online tutor. Innovations in Education and Teaching International, Vol 39 (1), 3-13.

Beard, L. A. & C. Harper (2002). Student perception of online versus campus instruction. Education, Vol. 122 (4), 658- 664.

Chamberlin, W. S. (2001). Face to face vs. cyberspace: finding the middle ground. Syllabus, Vol. 15, 11.

Cuellar, N. (2002). The Transition from Classroom to Online Teaching. Nursing Forum. July/Sep. pp. 5-13.

Dunbar, Amy E. (2004). Genesis of an Online Course. Issues in Accounting Education. Vol 19, No.3, pp 321-343.

The Institute for Higher Education Policy. (1999). What's the difference?: A review of contemporary research on the effectiveness of distance learning in higher education

Kendall, Margaret (2001). Teaching online to campus-based students: The experience of using WebCT for the community information module at Manchester Metropolitan University. Education for Information, Vol 19, 325-346.

Lightner, S. & C. O. Houston (2001). Offering a globally-linked international accounting course in real time: a sharing of experiences and lessons learned. Journal of Accounting Education, Vol 19 (4), 247-263.

Orde, Barbara J., J. Andrews, A. Awad, S. Fitzpatrick, C. Klay, C. Liu, D. Maloney, M. Meny, A. Patrick, S. Welsh & J. Whitney (2001). Online course development: Summative reflections. International Journal of Instructional Media, Vol. 2 (4), 397-403.

Perreault, H., Waldman L. & Zhao, M. (2002) . Overcoming Barriers to Successful Delivery of Distance-Learning Courses. Journal of Education for Business. July/August. 313-318.

Schulman, A. & Sims, R. (1999). Learning in an Online Format versus an In-class Format: An Experimental Study. Journal Online

Schwartzman, R. & H. Tuttle (2002). What can online course components teach about improving instruction and learning?. Journal of Instructional Psychology, Vol. 29, No. ,29-38.

Sullivan, Patrick. (2001). Gender differences and the online classroom: male and /female college students evaluate their experiences. Community College Journal of Research and Practice. Vol. 25, 805-818.

Woods Jr., Robert H (2002). How much communication is enough in online courses?--exploring the relationship between frequency of instructor-initiated personal email and learner's perceptions of and participation in online learning. International Journal of Instructional Media, Vol. 29(4), pp.377-394.

Yin, L. Roger, L. E. Urven, R. M. Schramm & S. J. Friedman (2002). Assessing the consequences of on-line learning: issues, problems, and opportunities at the University of Wisconsin-Whitewater. Assessment Update, Vol 14, No. 2, pp. 4-13.

Younger, Michael, M. Warrington & J. Williams (1999). The Gender Gap and Classroom Interactions: reality and rhetoric?. British Journal of Sociology of Education; Vol. 20. 325-341.

Sungkyoo Huh, California State University-San Bernardino

Sehwan Yoo, University of Advancing Technology

Jongdae Jin, University of Maryland-Eastern Shore

Kyungjoo Lee, Cheju National University

ENDNOTES

(1) A sample exam consisting of multiple choice and non-multiple choice questions is presented in the appendix.

(2) GPA is not included as an independent variable in the regression model because there is no significant difference in GPA between online learners and offline learners as shown in Table 1.

Table 1: Description of Sample

 Item Online Offline Difference

 Gender F:44 F:25 Mean diff:0.1208
 M:15 M:15 t-val: 1.281
 N:59 N:40 (p-val: 0.20337)

 Age Mean: 30.3333 Mean: 26.5500 Mean:3.783
 SD: 8.397 SD: 6.6984 t-val:2.018
 N:57 N:40 (p-val:0.0477)

 Married Mean: .3793 Mean: .3590 Mean:0.02
(No:0, Yes:1) SD: .4895 SD: .4859 t-val:0.1843
 N:59 N: 39 (p-val: 0.8542)

Distance(mile) Mean: 44.7797 Mean: 18.450 Mean:26.33
 SD: 29.6090 SD: 13.2702 t-val:5.270
 N:59 N:40 (p-val:8.23e-7)

 Working Hour Mean: 31.0702 Mean: 22.3077 Mean:8.763
 (hour) SD: 13.0628 SD: 14.6381 t-val:3.073
 N:57 N:39 (p-val:0.00277)

No. of taking Mean: 3.3898 Mean: 3.650 Mean:-0.26
 courses SD: .8308 SD: .9212 t-val:-1.6292
 N:59 N:40 (p-val: 0.1467)

No. of course Mean: 7.5789 Mean: 7.4500 Mean:0.129
 for graduate SD: 3.0469 SD: 3.063 t-val:0.2047
 N:57 N:40 (p-val:0.838)

 GPA Mean: 3.1458 Mean: 3.1421 Mean:0.04
 SD:0.4651 SD:0.49574 t-val:0.0364
 N:50 N:40 (p-val:0.9710)

Table 2: Simple Mean Comparisons Between Online and Offline Learners

Item Online Offline Difference

Total Score Mean: 68.55385 Mean: 73.06849 Mean diff:-4.51464
 SD:15.0973 SD:13.2578 t-val:-1.87045
 N:65 N:73 (p-val: 0.06357)

Multiple Mean: 43.50769 Mean:46.41781 Mean diff:-2.91011
Choice SD:7.7341 SD:6.5187 t-val: -2.39784
 N:65 N:73 (p-val: 0.01785)

Non-Multiple Mean: 26.05385 Mean: 26.66438 Mean diff:-0.6105
Choice SD:7.3108 SD:7.6981 t-val: -0.4807
 N:65 N:73 (p-val: 0.6315)

Table 3: Mean Comparisons after controlling for GPA

Panel A: total Scores

 Item Online Offline Difference

Low GPA Mean: 59.083 Mean: 68.544 Mean diff: 9.461
 SD: 15.3066 SD: 13.3822 t-val: 2.83
 N:30 N:45 (p-val: 0.00599)
High GPA Mean: 76.671 Mean: 80.339 Mean diff: 3.667
 SD: 9.0681 SD: 9.3779 t-val: 1.571
 N:35 N:28 (p-val: 0.12129)

Panel B: Multiple Choice Scores

 Item Online Offline Difference

Low GPA Mean: 38.716 Mean: 44.300 Mean diff: 5.583
 SD: 7.7589 SD: 6.5064 t-val: 3.36
 N:30 N:45 (p-val: 0.00121)
High GPA Mean: 47.614 Mean: 49.821 Mean diff: 2.207
 SD: 4.8614 SD: 4.9837 t-val: 1.771
 N:35 N:28 (p-val: 0.08159)

Panel C: Non-Multiple Choice Scores

 Item Online Offline Difference

Low GPA Mean: 24.266 Mean: 24.266 Mean diff: 1.716
 SD: 7.6291 SD: 7.4445 t-val: .969
 N:30 N:45 (p-val: 0.33589)
High GPA Mean: 29.057 Mean: 30.517 Mean diff: 1.4728
 SD: 5.5539 SD: 6.1153 t-val: .992
 N:35 N:28 (p-val: 0.32524)

Table 4: Mean Comparisons after Controlling of Gender

Panel A: Total Scores

Item Online Offline Difference

Male Mean: 69.41176 Mean: 74.58333 Mean diff: -5.17
 SD: 18.129 SD: 14.1616 t-val: 0.89031
 N:17 N:15 (p-val: 0.38039)
Female Mean: 70.28629 Mean: 72.65 Mean diff: -2.3637
 SD: 13.619 SD: 11.932 t = 0.75784
 N:62 N:25 p = 0.45064

Panel B: Multiple Choice Scores

Item Online Offline Difference

Male Mean: 45.79412 Mean: 46.76667 Mean diff: 0.97252
 SD: 6.339 SD: 7.088 t-val: 0.4098
 N:17 N:15 (p-val: 0.68487)
Female Mean: 44.39516 Mean: 46.16 Mean diff: -1.7649
 SD: 7.710 SD: 5.796 t = 1.03151
 N:62 N:25 p = 0.30523

Panel C: Non-Multiple Choice Scores

Item Online Offline Difference

Male Mean: 24.79412 Mean: 27.85 Mean diff:
 SD: 9.8924 SD: 7.611 t-val: 0.96917
 N:17 N:15 (p-val:0.34021)
Female Mean: 25.89113 Mean: 26.51 Mean diff: -0.6189
 SD: 6.8289 SD: 7.2033 t = 0.37658
 N:62 N:25 p = 0.70743

Table 5: Mean Comparisons after Controlling for Age

Panel A: Total Scores

Item Online Offline Difference

Young Mean: 72.1087 Mean: 72.96774 Mean diff:-0.8590
 SD: 13.579 SD: 12.769 t = 0.27877
 N:46 N:31 p = 0.78119

Old Mean: 66.375 Mean: 76.97222 Mean diff: -10.5972
 SD: 14.972 SD: 12.483 t = 1.96319
 N:38 N:9 p = 0.05582

Panel B: Multiple Choice Scores

Item Online Offline Difference

Young Mean: 45.51087 Mean: 46.35484 Mean diff:
 SD: 7.535 SD: 6.022 t = 0.5211
 N:46 N:31 p = 0.60383

Old Mean: 42.88158 Mean: 47.5 Mean diff: -4.618
 SD: 7.080 SD: 6.727 t = 1.77504
 N:38 N:9 p = 0.08265

Panel C: Non-Multiple Choice Scores

Item Online Offline Difference

Young Mean: 26.59783 Mean: 26.64516 Mean diff: -0.0474
 SD: 6.936 SD: 7.5987 t = 0.02826
 N:46 N:31 p = 0.97753
Old Mean: 24.01974 Mean: 29.47222 Mean diff: -5.4525
 SD: 8.0222 SD: 6.5555 t = 1.89009
 N:38 N:9 p = 0.0652

Table 6: Mean Comparisons after Controlling for Working Hours

Panel A: Total Scores

Item Online Offline Difference

Short Mean: 70.98611 Mean: 72.65385 Mean diff:-1.66774
working SD: 14.2171 SD: 14.395 t = 0.37974
hours N:18 N:26 p = 0.70605

Long Mean: 70.21795 Mean: 76.25 Mean diff:
working SD: 13.959 SD: 8.9320 t = 1.45639
hours N:39 N:13 p = 0.15154

Panel B: Multiple Choice Scores

Item Online Offline Difference

Short Mean: 44.66667 Mean: 45.80769 Mean diff:-.4109
working SD: 7.3083 SD: 6.9743 t = 0.52328
hours N:18 N:26 p = 0.60353

Long Mean: 43.42308 Mean: 48.11538 Mean diff:-4.6923
working SD: 7.9128 SD: 4.032 t = 2.04193
hours N:39 N:13 p = 0.04645

Panel C: Non-Multiple Choice Scores

Item Online Offline Difference

Short Mean: 26.31944 Mean: 26.88462 Mean diff:-0.56518
working SD: 7.8603 SD: 7.7773 t = 0.23598
hours N:18 N:26 p = 0.8146

Long Mean: 26.79487 Mean: 28.13462 Mean diff: -1.3397
working SD: 6.9497 SD: 7.1031 t = 0.59875
hours N:39 N:13 p = 0.55205

Table 7: Correlation Coefficients

 Gender Age Distance

Gender 1
Age -0.067 1
Distance 0.026 0.112 1
Working Hour 0.075 0.026 0.358 **
GPA -0.141 0.064 -0.079
On-Off 0.129 0.236 * 0.472 **

 Working Hour GPA On-Off

Gender
Age
Distance
Working Hour 1
GPA -0.075 1
On-Off 0.302 ** 0.004 1

*: Correction is significant at the 0.05

**: Correction is significant at the 0.01

Table 8: Single-Step Regression Analyses

Model Unstandardized Standardized
 Coefficients Coefficients

 B Std. Error Beta

Panel 1. Total Scores

Constant 77.831 14.354
Age .016 .358 .010
Distance .175 .233 .344
Working Hour .001 .202 .001
Distance * Working Hours .001 .006 .083
On-Off 16.655 16.234 -.616
Gender -.939 3.588 -.033
GPA -3.022 3.225 -.118
On-Off * Age -.220 .433 -.272
On-Off * Distance -.224 .224 -.508
On-Off * Working Hour -.235 .277 -.310

Panel 2. Multiple Choice Scores

Constant 49.091 7.713
Age .052 .192 .058
Distance .127 .125 .458
Working Hour .038 .108 .073
Distance * Working Hours -.002 .003 -.270
On-Off -7.496 8.724 -.508
Gender -.684 1.928 -.044
GPA -2.164 1.733 -.155
On-Off * Age -.147 .233 -.333
On-Off * Distance -.090 .120 -.374
On-Off * Working Hour -.090 .149 -.217

Panel 3. Non-Multiple Choice Scores

Constant 28.704 7.624
Age -.036 .190 -.042
Distance .047 .124 .177
Working Hour -.037 .107 -.074
Distance * Working Hours .003 .003 .436
On-Off -9.124 8.623 -.639
Gender -.240 1.906 -.016
GPA -.839 1.713 -.062
On-Off * Age -.073 .230 -.172
On-Off * Distance -.134 .119 -.575
On-Off * Working Hour -.145 .147 -.362

Model t Sig.

Panel 1. Total Scores

Constant 5.422 .000
Age .044 .965
Distance .75 .456
Working Hour .005 .996
Distance * Working Hours .171 .865
On-Off 1.026 .308
Gender -.262 .794
GPA -.937 .352
On-Off * Age -.507 .613
On-Off * Distance -.999 .321
On-Off * Working Hour -.848 .399

Panel 2. Multiple Choice Scores

Constant 6.365 .000
Age .273 .786
Distance 1.014 .314
Working Hour .351 .727
Distance * Working Hours -.565 .574
On-Off .859 .393
Gender -.355 .724
GPA -1.248 .216
On-Off * Age -.630 .531
On-Off * Distance -.746 .458
On-Off * Working Hour -.603 .548

Panel 3. Non-Multiple Choice Scores

Constant 3.765 .000
Age -.192 .849
Distance .382 .703
Working Hour -.348 .729
Distance * Working Hours .895 .374
On-Off 1.058 .294
Gender -.126 .900
GPA -.490 .626
On-Off * Age -.319 .751
On-Off * Distance -1.124 .265
On-Off * Working Hour -.985 .328

Table 9: Two-Step Regression Analyses

Panel 1. Total Scores

Model Unstandardized Standardized t Sig.
 Coefficients Coefficients

 B Std. Error Beta

Constant .664 2.510 .265 .792
On-Off -1.002 3.083 -0.036 -.325 .746

Panel 2. Multiple Choice Scores

Model Unstandardized Standardized t Sig.
 Coefficients Coefficients

 B Std. Error Beta

Constant 0.441 1.184 0.373 0.710
On-Off -.779 1.574 -0.055 -0.495 0.622

Panel 3. Non-Multiple Choice Scores

Model Unstandardized Standardized t Sig.
 Coefficients Coefficients

 B Std. Error Beta

Constant 0.404 2.214 0.183 0.856
On-Off -0.714 2.942 -0.027 -0.243 0.809

Table 10: Mann-Whitney Test

Panel 1. Total Scores

GRADE ON_OFF N Mean Rank Sum of Ranks

 Offline 40 53.09 2123.50
 Online 59 47.91 2826.50
 Total 99

 Z value = -0.881 p-value = 0.378

Panel 2. Multiple Choice Scores

GRADE ON_OFF N Mean Rank Sum of Ranks

 Offline 40 54.70 2188.00
 Online 59 46.81 2762.00
 Total 99

 Z value = -1.343 p-value = 0.179

Panel 3. Non-Multiple Choice Scores

GRADE ON_OFF N Mean Rank Sum of Ranks

 Offline 40 51.16 2046.50
 Online 59 49.21 2903.50
 Total 99

 Z value = -0.332 p-value = 0.740