How are they doing? Examining student achievement in virtual schooling.

Barbour, Michael K. ; Mulcahy, Dennis

Cosby and McDermott (1978) indicated that there was a perception that those living in rural areas represented "a small and insignificant segment of the population" of the United States (p. 6). The authors speculated that this was due to the urban dominance in matters of politics and commerce, along with a general shift in the population from rural to urban areas. These observations are still relevant almost thirty years later and are applicable not only to an American context, but in most rural jurisdictions. It is particularly true of the Canadian Province of Newfoundland and Labrador, which is located on the east coast of Canada. The province, which has both an island and mainland portions, has a total area of approximately 252,000 square miles and a population of a little less than 510,000 people. Although about half of the population resides on the Avalon Peninsula or within a 100 mile radius of the provincial capital, St. John's, the remainder of the province is sparsely populated. In fact 192 of the 294 schools in 2004-OS were located in these rural areas (Government of Newfoundland and Labrador, 2005), 85 of which are designated as necessarily existent (i.e., a term used to describe schools that cannot be closed because they are located so far from another school that it makes bussing the students not feasible due to distance (4)).

As with rural jurisdictions across North America, many of the schools in Newfoundland and Labrador do not have enough teachers and are unable to provide sufficient variety in the course offerings required by the provincially-mandated curriculum (Barker, 1985; Benson, 1998; Government of New York, 1992), often times due to their inability to attract or retain teachers in highly specialized subject areas (Collins, 1999; Furey & Murphy, 2005; Kannapel & DeYoung, 1999; Storey, 1993). In this environment, rural schools are unable to offer their students the same level of educational opportunity as their larger, urban counterparts. Since the late 1970s, the Government has published reports outlining these problems (e.g., Crocker, 1989; Crocker & Riggs, 1979; House, 1986; Riggs, 1987).

Based upon the recommendations of these Government reports, the province implemented a program of distance education for rural high school students in September 1988 using an audio-graphics system. The main purpose of the program was to provide secondary level students with courses that were important for post-secondary admission but that were difficult to offer in rural schools due to low levels of student enrolment. During the 198990 school year, 38 of the 548 schools in the province had fewer than 25 students (Government of Newfoundland, 1990).

In its first year of operation, the Newfoundland and Labrador distance education program consisted of just one course: Advanced Math 1201. This Tele-medicine/TETRA distance education program utilized an audio-graphics system (sometimes referred to as a telematics system) using bridging technology to provide conference calling facilities that were accompanied by a telegraphic device for reproducing handwriting by converting the manually controlled movements of a pen at one site into signals that controlled the movements of a similar pen at another site. Using the Tele-medicine/TETRA distance education program, students would spend 50% to 80% of their instructional time using this synchronous distance education system and the remainder of their time completing correspondence-style work which was submitted using a fax machine.

Over the next three years, additional courses were developed until the entire advanced mathematics curriculum was available. Following the release of a series of Government-sponsored reports (i.e., Crocker, 1989; Williams, 1993), the program was again expanded to include the complete physics and chemistry programs and upper level French as a second language courses. Over a period of twelve years, the program grew from an enrolment of 36 students from 13 rural schools in a single course to 11 courses with 898 course enrolments representing a total of 703 students in 77 different rural schools by 1999-2000 (Brown, Sheppard, & Stevens, 2000). However, there were still calls from Government-sponsored reports for a more comprehensive distance education program (i.e., Williams, 1993). One of the reasons for needing a more comprehensive approach was outlined by Mulcahy (2002) when he indicated that this current system of distance education "demonstrated that many students taking distance courses required and received a significant amount of pedagogical assistance with 'matters of content' from school based personnel" (Classroom Teachers: A Mediating Role, [paragraph] 5). Brown et al. (2000) also outlined another reason when they described how school administrators, teachers, and even parents were well aware that students enrolled in distance education needed to be successful academically, possess self-discipline, have academic ability and have demonstrated that ability in class, and be prepared for extra independent work. The distance education program in place at that time could not accommodate students who did not possess these skills and habits.

Literature Review

Although few jurisdictions in the USA faced geographic challenges as severe as those in Newfoundland and Labrador, distance education opportunities for high school students were also being explored in the United States. As was the case with the audio-graphics distance education system in Newfoundland and Labrador, many of the early examples of distance education programs across North America were primarily designed for a select group of high school students, specifically those with higher aptitudes, higher achievement, and greater aspirations for postsecondary education. For example, in their second year evaluation of the Virtual High School (VHS), Espinoza, Dove, Zucker and Kozma (1999) stated that "it was found that VHS was serving a fairly narrow range of students, those who were academically advanced and college bound" (p. 48). The courses developed by the VHS illustrate this trend. For example, courses such as Advanced Placement Statistics, Environmental Ethics, and Russian, Soviet, and Post-Soviet Studies, were designed and implemented in such a way that these courses excluded all but the most talented and motivated high school students. Research literature also substantiates this trend. Based upon a review of the literature, Roblyer and Elbaum (2000) concluded, "only students with a high need to control and structure their own learning may choose distance formats freely" (p. 61).

In an analysis of 19 studies investigating the effectiveness of interactive distance education technologies in K-12 education that included over 900 participants from 1980 to 1998, Cavanaugh (2001) found that there was "a small positive effect in favour of distance education" (p. 73). Given that distance education for high school students in North America had primarily served a more selective group of students, it should not surprise anyone that these early comparative studies in K-12 distance education yielded better results than most other comparative studies in other technology-based fields. (5) Simply put, the difference in results between distance education students and traditional classroom students in secondary education contexts may be largely explained by the selectivity of students registered in distance education programs. For example, in his analysis of 2,600 student enrolments as a mid-Western virtual high school, Mills (2003) found that the typical online student was an A or B student. In addition, in his report on the state of e-learning in Michigan, Watkins (2005) found that 45% of the students who participated in e-learning opportunities were "either advanced placement or academically advanced" students (p. 37) (also see Wigent & Oswalt, 2004).

Based upon these examples and the current literature in general, it seems plausible that the students in these distance education studies were the independent, self-motivated students who enrolled in the earliest forms of distance education opportunities in Newfoundland and Labrador and elsewhere in North America. It may also be that the students who would not have performed well in the distance education environment had already elected to drop the course before the outcome data were collected. The authors of recent research reports that have found higher student performance in virtual school courses over students in the traditional classroom have cited this as a potential causality (Cavanaugh, Gillan, Bosnick, Hess, & Scott, 2005; McLeod, Hughes, Brown, Choi, & Maeda, 2005). Other scholars have also indicated that in distance programs where student selectivity is not maintained, retention rates decrease significantly (Ballas & Belyk, 2000; Barker & Wendel, 2001; Bigbie & McCarroll, 2000; Kozma et al., 2000; Roblyer, 1999; Roblyer & Elbaum, 2000).

Depending on the source, the drop-out rates range from a low of 10% to a high of 40% to 60% (Oblender, 2002; Zucker & Kozma, 2003), and as Roblyer (2005) reminded us, many virtual schools allow a two to four week period that students can drop their courses without penalty and, in most instances, these students are not included in the official attrition rates. For example, in their sponsored report on K-12 online learning in Alberta, Ballas and Belyk (2000) suggested that the lack of retention of lower performing students in the virtual school sample meant that it was "not reflective of the total population of students" (p. 28). More recently, McLeod, Hughes, Brown, Choi and Maeda (2005) speculated that their own positive results in favour of virtual school students were due to the fact that many of the low-achieving students had dropped out prior to the assessment. Further, in her summary of research into distance education at the K-12 level, Rice (2006) described how many of the comparative studies were flawed because of their failure to account for variables such as early drop-outs, voluntary testing, and tests designed to favor distance education students. Unfortunately, this explanation cannot be verified because the studies included in Cavanaugh's review did not report sufficient attrition data.

Since Cavanaugh's review in 1996, there has a tremendous growth in virtual school opportunities in North America. The first two virtual schools in the United States were the Virtual High School (VHS) and the Florida Virtual School (FLVS). The VHS was created through a five year, $7.4 million federal grant (Pape, Adams, & Ribeiro, 2005), while the FLVS was established through an allocation of $200,000 from the state legislature (Friend & Johnston, 2005). The following school year (i.e., 1997-98) the VHS offered twenty-eight courses to twenty-eight schools that were a part of the initial consortium. The FLVS also began offering courses that same year with an enrollment of 157 students. Even before these first virtual schools in the USA, four schools in the Canadian province of Alberta created virtual schooling programs and offered courses to their students during the 1995-96 school year (Haughey & Muirhead, 2004). In the past decade it is estimated that the number of K-12 students who have engaged in distance education in the United States, including virtual schooling, is more than 300,000 (Setter & Lewis, 2005). A similar increase is speculated to have occurred in Canada.

Interestingly, three years after Cavanaugh's initial review, Cavanaugh, Gillan, Kromrey, Hess and Blomeyer (2004) reported a small negative effect size in their meta-analysis of an additional 14 studies representing over 7500 students from 1999 to 2004. Again, without specific evidence, it is primarily speculation, but it seems reasonable to conclude that this more recent sample of distance education comparative studies was conducted with a more diverse population of students stemming from the greater proliferation of web-based distance education in K-12 contexts.

Methodology

After a series of individual school districts and provincial web-based distance education projects in mathematics, science, and technology, the Government of Newfoundland and Labrador appointed a ministerial panel to, among other things, "examine the current educational delivery model and consider alternative approaches" in 1999 (Sparkes & Williams, 2000, p. 2). In their report, the ministerial panel recommended the creation of the Centre for Distance Learning and Innovation (CDLI) to be based upon the web-based model that had been evolving throughout the province. The vision of the CDLI was to provide access to educational opportunities for students, teachers and other adult learners in both rural and urban communities in a manner that rendered distance transparent; eliminated geographical and demographic barriers as obstacles to broad, quality educational programs and services; and developed a culture of e-learning in the schools which is considered to be an integral part of school life for all teachers and students.

The CDLI came into existence in 2000 and offered its first courses during the 2001-02 school year. During that first year a limited number of enrolments were made available in an effort to field test the method of delivery and the content material that had been developed. Beginning with the 2002-03 school year, any student from across the province was given permission to enrol in any course offered by the CDLI. No longer was secondary distance education intended just for the above average students. With their decision to develop a number of non-highly-academic courses, such as Art Technologies 1201, Communications Technology 2104/3104, and World Geography 3202, their student population should include students of all ability levels. In addition to the creation of courses such as these, the CDLI has a retention rate of over 90% (M. Barry, personal communication, May 19, 2006), a count which includes all students who initially register for their CDLI courses the May prior to beginning their course (i.e., the less than 10% that drop out include those who decide over the summer not to take the course, and essentially never actually start their course).

For this study, we obtained final course scores and the standardized public exam scores for every student in the Province of Newfoundland and Labrador for the school years 2001-02 to 2004-OS from the provincial Department of Education. Using the Statistics Canada definitions for rural and urban communities (6), we coded schools in the province based upon their geographic location. Then we combined this data with information from the High School Distance Education Course Report to determine which schools offered what courses using web-based distance education and which schools offered what courses in the traditional classroom environment. In some instances there were schools that had a number of students registered in a classroom version of a specific courses, but also had one of two students who are unable to fit the classroom delivered version into their schedule and the school simply registers them in the CDLI's web delivered version. In these situations, the data did not allow us to determine which one or two students of the twenty or thirty students were the CDLI students, so all of the students in that specific course at that specific school were excluded. Initially this was a practice common in only a few urban high schools, but in recent years has become more common in both urban and larger, regional rural schools.

As our sample included 95% and 99% of the population, depending on the year, and 97% of the population over the four year period, we decided that descriptive statistics were a sufficient method of data analysis for our purpose. If there was a higher percentage of missing cases, we would have selected another method of statistical analysis to compare these means. The number of missing cases has been increasing fairly dramatically in recent years (i.e., from 1% in 2001-02 for the final course averages and from 0.5% in 2002-03 for the public exam scores to 5% in 2004-05 for both measures). This is a growing limitation of this particular line of inquiry and, as this number increases, it will eventually render this kind of analysis unusable. Unfortunately this number will continue to increase with the current data sources.

Results and Discussions

The purpose was to determine if there were performance differences based on the method of delivery and the location of the student. Table 1 indicates the final course averages for all the students registered in courses offered by the CDLI sorted by delivery method of the course and student location.

The data above indicate that for each of the first three years that the CDLI has been in operation, students from rural areas in the web-based courses offered by the CDLI have performed as well or better than any of their classroom or urban counterparts. This is consistent with the findings of other virtual school researchers (e.g., Ballas & Belyk, 2000; Cavanaugh at al., 2004), although as it was indicated earlier their populations may not have been as representative in terms of student abilities at the one represented by the CDLI. In the fourth year, both web delivered groups of students (i.e., both rural and urban) performed slightly better than their classroom counterparts. However, with the tens of thousands of students represented, the small number of web delivered urban students in each year makes their scores less than reliable.

The dramatic increase in the number of cases in the first three years of CDLI operation was primarily due to the increase in the number of courses offered by the CDLI. For example, adding Art Technology 1201 in 2002-03 increased the number of web-based cases by seventy-eight web-based cases and classroom cases by 1578, or English 1201 in 2003-04 which added 19 web-based cases and 5306 classroom cases.

CDLI offerings, like those of traditional schools are based on enrolments. Even after a course has been developed for web-based deployment, there are some years the enrolment doesn't justify the allocation an e-teacher (such as English 1201 during the 2004-OS school year).

In addition to the final course averages, in certain level three (i.e., grade twelve) courses, students are required to take a standardized public exam. In Newfoundland and Labrador certain academic grade twelve courses have province-wide, standardized exams. These exams include a multiple-choice portion and an essay portion, the latter is graded by a single marking committee. Beginning in the 2002-03 school year, the CDLI began to offer certain courses that required students to take the public exam. A summary of the results of these CDLI students compared to their classroom counterparts can be found in Table 3.

Like the final course averages, during the 2002-03 the web-based students in rural areas performed as well as any other group of students (excluding the single urban student who was enrolled in a web-based CDLI course). However, during the 2003-04 school year the performance of both rural and urban students in the web-based courses offered by the CDLI scored lower on their public exams that students who received their instruction in a traditional classroom. This past year this trend was reversed again, with both rural and urban students in the web delivered courses scoring higher on their public exams than the classroom delivered students. While the finding from the 2003-04 school year is more consistent with the literature discussed in earlier (i.e., Cavanaugh et al., 2005; McLeod at al., 2005), it is the only instance of the two measures during this three year period where the performance of rural students in web-based environments did not do as well as or better than any of the other categories.

The findings for the first two years the CDLI offered public exam courses was consistent with earlier work we completed with a similar student population (see Barbour & Mulcahy, 2006). In this study, we examined the difference in retention and student performance with Advanced Placement (AP) courses in the province of Newfoundland and Labrador, again comparing the urban students with the rural students and the web-based students with the classroom students. In this study we found that the rural web-based students performed better than the rural classroom students and the urban students. While we attempted to control for selectivity by only utilizing a sample of only AP students, the low retention rate with the rural web-based students and even lower percentage of rural web-based students taking the exam likely meant that only those who were confident in their success actually completed the AP exam (and were included as a part of our analysis).

Even though a year by year comparison of the students' public exam scores and final course averages provides a valuable look at the data, an overall comparison of the four year period is also in order. Table 4 provides such a comparison.

This combined analysis indicated that over the four year period the CDLI has been in operation there was some fluctuation in both performance measures when both delivery model and location were considered, but little difference in the overall performance of students based upon delivery model in both their public exam scores (i.e., a 61.8% average for the web-based students compared to 62.8% for the classroom students) and final course averages (i.e., a 69.2% average for the web-based students compared to 68.1% for the classroom students). Overall, the individual yearly data comparisons indicate that rural students who accessed their courses from the CDLI performed better or the same as their classroom counterparts on final course scores in all four years. This trend was consistent in two of the three (i.e., the first and third years) of public exam data.

There are a number of possible reasons for the lack of performance differences between the CDLI students and their classroom counterparts. The reality that the method of delivery, a combination of synchronous and asynchronous instruction (described in Barbour, 2007), utilized by the CDLI is rather unique compared to other virtual schools in North America. It could be the CDLI's use of school-based mediating teams for the organization and administration of these virtual school opportunities. In addition, it may also be due to content-based assistance the students receive from their school-based teachers. In the previous distance education system in Newfoundland and Labrador Barbour and Mulcahy (2004) reported that it was a "widely known, but rarely documented, [fact] that students often required and received a significant amount of assistance with matters of content from school based personnel" (New Model for Distance Education, [paragraph] 11). However, in his dissertation research Barbour (forthcoming) has found in a case study of one rural school engaged in CDLI that the students at that school do not received significant amounts of assistance from their school-based teachers. Finally, it could be that the students that take CDLI courses are not representative of the entire student population and student selectivity is present within the CDLI.

Conclusion and Implications

As we argued earlier, in instances where distance education students scored higher than their classroom counterparts it was probably due to a greater degree of student selectivity found in most distance education programs. It was reasonable to surmise that where there is no student selectivity in a distance education program, these K-12 distance students would score lower than their classroom counterparts. However, the findings from the first four years of data with the CDLI do not follow this pattern. While there are a variety of reasons that may explain this difference, the first step is to determine whether or not there is student selectivity within the population of students involved in the CDLI.

If the lack of differences between the overall students' performance based on delivery model is not due to student selectivity it raises several important issues, including the need for more rigorous investigation to explore the reasons for why K-12 distance education high school students seem to do better than their regular classroom peers in some contexts and not in others. What are clearly needed are studies that address the reasons for high school student achievement in distance education.

The present model of delivery utilized by the CDLI includes from 30% to 80% of the students' scheduled time (i.e., 10 one-hour periods over a fourteen day cycle) in synchronous instruction using a real-time virtual classroom, Elluminate Live(r). This is one of the main differences between the CDLI and the majority of other virtual schools in North America, and may also be a reason for the lack of performance differences. However, in the original Ministerial Panel report was intended that the CDLI would use a primarily asynchronous method of delivery (Sparkes & Williams, 2000). The movement towards a more asynchronous delivery system also remains a goal of the CDLI, and one which they hope to act on in the near future (M. Barry, personal communication, May 19, 2006). This would make the CDLI's delivery model similar to that of other virtual schools. A better understanding of what students do while they are engaged in their web-based distance education, but not under the direct supervision of a teacher (regardless of whether the teacher is in the school or at a distance) is also important for developing more effective asynchronous teaching strategies and support systems for the students. Without this understanding of what students are actually doing and how they can be better supported when they are not engaged in synchronous instruction, the CDLI and other virtual schools in North America will have an insufficient foundation for designing better support for their students.

In one of the largest meta-analyses ever conducted related to distance education, Bernard, Abrami, Lou, Borokhovski, Wade, Wozney, Wallet, Fiset, and Huang (2004) found a very small, but statistically significant, positive mean effect size for interactive distance education over traditional classroom instruction on student achievement and a small, but statistically significant, negative effect for retention rate. While this meta-analysis is one of the best of its kind, its findings, as well as those derived from other related meta-analyses (Cavanaugh, 2001; Cavanaugh et al., 2004; Machtmes & Asher, 2000; Ungerleider & Burns, 2003), do not go far enough in specifying design and delivery guidelines for practitioners. Studies with interpretivist goals, such as those proposed here, are clearly needed to reveal the reasons for such findings. Once we know the reasons why some students are successful in distance education, while others are not, we will have a better foundation for designing more effective web-based learning opportunities for all students.

Selected Bibliography

Ballas, F. A., & Belyk, D. (2000). Student achievement and performance levels in online education research study. Red Deer, AB: Schollie Research & Consulting. Retrieved July 30, 2005 from http://www.ataoc.ca/files/pdf/AOCresearch_full_report.pdf

Barbour, M. K. (2007). Portrait of rural virtual schooling [Electronic Version]. Canadian Journal of Educational Administration and Policy. Retrieved March 31, 2007 from http://www.umanitoba.ca/publications/cjeap/articles/barbour.html.

Barbour, M. K., & Mulcahy, D. (2006). An inquiry into retention and achievement differences in campus based and web based AP courses. Rural Educator, 27(3), 8-12.

Barbour, M. K. (forthcoming) What are they doing and how are they doing it? Student experiences in virtual schooling. Unpublished doctoral dissertation, University of Georgia, Athens, Georgia.

Barker, B. (1985). Curricular offerings in small and large high schools: How broad is the disparity. Journal of Research in Rural Education, 3(1), 35-38.

Barker, K., & Wendel, T. (2001). e-Learning: Studying Canada's virtual secondary schools. Kelowna, BC: Society for the Advancement of Excellence in Education. Retrieved July 31, 2005 from http://www.saee.ca/pdfs/006.pdf

Benson, G. (1998). Opening opportunities by closing the distance. High School Magazine, 6(1), 42-43.

Bernard, R. M., Abrami, P.C., Lou, Y., Borokhovski, E., Wade, A., Wozney, L., Wallet, P.A., Fiset, M. & Huang, B. (2004). How does distance education compare with classroom instruction? A meta-analysis of the empirical literature. Review of Educational Research, 74(3), 379-439.

Bigbie, C., & McCarroll, W. (2000). The Florida high school evaluation 1999-2000 report. Tallahassee, FL: Florida State University. Retrieved February 25, 2007 from http://www.flvs.net/educators/documents/pdf/archived_evals /FLVS%20Annual%20Evaluations/99-2000/99-2000%20Year%20End%20Evaluation.pdf

Brown, J., Sheppard, B., & Stevens, K. (2000). Effective schooling in a telelearning environment. St. John's, NL: Centre for TeleLearning and Rural Education. Retrieved July 13, 2004 from http://www.tellearn.mun.ca/es_report/fright.html

Cavanaugh, C. (2001). The effectiveness of interactive distance education technologies in K-12 learning: A meta-analysis. International Journal of Educational Telecommunications, 7(1), 73-88.

Cavanaugh, C., Gillan, K. J., Bosnick, J., Hess, M., & Scott, H. (2005). Succeeding at the gateway: Secondary algebra learning in the virtual school. Jacksonville, FL: University of North Florida.

Cavanaugh, C., Gillan, K. J., Kromrey, J., Hess, M., & Blomeyer, R. (2004). The effects of distance education on K-12 student outcomes: A meta-analysis. Retrieved July 4, 2005, from http://www.noel.org/tech/distance/kl2distance.pdf.

Clark, R. (1983). Reconsidering research on learning from media. Review of Educational Research, 53(4), 445-459.

Collins, T. (1999). Attracting and retaining teachers in rural areas (Publication no. ED438152). from Office of Educational Research and Improvement (ED): ERIC Document Reproduction Services.

Cosby, A. G., & McDermott, V. P. (1978). Rural youth in context. In A. G. Cosby & I. Charner (Eds.), Education and work in rural America: The social context of early career decision and achievement (pp. 1-18). College Station, TX: Texas A&M University.

Crocker, R. (1989). Towards an achieving society. St. John's, NL: Queen's Printing for Newfoundland and Labrador.

Crocker, R., & Riggs, F. T. (1979). Improving the quality of education: Challenge and opportunity. St. John's, NL: Queen's Printing for Newfoundland and Labrador.

Espinoza, C., Dove, T., Zucker, A., & Kozma, R. (1999). An evaluation of the Virtual High School after two years in operation. Arlington, VA: SRI International. Retrieved July 31, 2005 from http://ctl.sri.com/publications/downloads/evalvhs2yrs.pdf

Friend, B., & Johnston, S. (2005). Florida Virtual School: A choice for all students. In Z. L. Berge & T. Clark (Eds.), Virtual schools: Planning for success (pp. 97-117). New York, NY: Teachers College Press.

Furey, D., & Murphy, E. (2005). K-12 virtual schools and schooling [Electronic Version]. The Morning Watch, 32. Retrieved November 11, 2006 from http: //www.mun.ca/educ/faculty/mwatch/win05/MurphyFurey.htm.

Government of New York. (1992). Rural education: Issues and strategies. Albany, NY: State University of New York.

Government of Newfoundland. (1990). Education Statistics: Elementary-Secondary. St. John's, NL: Queen's Printing for Newfoundland and Labrador.

Government of Newfoundland and Labrador. (2005). Education Statistics: Elementary-Secondary. St. John's, NL: Queen's Printing for Newfoundland and Labrador.

Haughey, M., & Muirhead, W. (2004). Managing virtual schools: The Canadian experience. In C. Cavanaugh (Ed.), Development and management of virtual schools: Issues and trends (pp. 50-67). Hersey, PA: Idea Group, Inc.

House, D. (1986). Education for self-reliance: A report on education and training in Newfoundland. St. John's, NL: Queen's Printing for Newfoundland and Labrador.

Kannapel, P. J., & DeYoung, A. J. (1999). The rural school problem in 1999: A review and critique of the literature. Journal of Research in Rural Education, 1 S(2), 67-79.

Kozma, R., Zucker, A., Espinoza, C., McGhee, R., Yarnall, L., Zalles, D., et al. (2000). The online course experience: Evaluation of the Virtual High School's third year of implementation, 1999-2000. Arlington, VA: SRI International. Retrieved July 31, 2005 from http://ctl.sri.com/publications/downloads/VHS_Online_Experience.pdf

Machtmes, K., & Asher, J. W. (2000). A meta-analysis of the effectiveness of telecourses in distance education. The American Journal of Distance Education, 14(1), 27-46.

McLeod, S., Hughes, J. E., Brown, R., Choi, J., & Maeda, Y. (2005). Algebra achievement in virtual and traditional schools. Learning Point Associates.

Mills, S. C. (2003, March 24-29). Implementing online secondary education: An evaluation of a virtual high school. Paper presented at the Society for Information Technology and Teacher Education, Albuquerque, NM.

Mulcahy, D. M. (2002). Re-conceptualizing distance education: Implications for the rural schools of Newfoundland and Labrador. The Morning Watch, 30(1-2). Retrieved July 31, 2005 from http://www.mun.ca/educ/faculty/mwatch/fa1102/Mulcahy.htm

Oblender, T. E. (2002). A hybrid course model-One solution to the high online drop-out rate Learning & Leading with Technology, 29(6), 42-46.

Pape, L., Adams, R., & Ribeiro, C. (2005). The Virtual High School: Collaboration and online professional development. In Z. L. Berge & T. Clark (Eds.), Virtual schools: Planning for success (pp. 118-132). New York, NY: Teachers College Press.

Reeves, T. C. (2005). No significant differences revisited: A historical perspective on the research informing contemporary online learning. In G. Kearsley (Ed.), Online learning: Personal reflections on the transformation of education (pp. 299-308). Englewood Cliffs, NJ: Educational Technology Publications.

Rice, K. L. (2006). A comprehensive look at distance education in the K-12 context. Journal of Research on Technology in Education, 38(4), 425-448.

Riggs, F. T. (1987). Report of the Small Schools Study Project. St. John's, NL: Queen's Printing for Newfoundland and Labrador.

Roblyer, M. D. (1999). Is choice important in distance learning? A

study of student motives for taking Internet-based courses at the high school and community college levels. Journal of Research on Computing in Education, 32(1).

Roblyer, M. D. (2005). Who plays well in the virtual sandbox? Characteristics of successful online students and teachers. SIGTeI Bulletin(2).

Roblyer, M. D., & Elbaum, B. (2000). Virtual learning? Research on virtual high schools Learning & Leading with Technology, 27(4), 58-61.

Russell, T. L. (1997). The "no significant difference " phenomenon as reported in 248 research reports, summaries, and papers (4 ed.). Raleigh, NC: North Carolina State University.

Setzer, J. C., & Lewis, L. (2005). Distance education courses for public elementary and secondary school students: 2002-03 Washington, DC: National Center for Education Statistics.

Sparkes, R., & Williams, L. (2000). Supporting learning: Ministerial panel on educational delivery in the classroom. St. John's, NL: Queen's Printing for Newfoundland and Labrador.

Storey, V. J. (1993). Factors, issues, and problems in the recruitment and retention of teachers for rural schools. Journal of Research in Rural Education, 9(3), 160-169.

Ungerleider, C. S., & Burns, T. C. (2003). A systematic review of the effectiveness and efficiency networked ICT in education: A state of the field repot to the Council of Ministers of Education, Canada and Industry Canada. Retrieved October 26, 2005 from http://www.cmec.ca/stats/SystematicReview2003.en.pdf

Watkins, T. (2005). Exploring e-learning reforms for Michigan: The new educational (r)evolution. Detroit, MI: Wayne State University.

Wigent, C., & Oswalt, M. (2004). Virtual learning inventory survey executive summary. Lansing, MI: Michigan Association of Intermediate School Administrators.

Williams, L. (1993). Our children, our future: Report of the royal commission of inquiry into the delivery of program and services in primary, elementary, secondary education. St. John's, NL: Queen's Printing for Newfoundland and Labrador.

Zucker, A., & Kozma, R. (2003). The Virtual High School: Teaching generation V New York, NY: Teachers College Press.

Michael K. Barbour

Wayne State University

Dennis Mulcahy

Memorial University of Newfoundland

(4) This term was first used by the Minister of Education in March 1999, when 93 schools were designated as necessarily existent or small schools. There are now only 80 necessarily existent or small schools for the 2006-07 school year, as school construction in strategic locations has meant that some of these schools were no longer too far away from another school to be closed (H. May, personal communication, October 30, 2006).

(5) The "no significant differences" problem that usually occurs when innovative educational technologies are compared with traditional approaches has been well documented by Clark (1983), Reeves (2005) and Russell (1997) among others.

(6) Urban area include Census Metropolitan Areas, Census Agglomerations and other communities 5,000 and over (Government of Newfoundland, 2002).

Table 1. Students' final course averages based upon delivery
model and location by year

 2001-02 2002-03 2003-04

Web delivered rural 71.3 68.1 69.3
 (n = 291) (n = 886) (n = 1,143)

Web delivered urban 64.2 56.5 67.5
 (n = 12) (n = 20) (n = 10)

Classroom delivered 68.2 68.1 68.5
rural (n = 11,233) (n = 21,334) (n = 26,601)

Classroom delivered 67.1 66.6 67.8
urban (n = 13,390) (n = 27,227) (n = 35,555)

# of missing cases 259 (1%) 464 (1%) 1366 (2%)

Total # of cases 25,185 49,931 64,675

# of courses 11 21 24

 2004-05

Web delivered rural 69.6
 (n = 1,132)

Web delivered urban 71.8
 (n = 39)

Classroom delivered 69
rural (n = 31.022)

Classroom delivered 68.5
urban (n = 38,857)

# of missing cases 3693 (5%)

Total # of cases 74,743

# of courses 30

Table 2. Courses offered by the CDLI by year

Courses 2001-02 2002-03 2003-04 2004-O5

Academic Mathematics 2204 X X X X
Academic Mathematics 3103 X X X
Academic Mathematics 3204 * X X X
Advanced Mathematics 2205 X X X X
Advanced Mathematics 3205 * X X X
Advanced Mathematics 3207 X X X
Art and Design 3200 X
Art Technologies 1201 X X X
Biology 2201 X
Canadian Economy 2203 X
Canadian History 1201 X X X X
Career Exploration 1100 X X
Chemistry 2202 X X X X
Chemistry 3202 * X X X
Communications Technology X X X X
 2104
Communications Technology X X X X
 3104
English 1201 X
English 2201 X
English 3201 X
Enterprise Education 3205 X X X
Experiencing Music 2200 X
French 2200 X X X X
French 3200 * X X X
French 3201 X X X
Integrated Systems 1205 X
Mathematics 1204 X X X X
Physics 2204 X X X X
Physics 3204 * X X X
Science 1206 X X X
World Geography 3202 * X X
Writing 2203 X X X X

* denotes course with public examination

Table 3. Students' public exam scores based upon delivery model and
location by year

 2002-03 2003-04 2004-OS

Web delivered rural 61.4 60.5 63.4
 (n = 210) (n = 323) (n = 293)

Web delivered urban 71.0 60.5 66.4
 (n = 1) (n = 2) (n = 8)

Classroom delivered rural 60.6 64.5 61.7
 (n = 3,919) (n = 4,907) (n = 6,558)

Classroom delivered urban 61.4 64.7 62.6
 (n = 5,623) (n = 8,153) (n = 9,304)

# of missing cases 40 (0.5%) 189 (1%) 800 (5%)

Total # of cases 9,793 13,574 16,963

# of courses with public 5 6 7
exams

Table 4. Students' scores based upon delivery model and location

 Public Exam Final Course Average

Web delivered rural 61.7 69.3
 (n = 826) (n = 3,452)

Web delivered urban 65.7 66.3
 (n = 11) (n = 81)

Web delivered total 61.8 69.2
 (n = 837) (n = 3533)

Classroom delivered 62.3 68.5
rural (n = 15,384) (n = 90,190)

Classroom delivered 63.1 67.7
urban (n = 23,080) (n = 115,029)

Classroom delivered total 62.8 68.1
 (n = 38464) (n = 205219)

# of missing cases 1,029 (2.6%) 5,650 (2.6%)

Total # of cases 40,330 214,402