Integrating ICT into higher education: a study of onsite vs online students' perceptions.

Fillion, Gerard ; Limayem, Moez ; Laferriere, Therese 等

ABSTRACT

For the past two decades, information and communication technologies (ICT) have transformed the ways professors teach and students learn. The purpose of this study is to investigate the perceptions of onsite students (hybrid or blended mode) and of those taking the same courses on the Internet (online mode). To guide the study, a moderator-type theoretical research model was developed, out of which nine hypotheses were formulated. The model was tested in a field experiment. To collect data, we used a multimethod approach, that is, a Web survey involving open- and closed-ended questions. The sample was formed of 313 onsite and online students from eight undergraduate and graduate courses offered at the Faculty of Administration of a large Canadian university. The quantitative data analysis was performed using a structural equation modeling software, that is, Partial Least Squares (PLS); the qualitative data were analyzed following a thematic structure using QSR NVivo. In this paper we present a summary of the quantitative results (closed-ended questions) supported and enriched by the qualitative results of the students (open-ended questions).

INTRODUCTION

For the past two decades information and communication technologies (ICT) have transformed the ways professors teach and students learn. Some professors have actively shifted the information flow from a face-to-face mode (student listening, onsite presence) to an entirely online mode (student reading, onsite non presence); that is, they have designed courses and curricula offered completely online using the Internet and the Web. Others have developed the hybrid or blended mode (a combination of face-to-face and online activities; less student onsite presence, ongoing use of ICT both inside and outside the classroom). Hence, knowledge acquisition and dissemination have been reconceptualized, and new methods developed in order to satisfy the rapidly evolving needs of a population of individuals in search of more knowledge, more and more heterogeneous, in a geographically distributed environment.

In today's global economy, organizations (including universities) who want to survive and strive to stay highly competitive must continually innovate at the human, material, and technological levels. Alavi and Leidner (2001) pointed out that, during the past decade, universities and corporate training facilities have at an increasing rate invested into ICT to improve education and training. Marshall (2002) added that actual classrooms are more and more enriched by technology. Further, Giddens (1999) argued that one of the more important functions of the university is to allow people to play a significant role in today's new economy. Thus, universities, faculties, and professors are currently looking for ways to improve teaching and curricula, as well as develop new modes capable of satisfying the actual and future needs of organizations and societies. Out of their recursive attempts, the four fundamental questions often revisited are the following: (1) What are we teaching? (2) What should we be teaching? (3) What is the best way to teach it (pedagogy)? and (4) What are the impacts on students?

The study described in this paper aims at helping universities to stay highly competitive in the current global shift in higher education, an approach that is innovative in its exploration of new directions as regards the last two above-mentioned questions related to pedagogy and student impact. We examine the relation between students' learning outcomes (undergraduate and graduate students) and learning environments integrating ICT. Specific relations between student onsite presence and student online presence are examined as to identify their effect on the basic relation between learning environments and students' learning outcomes. In particular, this study compares onsite technology-rich hybrid or blended learning environments and online learning environments. Moreover, this study brings to the foreground several moderator variables related to students' characteristics (psychology) and professors' pedagogy in order to better understand the relation between learning environments and students' learning outcomes.

Building on the two last questions raised previously, this innovative study focuses on the following three research questions: (1) Are there differences between learning outcomes of onsite students and of those taking the same courses online? If so, which ones? (2) Do students' characteristics influence the relation between learning environments and students' learning outcomes, and are there differences in this influence between onsite and online students? If so, which ones? and (3) Does professors' pedagogy influence the relation between learning environments and students' learning outcomes, and are there differences in this influence between onsite and online students? If so, which ones?

This paper builds on a framework suggested by Fillion (2004) in the conduct of hypothetico-deductive scientific research in organizational sciences, and it is structured as follows. First, the theoretical background supporting the study is examined; second, the methodology followed to conduct the study is presented; third, the results of the study are reported; and the paper ends with a discussion of the results and recommendations for further research.

THEORETICAL BACKGROUND

This study is theoretically-based on Leidner and Jarvenpaa's, and Phipps and Merisotis' key research works. On the basis of three case studies, Leidner and Jarvenpaa (1993) developed a theoretical research model for other researchers to test in future studies. And, in a literature review, Leidner and Jarvenpaa (1995) inventoried numerous educational variables to be examined in future studies according to different scenarios using ICT. Several of the variables suggested by these authors are used in this study.

In their literature review on distance learning effectiveness in the 1990's, Phipps and Merisotis (1999) pointed out that the studies that compared the distance ICT-based learning environments with conventional learning environments (face-to-face without ICT use) fall into three categories: (1) students' results (performance); (2) students' attitude toward learning in these two types of environments; and (3) students' general satisfaction. We use these three categories as dependent variables in this study.

Of the 8,110 papers over a period of 15 years that were published in the journals and reviews examined, Chin et al. (2003) found only 74 that contained moderator variables. Moreover, several IS dominant theories (e.g., Davis' 1989 Technology Acceptance Model (TAM) and Doll and Torkzadeh's 1991 user participation/involvement model; quoted in Chin et al., 2003, p. 192) as well as the streams of research that have extended these models (e.g., Carswell & Venkatesh, 2002; Davis & Venkatesh, 2004; Hartwick & Barki, 1994; Venkatesh & Davis, 2000; Venkatesh & Speier, 1999; Venkatesh & Speier, 2000; Venkatesh & Johnson, 2002; and Venkatesh et al., 2003) suggest that moderator variables are an important avenue of future development. Furthermore, numerous researchers within the IS field have suggested that models using moderator variables be tested (Anderson, 1985; Doll & Torkzadeh, 1989; Ives & Olson, 1984; McKeen et al., 1994; Sambamurthy & Zmud, 1999; Tait & Vessey, 1988; quoted in Chin et al., 2003, p. 192) as have researchers in other fields (Chin et al., 2003). Hence, most of the variables identified by Leidner and Jarvenpaa (1993, 1995) are used as moderator variables in this study. The resulting theoretical research model is shown in Figure 1.

Figure 1 shows that the theoretical research model that guide the study is articulated around an independent construct, learning environments, a dependent construct, student learning outcomes, as well as two moderator constructs, student characteristics and professor pedagogy. In the next subsections, these constructs and their variables are defined, and the research hypotheses formulated.

[FIGURE 1 OMITTED]

Learning Environments

The construct of learning environments is made up of two variables, that is, student onsite presence or hybrid mode (e.g., the wired classroom and the networked classroom) and student onsite non presence or online mode (e.g., Internet- or Web-based course). In the first learning environment, the wired classroom environment, students come to class each week during the semester and use ICT (computer, e-mail, chat, discussion forum, Web browser, Internet-based software, videoconferencing system, etc.) both inside and outside the classroom. Typically, in the wired classroom, students come to class with their own laptop computer. In the second learning environment, the networked classroom environment, students come to class less often, sporadically or at set times (generally five or six times) during the semester and use the ICT mentioned above for communication and collaborative purposes both inside and outside the classroom. In contrast, in the third learning environment, the Internet- or Web-based environment, students do not come to class during the semester but use the same ICT from their home or other space where they have Internet access. It is, thus, the level of students' onsite presence that primarily distinguishes the three learning environments selected for this study.

Student Learning Outcomes

Student Learning Effectiveness

Student learning effectiveness refers to elements such as increase in critical thinking skills, increase in ability to integrate facts, ability to critically analyse issues, learning to interrelate important topics and ideas, and increase in understanding of basic concepts (Alavi, 1994). On the basis of their extensive literature review of the 1990's on distance learning, Phipps and Merisotis (1999) concluded that students' learning is as effective at a distance as in conventional education ("the no significant difference phenomenon"). Further, in their report on a large study conducted by the Sloan Consortium, involving more than 1,100 US colleges and universities, Allen and Seaman (2004) drew the same conclusion. Thus, the findings of these authors and the fact that, in this study, onsite students were given the same permanent access to ICT as those online (in order to compare their learning) lead us to think that learning will be more effective for onsite students.

H1: Students whose onsite presence is required to take courses (hybrid mode) find learning more effective than those whose onsite presence is not required (online mode).

Student Performance

As all researchers who have examined student performance (e.g., Fillion et al., 1999; Piccoli et al., 2001; and Scheck et al., 1994), this study defines performance by students' grades (assignments and exams). According to Phipps and Merisotis (1999), the bulk of literature of the 1990's came to the conclusion that learning results of students using distance learning technologies were as good as those of students using conventional education. A review of 355 comparative studies carried out by Russell (1999) also showed no significant difference on students' performance between technology-supported environments and conventional ones ("the no significant difference phenomenon"). In addition, Allen and Seaman's (2004) report on a large study conducted by the Sloan Consortium indicated that, in most of these educational institutions, learning results of students taking the courses online were similar or higher than those of face-to-face students. Thus, the findings of these authors and the fact that, in this study, onsite students were given the same permanent access to ICT as those online (in order to compare their performance) lead us to believe that performance will be better for onsite students.

H2: Students whose onsite presence is required to take courses (hybrid mode) perform better than those whose onsite presence is not required (online mode).

Student Satisfaction

Student satisfaction refers to elements specific to the feeling of well-being experienced by students in the course, as much on the technical point of view as on the pedagogical one (Hobbs & Osburn, 1989). In their extensive literature review of the 1990's on distance learning, Phipps and Merisotis (1999) observed that most studies inventoried came to the conclusion that distance learning using ICT compared favorably to conventional education and showed a high level of student satisfaction ("the no significant difference phenomenon"). Furthermore, in a recent report on a large study conducted by the Sloan Consortium, Allen and Seaman (2004) pointed out that most of these educational institutions claimed that students taking online courses were equally satisfied as their peers taking face-to-face courses. Thus, the findings of these authors and the fact that, in this study, onsite students were given the same permanent access to ICT as those online (in order to compare their satisfaction) lead us to think that satisfaction will be higher for onsite students.

H3: Students whose onsite presence is required to take courses (hybrid mode) are more satisfied than those whose onsite presence is not required (online mode).

Student Characteristics

Student Autonomy

Student autonomy refers to elements such as the development of good study habits, time management skills, autonomous work habits, a great sense of personal responsibility (Wilson, 1990), initiative and judgement in carrying out the work, and independence and freedom in how the work gets done (Hackman & Oldham, 1975). Hiltz and Turoff (1997) noted that the networked classroom allowed an increase in student autonomy (students can choose the time, place, and pace of their learning). Similarly, Urban-Lurain and Weinshank (2000) observed an increase in student autonomy in the wired classroom. Student autonomy has not been extensively studied in the three learning environments taken into account here. And, to our knowledge, it has not been studied as a moderator variable. Thus, we believe that students' autonomy will have an influence on the relation between learning environments and their learning outcomes, and that this influence will be more pronounced for students taking the courses online.

H4: Students' autonomy has an influence on the relation between learning environments (students' onsite presence and non presence) and their learning outcomes ((a) learning effectiveness; (b) performance; and (c) satisfaction), and (d) this influence is more pronounced for students whose onsite presence is not required.

Student Anxiety

Anxiety is a pervasive emotion frequently experienced by students (Fraser et al., 1983). The term anxiety may be taken simply to mean the experience of dread and foreboding based on some diffuse or specific expectation of harm rather than on an obvious external threat (Sieber et al., 1977). In their study involving 116 classes of undergraduate students, Fraser et al. (1983) concluded that the weakest levels of anxiety were found in classes characterized by, among others, greater student participation, clarity of rules to follow, and less control on the part of the professors. But their study was conducted in conventional environments (face-to-face without ICT use). On the other hand, Jegede and Kirkwood's (1992) study in a distance learning context indicated that students experienced a very high level of anxiety and were more anxious concerning their studies at the end of the semester than at the beginning. In the context of student autonomy above, student anxiety has not been extensively studied in the three learning environments taken into account in this study. And, to our knowledge, it has not been studied as a moderator variable. Thus, we are led to think that students' anxiety will have an influence on the relation between learning environments and their learning outcomes, and that this influence will be more pronounced for students taking the courses online.

H5: Students' anxiety has an influence on the relation between learning environments (students' onsite presence and non presence) and their learning outcomes ((a) learning effectiveness; (b) performance; and (c) satisfaction), and (d) this influence is more pronounced for students whose onsite presence is not required.

Student Motivation

Motivation is defined as student interest in the course and the work invested to prepare for it. In the eight-phase learning process proposed by Gagne (1975), motivation is a highly important factor, perhaps the most important. On the basis of their first experiences in the networked classroom, Riel (1993) and Harasim et al. (1995) reported an increase in student motivation. Hiltz and Wellman's (1997) study also showed an increase in students' motivation in the same type of environment. Blyth (2000) observed an increase in student motivation, but in the Internet and the Web environment. In the context of student anxiety and autonomy above, student motivation has not been extensively studied in the three learning environments taken into account in this study. And, to our knowledge, it has not been studied as a moderator variable. Thus, we believe that students' motivation will have an influence on the relation between learning environments and their learning outcomes, and that this influence will be more pronounced for students taking the courses online.

H6: Students' motivation has an influence on the relation between learning environments (students' onsite presence and non presence) and their learning outcomes ((a) learning effectiveness; (b) performance; and (c) satisfaction), and (d) this influence is more pronounced for students whose onsite presence is not required.

Student Participation

A student who is participating in the course is viewed as making suggestions, showing attention and interest, providing information to other students, and asking other students for their thoughts or opinions (Green & Taber, 1980). Students' involvement and participation are seen as being essential in several studies that look into different distance learning environments (Alavi et al., 1995; Leidner & Jarvenpaa, 1993; Webster & Hackley, 1997). In the networked classroom environment, Hiltz (1990) as well as Hiltz and Wellman (1997) noted an increase in student participation. In the Internet and the Web environment, Arbaugh's (2000a, 2000b) studies indicated a higher participation of females than males. And two recent studies showed a significant influence of students' participation on their learning outcomes (Rovai & Barnum, 2003; Webb et al., 2004). Nevertheless, similar to student anxiety, autonomy, and motivation above, student participation has not been extensively studied in the three learning environments taken into account in this study. And, to our knowledge, it has not been studied as a moderator variable. Thus, we are led to think that students' participation will have an influence on the relation between learning environments and their learning outcomes, and that this influence will be more pronounced for students taking the courses online.

H7: Students' participation has an influence on the relation between learning environments (students' onsite presence and non presence) and their learning outcomes ((a) learning effectiveness; (b) performance; and (c) satisfaction), and (d) this influence is more pronounced for students whose onsite presence is not required.

Professor Pedagogy

Type of Professor

Type of professor refers to elements such as professor organization, impartiality in grading, general attitude, knowledge about the subject taught, presentation skills (Hiltz, 1990), use of a variety of learning activities, and use of technology in a competent manner (Thach & Murphy, 1995). Clearly, a professor's attitude has a great influence on students' interest in the course. So it is not very surprising that a professor can accelerate a student's learning rate (Joyce & Weil, 1996). In the Internet and the Web environment, Barnes et al. (1999) reported a clear improvement of teaching success when institutional support was significant and professors were completely involved in this type of course. However, their study was not conducted in a course entirely taught online, but in a course enriched by the use of the Internet and the Web. Obviously, type of professor has not been extensively studied in the three learning environments taken into account in this study. And, to our knowledge, it has not been studied as a moderator variable. Thus, we believe that type of professor will have an influence on the relation between learning environments and students' learning outcomes, and that this influence will be more pronounced for onsite students.

H8: Type of professor has an influence on the relation between learning environments (students' onsite presence and non presence) and students' learning outcomes ((a) learning effectiveness; (b) performance; and (c) satisfaction), and (d) this influence is more pronounced for students whose onsite presence is required.

Teaching Practice

According to Chickering and Gamson (1987), a professor having a good teaching practice encourages contact between students and faculty, develops reciprocity and cooperation among students, uses active learning techniques, provides prompt feedback, emphasizes time on task, communicates high expectations to students, and respects diverse talents and ways of learning of students. Laferriere et al. (1999) pointed out that new learning technologies question some social practices established in higher education, particularly the student file management and professor performance in the classroom. Further, Kozma and Schank (1998) argued that it is essential for education to focus on community and innovative teaching practices, and to be supported by technological resources as they become available. Teh's (1999) study showed an evolution of the professors' innovative ability in their teaching practice. However, this study was not conducted in a course entirely taught online, but in a course enriched by the use of the Internet and the Web. In the networked classroom, Hiltz's (1990) study indicated that students having experienced high levels of communication with other students and their professor judged learning outcomes of virtual courses superior to those of conventional courses. Obviously, teaching practice has not been extensively studied in the three learning environments taken into account in this study. And, to our knowledge, it has not been studied as a moderator variable. Thus, we are led to think that teaching practice will have an influence on the relation between learning environments and students' learning outcomes, and that this influence will be more pronounced for onsite students.

H9: Teaching practice has an influence on the relation between learning environments (students' onsite presence and non presence) and students' learning outcomes ((a) learning effectiveness; (b) performance; and (c) satisfaction), and (d) this influence is more pronounced for students whose onsite presence is required.

In the next section of the paper, we describe the methodology followed to conduct the study.

METHODOLOGY

Sample and Data Collection

The theoretical research model depicted in Figure 1 was tested in a field experiment at the Faculty of Administration of a large Canadian university. The sample was composed of students of five undergraduate and three graduate courses, which were offered at the same university in the two modes taken into account in this study: hybrid mode and online mode. Students were not randomly assigned, that is, for each course selected, the students were asked to participate in the study. The study was spread over two semesters, fall and winter, and in each semester four courses were studied. Each course had to meet the four following criteria: (1) to use a similar set of ICT in the two modes; (2) to be taught by a different professor in the two modes; (3) to have the same course content in the two modes; and (4) to have, as much as possible, a similar group size in the two modes. In addition, each course was selected so that groups of students in the two modes were the most homogeneous possible in terms of age and ICT experience. Finally, the course selection was made in order to cover a large area of disciplines offered at the Faculty of Administration of the university chosen for the study. Thus, the sample of the study consisted of 841 students, 438 (242 in fall and 196 in winter) in hybrid mode courses and 403 (198 in fall and 205 in winter) in online mode courses.

Three weeks before the end of each semester of the data collection, students were asked to fill out an electronic survey on a Web site. To that end, an e-mail, including a URL and a password allowing access to the electronic survey, was sent to students. As follow up, ten days after the students had been asked to fill out the survey on the Web site, an e-mail was sent to students relating the importance of filling out the electronic survey for the advancement of scientific knowledge on integration of ICT into higher education. Finally, a few days later, all professors were asked to relay the importance of the study to students during class or in the discussion forums of the online courses.

In the fall semester, 174 students (113, hybrid mode; 61, online mode) out of 440 completed the electronic survey for a response rate of 40%; in the winter semester, 139 students (70, hybrid mode; 69, online mode) out of 401 completed the electronic survey for a response rate of 35%. Overall, 313 students (183, hybrid mode; 130, online mode) out of 841 completed the electronic survey on the Web site for a global response rate of 37%. And, of these 313 students, 262 (156, hybrid mode; 106, online mode) completed the qualitative section (open-ended questions) of the Web survey for a response rate of 84%.

Data Analysis

The quantitative data analysis was performed using a structural equation modeling software, that is, Partial Least Squares (PLS-Graph 3.0). To ensure the stability of each model developed in order to test the research hypotheses, we used the PLS bootstrap resampling procedure with an iteration of 100 sub-sample extracted from the initial sample (313 students). Some analyses were also performed using the Statistical Package for the Social Sciences software (SPSS 11.5). As for the qualitative data analysis, it was carried out using the Qualitative Solutions & Research NVivo software (QSR NVivo 2.0). We performed thematic analyses on the qualitative data of students; the results are presented on the form of within-case/cross-case matrix as suggested by Miles and Huberman (1994). They follow.

RESULTS

Test of Hypotheses

To test hypotheses involving independent and dependent variables (H1-H3), we developed a PLS model similar to those of Limayem and DeSanctis (2000), Limayem et al. (2002), and Yoo and Alavi (2001). And to test hypotheses involving moderator variables (H4-H9), we developed several PLS models according to the Chin et al.'s (2003) and Carte and Russell's (2003) procedures. We used PLS bootstrap resampling procedure with an iteration of 100 sub-sample extracted from the initial sample (313 students) to ensure the stability of each of these models. Table 1 presents a summary of the test of hypotheses.

As shown in Table 1, onsite students performed better than those online (p < 0.001). On the other hand, online students were more satisfied than those onsite (p < 0.01). As for the moderator variables, autonomy had an influence on the relation between learning environments and student learning effectiveness (p < 0.05), and this influence was more pronounced for online students than for those onsite (p < 0.001). Anxiety and motivation had an influence on the relation between learning environments and student performance (p < 0.01 and p < 0.05, respectively). And participation had an influence on the relation between learning environments and student learning effectiveness (p < 0.001), performance (p < 0.01), and satisfaction (p < 0.01). To summarize, the quantitative data analysis of the study provided very interesting and somewhat surprising results, particularly with regard to students' performance and satisfaction, as well as professors' pedagogy. The results of the qualitative data analysis follow.

Open-Ended Questions

In the first open-ended question of the Web survey students were asked to indicate what they appreciated the most in the course. Table 2 shows the themes extracted from the thematic analysis of the onsite and online students' responses. Boldfaced themes represent the interrelation between onsite and online students' responses.

We can see in Table 2 that the elements most appreciated by both onsite and online students (in order of priority) are professor, course usefulness, course material, ICT use, assignments, access to the course material on the Web site, discussion forums, prompt feedback, student/student and student/professor interaction, course structure, evaluations, nothing, participation, and collaboration. Thus, we can conclude that whether or not students come to class to take courses, when the same set of ICT is used, they appreciate the same elements related to these courses. And, among the elements they appreciate the most, professor and course usefulness in every day life and for their career are by far in the lead. Clearly, professors again take a predominant place in the formation of students at the beginning of the 21st century.

In the second question, students were asked to suggest ways of improving the course. The themes derived from the thematic analysis of the onsite and online students' responses are presented in Table 3. Boldfaced themes represent the interrelation between onsite and online students' responses.

The results in Table 3 show that the elements the students want improved in the course (in order of priority) are professor, presentation of the material, course material, assignments, amount of work, course content, nothing, evaluations, student/student and student/professor interaction, discussion forums, and WebCT use. Thus, we can conclude that whether or not the students come to class to take courses, when the same set of ICT is used, generally both sets of students suggest improving the same elements related to these courses. And, of the elements proposed, professor and presentation of the material are by far in the lead. As a result, whether the students take courses onsite or online, they place crucial importance on the professor and his/her teaching practice, as much to appreciate them when they are satisfied (as we have seen in the analysis of the first open-ended question previously in Table 2) as to criticize them when they are dissatisfied (as see in Table 3).

The third open-ended question of the Web survey asked students whether the onsite presence provided benefits to them with the integration of ICT into higher education, and why? The themes extracted from the thematic analysis of the onsite and online students' responses are regrouped in Table 4. Boldfaced themes represent the interrelation between onsite and online students' responses.

As shown in Table 4, students' responses to this question are regrouped in three categories: advantageous, non advantageous, and more or less advantageous. In the first category, the two themes that are by far in the lead are that onsite presence allows a better understanding of the material and promotes student/student and student/professor interaction. As for the second category, the two themes that are most evident are that the students can learn as well at home with a book and that ICT allow students to take courses at a distance without onsite presence. For the third category, there is no interrelation between onsite and online students' responses.

In the fourth open-ended question of the survey, students were asked to indicate the impacts of students' characteristics (autonomy, anxiety, motivation, and participation) into higher education integrating ICT. Table 5 shows the themes extracted from the thematic analysis of the onsite and online students' responses. Boldfaced themes represent the interrelation between onsite and online students' responses.

As shown in Table 5, the three impacts that have been by far the most important for students are that ICT use at the university increases the level of autonomy and motivation, and that the students' characteristics (autonomy, anxiety, motivation, and participation) taken into account in this study have an influence on their learning outcomes. The two next most important impacts for students of the two modes are that ICT use at university increases their level of anxiety and participation.

Finally, in the fifth and last open-ended question of the Web survey, students were asked to indicate the impacts of professors' pedagogy (type of professor and teaching practice) into higher education integrating ICT. The themes derived from the thematic analysis of the onsite and online students' responses are regrouped in Table 6. Boldfaced themes represent the interrelation between onsite and online students' responses.

Table 6 shows that the four impacts that have been by far the most important for students are: when we use ICT at the university, professors must be dynamic to keep students' interest, they must make good use of ICT to bring motivation to students, use active learning techniques, and be there for students. We can see here that these impacts related to professors and their teaching practices (the two variables taken into account in this study to assess the quality of professors' pedagogy) are of crucial importance to students. And the next most important impacts for students of the two modes are that, when professors are using ICT at the university, they must have a well organized course and get more familiarized with ICT. The last section of the paper is devoted to a discussion of the findings..

DISCUSSION

Comparison of the Research Findings with Existing Theories

First, with respect to student learning effectiveness, our findings provide support for the conclusions drawn by Allen and Seaman (2004), and Phipps and Merisotis (1999). In fact, the results of our study suggest that, even with the addition of the permanent use of ICT in conventional environments, students' learning is as effective online as in the classroom ("the no significant difference phenomenon").

On the other hand, concerning student performance, our findings are in opposition to those of Ahmed (2000; quoted in Alavi & Leidner, 2001, p. 5), Allen and Seaman (2004), Phipps and Merisotis (1999), Russell (1999), and van Schaik et al. (2003), who concluded that students' performance is as good at a distance as in conventional education ("the no significant difference phenomenon"). Moreover, our results are in opposition to those of Matthews (2000), Ricketts et al. (2000) and Vigilante (2000), who indicated an improvement in students' performance in the Internet and the Web environment compared to the conventional environment. In short, according to the results of our study, with the addition of the permanent use of ICT into conventional learning environments, onsite students performed better than their peers taking the courses online. And assignment grades made all the difference. Clearly, this is a very surprising result, one which will require further investigation in future studies.

With respect to student satisfaction, similar to student performance above, our findings are in opposition to those of Allen and Seaman (2004), and Phipps and Merisotis (1999), who concluded that students taking the courses at a distance are as satisfied as those in conventional education ("the no significant difference phenomenon"). In fact, in our study, even with the addition of the permanent use of ICT into conventional learning environments, online students were more satisfied than those onsite. So here is another very surprising result which will require further investigation.

Let us now examine the results of the verification of hypotheses involving moderator variables. Our findings show a significant influence of student autonomy on the relation between learning environments and student learning effectiveness, and this influence is more pronounced for online students than for those onsite. Thus, if we add the permanent use of ICT into conventional learning environments, as in our study, given that online students were more autonomous than those onsite, our results provide support for what Bilodeau (1995) stressed, that is, students at a distance are less dependent on their professor and then become more autonomous. On the other hand, our results seem not to be in accordance with the conclusion drawn by Hiltz and Turoff (1997), as well as Urban-Lurain and Weinshank (2000), which indicated an increase in students' autonomy in the networked classroom and the wired classroom environments, respectively, compared to the conventional environment.

Past research has shown that students experience moderate to high levels of anxiety in courses, as much in conventional environments as in online ones. In this study, we found that the level of anxiety was very low both for onsite and online students. Here is another very surprising result which will require further investigation in future studies. In addition, our findings suggest that anxiety has a significant influence on the relation between learning environments and student performance, and that this influence is more pronounced for students taking the courses in the classrooms than for those online. So if we add the permanent use of ICT into conventional learning environments, our research results provide support for what Harasim (1987a, 1987b; quoted in Harasim et al., 1995, p. 15) and Hiltz and Turoff (1997) said, that is, networked classrooms might bring anxiety into communication. On the other hand, they are in opposition to those observed by Cowan and Piepgrass (1997), and Hembree (1988; quoted in Cowan & Piepgrass, 1997, p. 105) into conventional environments, where the researchers suggested that anxiety has harmful effects on students' performance (the more anxious students performed better than the others here), and also in opposition to those of the study carried out by Jegede and Kirkwood (1992) in a distance learning environment, which indicated that students experienced a high level of anxiety and were more anxious at the end of the semester than at the beginning.

In our study, we found that student motivation has a significant influence on the relation between learning environments and student performance, and that this influence is more pronounced for onsite students than for those online. Thus, when adding the permanent use of ICT into conventional learning environments, and given that onsite students were more motivated than their peers taking the courses online, our results provide support for the conclusion drawn by Riel (1993), Harasim et al. (1995) as well as Hiltz and Wellman (1997) indicating an increase in students' motivation in the networked classroom environment compared to the conventional environment. On the other hand, our findings seem to be in opposition with the conclusion of the studies conducted by Barron and Orwig (1997), and Blyth (2000) that pointed to an increase in students' motivation in the Internet and the Web environment compared to the conventional environment.

Previous studies have argued that participation is crucial in distance learning environments (Alavi et al., 1995; Leidner & Jarvenpaa, 1993; Webster & Hackley, 1997). Our findings somewhat challenge these results, as we found that student participation is crucial in both onsite and online environments. Indeed, although we noted relatively weak levels of student participation, participation had a strong influence on the relations between learning environments and student learning effectiveness, student performance, and student satisfaction. And, surprisingly, this influence is more pronounced for students taking courses onsite rather than online. Thus, if we add the permanent use of ICT into conventional learning environments, and given that onsite students participated more than those online, our results provide support for the conclusion of the studies conducted by Hiltz (1990) and Hiltz and Wellman (1997) which indicated an increase in students' participation in the networked classroom environment. On the other hand, our findings are somewhat in opposition to those of Karp and Yoels (1976) who, while following the observation of 10 undergraduate courses, noted that even in small classrooms, only few students participated in the discussions. Clearly, in our study, student participation is the moderator variable that showed having the greater influence on the relations between learning environments and the three dependent variables of our theoretical research model. Consequently, it is an extremely important variable to take into account in future development of courses and curricula, and in future studies.

Finally, the results of our study suggest that type of professor and teaching practice did not have a significant influence on the relations between learning environments and student learning effectiveness, student performance, and student satisfaction. Here again, these are very surprising results. In our view, a fact that may explain these surprising results is that type of professor and teaching practice had such a strong direct (independent) influence on the dependent variables that they did not have a significant indirect one (moderator), at least to a level of significance p = 0.05. These variables require further investigation in future studies.

Limitations

First, the experimental research design (a field experiment) of this study inherits the limits of this research approach: a weak level of control on independent variables and a weak level of internal validity compared to the laboratory experiment. But, inversely, it presents a higher level of external validity as it was conducted into a real life environment instead of a laboratory. In addition, this study was carried out at only one faculty of a higher education institution instead of several faculties. If it would have been conducted in several faculties of several universities, the external validity would have been even higher.

Second, as this study tested a new moderator-type theoretical research model which, to our knowledge, had never been used before, it was necessary to interpret the findings using different perspectives that make sense of one or several independent variables influencing one or several dependent variables. In this study, we used moderator variables that cannot have a direct influence on dependent variables, but rather an indirect one. As a result, we needed to use a different approach to compare the findings with existing theories.

Third, to compare the results of this study with existing theories, we stress that both onsite and online students were using a similar set of ICT. In other words, both learning environments were ICT-supported or technology-rich.

Theoretical and Practical Contributions

From a theoretical point of view, this study provides academic and organizational communities with theoretical foundations which are innovative, interesting, useful to strategic decision-makers to anticipate the future with a greater certainty, and generalizable to other faculties and universities with regard to the impacts of students' onsite presence and non presence on their learning outcomes, as well as to the influence of numerous moderating variables on the relation between highly technological learning environments and students' learning outcomes. This study is also opening the door to the comparison of different ICT-supported or technology-rich learning environments, whereas until now researchers have always compared an ICT-supported learning environment with the conventional learning environment (face-to-face without ICT use). To our knowledge, this study is also the first to explore the impact of several important moderating variables related to students' characteristics (psychology) and professors' pedagogy in order to better understand the relation between learning environments and students' learning outcomes. Hence, it sheds some light on the role of students' characteristics and professors' pedagogy in the students' learning process while they are in ICT-based learning environments. In addition, our new and creative moderator-type theoretical research model might be tested by other researchers in other universities and/or other situations.

From a practical point of view, this study will help educational institutions to develop curricula better adapted to ICT-supported or technology-rich learning environments so that students take full advantage of their learning activities into these new environments. It will also allow decision-makers of educational institutions to target professors likely to be "the best" in these highly technological learning environments or at least to make such that those already teaching in these environments become more aware of the importance of adapting their pedagogy to these new environments and to continually be innovative in the ways of presenting their material to students. Moreover, this study will allow ICT providers to be more proactive in the design of these new technology-rich learning environments in choosing "the best" technologies to support them.

CONCLUSION

The purpose of this study was to investigate the perceptions of onsite students (hybrid or blended mode) and of those taking the same courses on the Internet (online mode). To guide the study, a moderator-type theoretical research model was developed, out of which nine hypotheses were formulated. The model was tested in a field experiment. To collect data, we used a multimethod approach, that is, a Web survey involving open- and closed-ended questions. The sample was formed of 313 onsite and online students from eight undergraduate and graduate courses offered at the Faculty of Administration of a large Canadian university. The quantitative data analysis was performed using a structural equation modeling software, that is, PLS; the qualitative data were analyzed following a thematic structure using QSR NVivo.

The results indicate that onsite students have not found learning more effective than their peers taking the same courses online. Onsite students performed better than those online. Online students were more satisfied than onsite students. As regards to students' characteristics, students' autonomy had an influence on the relation between learning environments (hybrid mode and online mode) and the effectiveness of their learning, and this influence was more pronounced for online students. Students' anxiety and motivation had an influence on the relation between learning environments and their performance, and this influence was more pronounced for onsite students. And students' participation had an influence on the relation between learning environments and the effectiveness of their learning, their performance, and their satisfaction, and this influence was more pronounced for onsite students.

As for the qualitative results, grosso modo, they are the following: the elements the most appreciated by students were professor and course usefulness; the elements that the students suggest improving are professor and presentation of the material; students' onsite presence is again advantageous when using ICT as it allows a better understanding of the material and promotes student/student and student/professor interaction; ICT use at the university increases the level of autonomy and motivation in students, and students' characteristics (autonomy, anxiety, motivation, and participation) have an influence on their learning outcomes; and when using ICT at the university, professors must be dynamic to keep students' interest, make a good use of ICT to bring motivation to students, use active learning techniques, and be there for students.

Finally, much more research will be needed as technology-rich environments unfold. Better understanding of their impacts on students, professors, and educational institutions will be required in order to improve them or design new ones still better adapted to higher education students. We will continue to inquire into this exciting innovative field.

ACKNOWLEDGMENTS

The authors would sincerely like to thank professors Robert Bracewell (McGill University), Lucio Teles (Simon Fraser University), and Claude Banville (Laval University), for their helpful comments to this study. Thanks also to professor Wynne W. Chin (University of Houston at Texas) who kindly offered to us a license of the last version of his structural equation modeling software PLS to perform the data analysis of this study. Finally, we are grateful to the Fonds pour la Formation de Chercheurs et l'Aide a la Recherche (FCAR) for its financial contribution to this study.

REFERENCES

Ahmed, R. (2000). Effectiveness of Web-based virtual learning environments in business education: Focusing on basic skills training for information technology. Unpublished Doctoral Dissertation, Louisiana State University.

Alavi, M. (1994). Computer-mediated collaborative learning: An empirical evaluation. MIS Quarterly, 18(2), 159-174.

Alavi, M., B.C. Wheeler & J.S. Valacich (1995). Using IT to reengineer business education: An exploratory investigation of collaborative telelearning. MIS Quarterly, 19(3), 293-312.

Alavi, M. & D.E. Leidner (2001). Research commentary: Technology-mediated learning--A call for greater depth and breadth of research. Information Systems Research, 12(1), 1-10.

Allen, I.E. & J. Seaman (2004). Entering the mainstream: The quality and extent of online education in the United States, 2003 and 2004. Sloan Center for Online Education, MA. Retrieved April 30, 2006, from http://www.sloan-c.org.

Anderson, E.E. (1985). Managerial considerations in participative design of MIS/DSS. Information and Management, 9(4), 201-208.

Arbaugh, J.B. (2000a). Virtual classroom versus physical classroom: An exploratory study of class discussion patterns and student learning in an asynchronous Internet-based MBA course. Journal of Management Education, 24(2), 213-233.

Arbaugh, J.B. (2000b). An exploratory study of gender on student learning and class participation in an Internet-based MBA course. Management Learning, 31(4), 503-519.

Barnes, D.M., J.-P. Sims & W. Jamison (1999). Use of Internet-based resources to support an introductory animal and poultry science course. Journal of Animal Science, 77(5), 1306-1313.

Barron, A.E. & G.W. Orwig (1997). New technologies for education: A beginner's guide. Englewood Cliffs, CO: Libraries Unlimited.

Bilodeau, H. (1995). Enseignement a distance a l'aide de tableaux electroniques. Report prepared for the Universite du Quebec en Abitibi-Temiscamingue.

Blyth, C.S. (2000). Enhancing the study of foreign language with technology: A CD-ROM and companion Web site for first-year french. In D.G. Brown (Ed.), Interactive learning: Vignettes from America's most wired campus (pp. 258-260). Bolton, MA: Anker Publishing Company, Inc.

Carswell, A.D. & V. Venkatesh (2002). Learner outcomes in an asynchronous distance education environment. International Journal of Human-Computer Studies, 56, 475-494.

Carte, T.A. & C.J. Russell (2003). In pursuit of moderation: Nine common errors and their solutions. MIS Quarterly, 27(3), 479-501.

Chickering, A.W. & Z.F. Gamson (1987). Seven principles for good practice in higher education., AAHE Bulletin, 39(7), 3-7.

Chin, W.W., B.L. Marcolin & P.R. Newsted (2003). A partial least squares latent variable modeling approach for measuring interaction effects: Results from a Monte Carlo simulation study and an electronic-mail emotion/adoption study. Information Systems Research, 14(2), 189-217.

Cowan, M.M. & K.W. Piepgrass (1997). The first day of class on a two-year campus: What students really want us to know about their science classroom experiences. Journal of College Science Teaching, 27(2), 104-106.

Davis, F.D. (1989). Perceived usefulness, perceived ease of use and user acceptance of information technology. MIS Quarterly, 13(3), 319-340.

Davis, F.D. & V. Venkatesh (2004). Toward preprototype user acceptance testing of new information systems: Implications for software project management. IEEE Transactions on Engineering Management, 51(1), 31-46.

Doll, W.J. & G. Torkzadeh (1989). A discrepancy model of end-user computing involvement. Management Science, 35(10), 1151-1171.

Doll, W.J. & G. Torkzadeh (1991). The measurement of end-user computing satisfaction: Theoretical and methodological issues. MIS Quarterly, 15(1), 5-10.

Fillion, G. (2004). Publishing in the organizational sciences: An extended literature review on the dimensions and elements of an hypothetico-deductive scientific research, and some guidelines on "how" and "when" they should be integrated. Academy of Information and Management Sciences Journal, 7(1), 81-114.

Fillion, G., M. Limayem & L. Bouchard (1999). Videoconferencing in distance education: A study of student perceptions in the lecture context. Innovations in Education and Training International, 36(4), 302-319.

Fraser, B.J., R. Nash & D.L. Fisher (1983). Anxiety in science classrooms: Its measurement and relationship to classroom environment. Research in Science & Technological Education, 1(2), 201-208.

Gagne, R.M. (1975). Essentials of learning for instruction. New York: Holt, Rinehart and Winston.

Giddens, A. (1999). The third way: The renewal of social democracy. Melden, MA: Polity Press.

Green, S.G. & T.D. Taber (1980). The effects of three social decision schemes on decision group process. Organizational Behavior and Human Performance, 25, 97-106.

Hackman, J. & G. Oldham (1975). Development of the job diagnostic survey. Journal of Applied Psychology, 60, 159-170.

Harasim, L.M. (1987a). Computer-mediated cooperation in education: Group learning networks. Proceedings of the Second Guelph Symposium on Computer Conferencing, June 1-4.

Harasim, L.M. (1987b). Teaching and learning online: Issues in designing computer-mediated graduate courses. Canadian Journal of Educational Communications, 16(2), 117-135.

Harasim, L.M., S.R. Hiltz, L. Teles & M. Turoff (1995). Learning networks: A field guide to teaching and learning online. Cambridge, MA: MIT Press.

Hartwick, J. & H. Barki (1994). Explaining the role of user participation in information system use. Management Science, 40(4), 440-465.

Hembree, R. (1988). Correlates, causes, effects, and treatment of test anxiety. Review of Educational Research, 58, 47-77.

Hiltz, S.R. (1990). Evaluating the virtual classroom. In L.M. Harasim (Ed.), Online education: Perspectives on a new environment (pp. 133-183). New York: Praeger Publishers.

Hiltz, S.R. & M. Turoff (1997). Teaching in a virtual classroom. Collection of 81 PowerPoint slides used to structure an online course. Retrieved April 30, 2006, from http://eies.njit.edu/~hiltz/CRProject/VC/VCtraining/ index.htm.

Hiltz, S.R. & B. Wellman (1997). Asynchronous learning networks as a virtual classroom. Communications of the ACM, 40(9), 44-49.

Hobbs, V.M. & D.D. Osburn (1989). Distance learning evaluation study report II: A study of North Dakota and Missouri schools implementing German I by satellite. ERIC ED 317 195.

Ives, B. & M. Olson (1984). User involvement and MIS success: A review of research. Management Science, 30(5), 586-603.

Jegede, O.J. & J. Kirkwood (1992). Student's anxiety in learning through distance education. This report is part of a larger project funded by a University of Southern Queensland Internal Research Grant, Toowoomba, Australia. ERIC ED 360 476.

Joyce, B.R. & M. Weil (1996). Models of teaching (Fifth Edition). Needham Heights, MA: Allyn & Bacon (a Simon & Schuster Company).

Karp, D.A. & W.C. Yoels (1976). The college classroom: Some observations on the meanings of student participation. Sociology and Social Research, 60, 421-439.

Kozma, R.B. & P. Schank (1998). Connecting with the 21st century: Technology in support of educational reform. In C. Dede (Ed.), Learning with technology: 1998 ASCD yearbook (pp. 3-27). Alexandria, VA: Association for Supervision and Curriculum Development.

Laferriere, T., A. Breuleux & M.N. Campos (1999). L'apprentissage en reseau, une realite pedagogique a definir. Paper presented at the AUPEL_UREF Conference entitled Initiatives 99.

Leidner, D.E. & S.L. Jarvenpaa (1993). The information age confronts education: Case studies on electronic classrooms. Information Systems Research, 4(1), 24-54.

Leidner, D.E. & S.L. Jarvenpaa (1995). The use of information technology to enhance management school education: A theoretical view. MIS Quarterly, 19(3), 265-291.

Limayem, M. & G. DeSanctis (2000). Providing decisional guidance for multicriteria decision making in groups. Information Systems Research, 11(4), 386-401.

Limayem, M., P. Banerjee & L. Ma (2002). Impact of GDSS: Opening the black box. Proceedings of the 6th Pacific Asia Conference on Information Systems (PACIS 2002), Tokyo, Japan, Sept. 2-4.

Marshall, J.M. (2002). Learning with technology: Evidence that technology can, and does, support learning. Report prepared for Cable in the Classroom Foundation, San Diego State University, San Diego. Retrieved April 30, 2006, from http://www.medialit.org/reading_room/article545.html.

Matthews, H.R. (2000). Virtual lectures free students to think critically. In D.G. Brown (Ed.), Interactive learning: Vignettes from America's most wired campus (pp. 111-113). Bolton, Ma: Anker Publishing Company, Inc.

McKeen, J.D., T. Guimaraes & J.C. Wetherbe (1994). The relationship between user participation and user satisfaction: An investigation of four contingency factors. MIS Quarterly, 18(4), 427-451.

Miles, M.B. & A.M. Huberman (1994). Qualitative data analysis: An expanded sourcebook (Second Edition). Thousand Oaks, CA: Sage Publications.

Phipps, R. & J. Merisotis (1999). What's the difference? A review of contemporary research on the effectiveness of distance learning in higher education. The Institute for Higher Education Policy, Washington, DC. Retrieved April 30, 2006, from http://www.ihep.org/Pubs/PDF/Difference.pdf.

Piccoli, G., R. Ahmad & B. Ives (2001). Web-based virtual learning environments: A research framework and a preliminary assessment of effectiveness in basic IT skills training. MIS Quarterly, 25(4), 401-426.

Ricketts, J., F.H. Wolfe, E. Norvelle & E.H. Carpenter (2000). Multimedia: Asynchronous distributed education--A review and case study. Social Science Computer Review, 18(2), 132-146.

Riel, M. (1993). The SCANS report and the AT&T learning network: Preparing students for their future. Telecommunications in Education News, 5(1), 10-13.

Rovai, A.P. & K.T. Barnum (2003). Online course effectiveness: An analysis of student interactions and perceptions of learning. Journal of Distance Education, 18(1), 57-73.

Russell, T.L. The no significance difference phenomenon. Office of Instructional Telecommunications, Chapel Hill, NC: North Carolina State University Press.

Sambamurthy, V. & R. Zmud (1999). Arrangements for information technology governance: A theory of multiple contingencies. MIS Quarterly, 23(2), 261-290.

Scheck, C.L., A.J. Kinicki & J.L. Webster (1994). The effect of class size on student performance: Development and assessment of a process model. Journal of Education for Business, 70(2), 104-113.

Sieber, J.E., H.F. O'Neil & S. Tobias (1977). Anxiety, learning and instruction. New York: Wiley.

Tait, P. & I. Vessey (1988). The effect of user involvement on system success: A contingency approach. MIS Quarterly, 12(1), 90-107.

Teh, G.P.L. (1999). Assessing student perceptions of Internet-based online learning environments. Journal of Instructional Media, 26(4), 397-402.

Thach, E.C. & K.L. Murphy (1995). Competencies for distance education professionals. Educational Technology Research & Development, 43(1), 57-79.

Urban-Lurain, M. & D. Weinshank (2000). Computing concepts and competencies. In D.G. Brown (Ed.), Interactive learning: Vignettes from America's most wired campus (pp. 73-74). Bolton, MA: Anker Publishing Company, Inc.

van Schaik, P., P. Barker & S. Beckstrand (2003). A comparison of on-campus and online course delivery methods in Southern Nevada. Innovations in Education and Teaching International, 40(1), 5-15.

Venkatesh, V. & C. Speier (1999). Computer technology training in the workplace: A longitudinal investigation of the effect of mood. Organizational Behavior and Human Decision Processes, 79(1), 1-28.

Venkatesh, V. & F.D. Davis (2000). A theoretical extension of the technology acceptance model: Four longitudinal field studies. Management Science, 46(2), 186-204.

Venkatesh, V. & C. Speier (2000). Creating an effective training environment for enhancing telework. International Journal of Human-Computer Studies, 52, 991-1005.

Venkatesh, V. & P. Johnson (2002). Telecommuting technology implementations: A within- and between-subjects longitudinal field study. Personnel Psychology, 55(3), 661-687.

Venkatesh, V., M.G. Morris, G.B. Davis & F.D. Davis (2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 27(3), 425-478.

Vigilante, R. (2000). Systems analysis: A graduate course. In D.G. Brown (Ed.), Interactive learning: Vignettes from America's most wired campus (pp. 75-77). Bolton, MA: Anker Publishing Company, Inc.

Webb, E., A. Jones, P. Barker & P. van Schaik (2004). Using e-learning dialogues in higher education. Innovations in Education and Teaching International, 41(1), 93-103.

Webster, J. & P. Hackley (1997). Teaching effectiveness in technology-mediated distance learning. Academy of Management Journal, 40(6), 1282-1309.

Wilson, B. (1990). Student's assessment of distance learning. Paper presented at the Annual Meeting of the Midsouth Educational Researcher Association, New Orleans, Louisiana, November. ERIC ED 326 558.

Yoo, Y. & M. Alavi (2001). Media and group cohesion: Relative influences on social presence, task participation, and group consensus. MIS Quarterly, 25(3), 371-390.

Gerard Fillion, University of Moncton Moez Limayem, University of Lausanne Therese Laferriere, Laval University Robert Mantha, Laval University Table 1: Summary of the Test of Hypotheses Hypotheses Results Software (Sig.) H1 Not supported SPSS (0.692) PLS (0.022) H2 Supported SPSS (0.000 ****) PLS (0.228 ****) H3 Not supported (a) SPSS (0.006 **) PLS (0.059 **) H4 (a) Supported PLS (-0.113 *) (b) Not supported PLS (0.056) (c) Not supported PLS (-0.030) (d) Supported SPSS (0.000 ****) H5 (a) Not supported PLS (0.049) (b) Supported PLS (0.121 **) (c) Not supported PLS (0.000) (d) Not supported (a) SPSS (0.000 ****) H6 (a) Not supported PLS (-0.038) (b) Supported PLS (0.085 *) (c) Not supported (b) PLS (-0.049=) (d) Not supported (a) SPSS (0.000 ****) H7 (a) Supported PLS (-0.155 ****) (b) Supported PLS (0.092 **) (c) Supported PLS (-0.116 **) (d) Not supported (a) SPSS (0.000 ****) H8 (a) Not supported PLS (-0.045) (b) Not supported PLS (0.054) (c) Not supported (b) PLS (-0.037=) (d) Not supported (a) SPSS (0.000 ****) H9 (a) Not supported PLS (0.034) (b) Not supported PLS (0.037) (c) Not supported (b) PLS (-0.034=) (d) Not supported (a) SPSS (0.000****) (a) The test is significant, but the result is in opposition with which is formulated in the hypothesis. (b) The hypothesis is not supported given the level of significance of the test is too low (p < 0.10). = p < 0.10; * p < 0.05; ** p < 0.01; **** p < 0.001. Table 2: The Elements Most Appreciated by Students in the Course When Using ICT Onsite Students (n = 156) Themes n Professor 62 Course usefulness 59 Access to the course material on the Web site 19 Course material 16 ICT use 16 Assignments 11 Student/student, student/professor interaction 9 Discussion forums 8 Nothing 4 Course structure 4 Evaluations 4 Participation 3 Help 3 Onsite sporadic presence 2 Prompt feedback 2 Learning 2 Collaboration 1 Freedom of expression 1 Online students (n = 106) Themes n Professor 27 Course flexibility and schedule 24 Course material 20 Prompt feedback 19 Assignments 17 Discussion forums 16 Course usefulness 15 Distance course via the Internet 14 ICT use 13 Access to the course material on the Web site 8 Course structure 6 Student/student, student/professor interaction 4 Evaluations 3 Collaboration 1 Participation 1 Fulfillment 1 Buying some things on the Internet 1 Nothing 1 Table 3: The Elements that the Students Suggest to Improve in the Course When Using ICT Onsite students (n = 156) Themes n Professor 41 Presentation of the material 38 Course content 22 ICT use 21 Amount of work 16 Assignments 13 Student/student, student/professor interaction 8 Evaluations 7 Course material 7 Course organization 7 Classroom 5 Discussion forums 4 Nothing 4 Group size 2 Attribution of the courses to professors 1 Discipline 1 WebCT use 1 Online students (n = 106) Themes n Course material 21 Professor 18 Nothing 15 Presentation of the material 12 Assignments 11 Course structure 10 Evaluations 8 Web site 8 Amount of work 7 Discussion forums 7 Feedback 5 Student/student, student/professor interaction 4 Technical aspects 1 Course content 1 Correction of assignments and exams (corrector) 1 WebCT use 1 Table 4: To What Extent Students' Onsite Presence Is Advantageous When Using ICT Onsite students (n = 156) Themes n Advantageous 4 Allows a better understanding of the material 55 ICT complement the conventional classroom 26 Promotes student/student and student/professor 21 interaction If professor is not just reading PowerPoint slides 11 Allows social contact 9 It depends for which course and type of student 8 No interest without onsite presence 6 Some students need onsite presence to succeed 1 If the network is well functioning 1 Brings a personal satisfaction 1 Non advantageous 2 We can learn as well at home with a book 14 Many students are playing with their laptop 5 without listening to the professor ICT allow taking courses at a distance 3 Professors have some difficulties to use ICT 1 All the material is on the Web site 1 Much waste of time onsite 1 Neutral (more or less advantageous) 1 No time to go to class, too much work 1 Online students (n = 106) Themes n Advantageous 1 Allows a better understanding of the material 25 It depends for which course and type of student 15 Promotes student/student and student/professor 9 interaction If professor is not just reading PowerPoint slides 5 Allows social contact 4 Allows having more informations 4 Some students need onsite presence to succeed 2 Promotes student motivation 2 It depends on undergraduate/graduate course 1 Promotes collaboration between students 1 Onsite and online formation are complementary 1 Non advantageous 11 ICT allow taking courses at a distance 12 We can learn as well at home with a book 8 Much waste of time onsite 4 Some professors do not have a good teaching 1 practice Neutral (more or less advantageous) 1 Table 5: The Impacts of Students' Characteristics When Using ICT Onsite students (n=156) Themes n Autonomy ICT increase autonomy 34 Autonomous students appreciate more distance 1 courses Anxiety ICT increase anxiety 13 ICT decrease anxiety 6 Motivation ICT increase motivation 22 ICT decrease motivation 2 Onsite presence can have an influence on 1 student motivation Participation ICT increase participation 15 ICT decrease participation 9 Others These characteristics have an influence on 34 student learning outcomes ICT use at the university is excellent for the 6 workplace No impact 6 I don't know 3 ICT use brings a certain security 2 Onsite students (n=106) Themes Autonomy ICT increase autonomy 32 Anxiety ICT increase anxiety 12 ICT decrease anxiety 6 Students have no apprehension about ICT 1 Motivation ICT increase motivation 28 ICT decrease motivation 4 Participation ICT increase participation 9 ICT decrease participation 2 Others These characteristics have an influence on 25 student learning outcomes No impact 5 It depends on students 2 ICT use at the university is excellent for the 1 workplace Major impacts 1 I don't know 1 Table 6: The Impacts of Professors' Pedagogy When Using ICT Onsite students (n = 156) Online students (n = 106) Themes n Themes n Type of professor (professor must:) Be dynamic to keep 32 Make good use of ICT to 25 students' interest bring motivation to students Make good use of ICT to 29 Be dynamic to keep 24 bring motivation to students' interest students Get more familiarized 15 Be there for students 22 with ICT Be there for students 9 Have a well organized 16 course Be very engaged 7 Get more familiarized 6 with ICT Have a well organized 6 Be very engaged 1 course Promote ICT use 4 Promote ICT use 1 Be confident 2 Teaching practice (professor must:) Use active learning 27 Use active learning 22 techniques techniques Motivate students 9 Provide prompt feedback 10 Establish links between 2 Motivate students 2 theory and real life Provide prompt feedback 2 Promote student/student 1 and student/professor interaction Others Very important impacts 6 Few impacts 7 ICT use provides students 3 Very important impacts 1 with a good experience Professors' pedagogy has 2 No impact 1 an influence on student learning outcomes Most of the professors do 2 not have a good pedagogy Technology is too present 2 in the courses No impact 2