Creating a Virtual Meeting System (VMS) for public agency employees: a field study of the Illinois Division of oral Health.

Kwon, Ojoung ; Dufner, Donna ; Kwon, Maria 等

INTRODUCTION

A Web-enabled Virtual Meeting System (CyberCollaboratory) can improve productivity and job satisfaction.

Over a decade ago, Huber (1984) stated that there was a perceived need for the means of aiding group processes to compensate for the limitations of human decision-making. At approximately the same time in 1984, Rice and Bair (1984) pointed out the need to improve white-collar productivity. These observations by Huber and Rice and Bair were almost prophetic for American businesses and are even more true today, given the economic climate of deregulation, intense competition, and corporate downsizing.

It is a commonly held belief that group support systems such as the Virtual Meeting System or CyberCollaboratory (presented and discussed on the following pages) offer many exciting opportunities to improve the quality of work and decision-making and, at the same time, increase the productivity of groups. Groups hold the promise of more democratic decision-making, synergy, more objective evaluation, stimulation, and learning, but only if those group performance inefficiencies called "process losses" (McGrath and Hollingshead, 1994) can be prevented (Hoffnam, 1979; Nunakamer et al., 1992).

It is also a commonly held belief that use of group support systems can generate "process gains" such as increased group productivity, effectiveness, creativity, and member satisfaction, since they eliminate the need to coordinate and attend face-to-face-meetings. Group support systems also have the potential to improve communication, learning, timeliness of decisions, and satisfaction with the process and with outcomes (Bostrom, Van Over and Watson, 1990; DeSanctis and Gallupe, 1985, 1987; Dufner, Hiltz and Turoff, 1993; Hiltz, 1988; Hiltz et al., 191; Hiltz and Turoff, 1981, 1982, 1992, 1993; Huber, 1984; Jarvenpaa et al., 1988; Kraemer and King, 1988; Nagasundaram and Bostrom, 1994). Based on these findings, the Director of the Division of Oral Health (DOH) was interested in using the Group Support System, called "CyberCollaboratory" to explore methods for improving productivity by reducing of travel for meetings, as well as by improving the quality of DOH's group decisions.

The Illinois Department of Public Health (IDPH) has functioned with significantly reduced levels of staffing due major budget cuts since 1991 (State Journal Register, 1991). The Division of Oral Health, which is the focus of the study presented here, is a part of the Illinois Department of Public Health, and is located in the Office of Health and Wellness. Consequently, the DOH has suffered from these budget cuts as well. Following the budget cuts, the DOH now has responsibility for serving the entire state of Illinois which covers a geographical area of 55,593 square miles with a staff of eight full-time employees and four consultants as shown in Figure 1.

[FIGURE 1 OMITTED]

The DOH establishes programs designed to prevent and control oral diseases through organized community efforts focused on community water fluoridation, dental sealants, baby bottle tooth decay, school-based fluoride mouth-rinsing, craniofacial anomalies, orofacial injuries, oral health needs assessment and planning, smokeless tobacco, and a variety of oral health education programs designed to meet the oral health needs of groups throughout the state of Illinois. The goal of the DOH is optimal oral health for all of the residents of Illinois.

Currently, the DOH has a central office in Springfield, Illinois, and seven regional offices located through out the state in Chicago, West Chicago, Rockford, Peoria, Champaign, Edwardsville, and Marion. A regional office of the DOH must serve approximately 30 counties, each with an area of about 6,220 square miles. A regional office has the responsibility for providing oral health technical assistance to the local agencies located within the counties of the region. As a result of the budget cuts, four of the seven regional DOH offices (Rockford, Chicago, West Chicago and Marion) do not have permanent staff. Rockford, Chicago, and West Chicago have not had permanent staff for the last five years and the Marion office has been without permanent staff for over two years.

A consultant located in Peoria now serves the Rockford region and must travel a distance of approximately 143 miles to work. A consultant located in Champaign must now travel 123 miles and 135 miles, respectively, to service the Chicago and West Chicago offices. A consultant located in Edwardsville, a distance of 92 miles away, serves the Marion region.

Because of the size of each region, the distance between the offices, and the amount of travel required to service a region, more than 80% of the DOH staff time is consumed simply by traveling and meetings (Jenssen, 1998). The DOH staff would prefer to spend their time doing the tasks they were trained and hired to perform such as providing oral health technical assistance to the local agencies, and providing oral health education and training, rather than in traveling extensive distances to and in coordinating meetings. While the private sector has embraced the use of asynchronous tools similar to the CyberCollaboratory (described below), governmental agencies have lagged behind, often due to lack of funding.

Groupware such as the CyberCollaboratory in the form of Group Decision Support Systems (GDSS) and Distributed Group Support Systems (DGSS), as mentioned above, are thought of as possible tools for the improvement of group processes and decision-making (Desanctis and Gallupe, 1985, 1987; Stasser, 1992; Stasser and Titus, 1985; Hiltz et al., 1991; Dufner, Hiltz and Turoff, 1994) and for improvement in group productivity (Hiltz, 1988).

Some examples of enhanced outcomes include subjective satisfaction (Watson, 1987), more effective problem solving (DeSanctis and Gallupe, 1985), productivity gains which translates into hard dollar savings for an organization (Hiltz, 1988), better decision analysis via facilitated group communication (Turoff and Hiltz, 1982), and model building to facilitate planning and policy making (Nunamaker, Applegate and Konsynski, 1988). In addition, according to Rice et al. (1984), there seems to be greater equality of influence and participation among group members using GDSS for group decision-making.

DEVELOPMENT HISTORY

Work on the Group Decision Support System (GDSS) environment, the preliminary step in the development of the CyberCollaboratory, was begun in 1993. As has been true at most other institutions, the GDSS was initially designed for a face-to-face (same time and same place) decision room meeting environment. The Phase I system functionality is shown in Table 1. The four types of collaborative interaction are also shown in Table 2. The first GDSS, developed over a period of two years, included five tools: Session Manager, Electronic Brainstorming, Idea Organizer, Voting, and Alternatives Evaluator. The software was tested and operated in a computer lab on campus.

Because of the high costs of building a decision room, we moved to the second phase, distributed, as shown in Table 2. The necessary modifications to make the software capable of handling the distributed, synchronous mode of communication were made in 1995. During Phase II using the campus LAN and WAN connections, group members no longer were required to meet in the same place, although they were required to be logged on at the same time for their distributed meeting. The meeting agenda could be distributed via e-mail and members could log in to the system from their computers or from the computer laboratory. Facilitation and meeting coordination were still required.

In 1996, when the design work for the CyberCollaboratory as it exists today was begun, the GDSS was included as an important component of the asynchronous tool kit for group coordination and process control. Integration of the GDSS into the Notes platform provided decision groups with asynchronous GDSS and with the full functionality of a robust CMC (Hiltz and Turoff, 1978) system. The three levels of GDSS shown in Table 1 are presented and discussed in detail in DeSanctis and Gallupe (1987).

THE VIRTUAL MEETING SYSTEM: A CYBERCOLLABORATORY

The term "Collaboratory," defined by William Wulf (1993), is as follows:

"'... center without walls, in which the nation's researchers can perform their research without regard to geographical location-interacting with colleagues, accessing instrumentation, sharing data and computational resource, and accessing information in digital libraries.'" (Kouzes, Meyers, and Wulf, 1996).

For our CyberCollaboratory, we have extended Wulf's definition to include centers without walls specifically designed and built to support team work for practitioners such as the staff of the DOH can experiment with the usefulness of asynchronous team technologies such as group decision-making in the asynchronous mode of communication. The CyberCollaboratory interface is shown in Figure 2.

[FIGURE 2 OMITTED]

Members do not need to attend frequent face-to-face meetings because the CyberCollaboratory is designed to be used in the asynchronous mode of communication as shown in Figure 3 and is accessible using a standard Web browser and a PC or workstation. A group member may login to participate in a "virtual" meeting from the location of his/her choice at the time of the day or night he/she prefers.

[FIGURE 3 OMITTED]

Given the need for group coordination for successful and satisfying group collaboration the CyberCollaboratory needed:

tools (artifacts)
process structures (e.g. agenda)
process support (e.g. group memory, anonymity)
task structure and task support (e.g. GDSS tools, collaborative
document editing tools, project management software, etc.)

Hiltz et al. (1991) discusses three primary methods for coordination of asynchronous groups. These are:

an agenda or some set of directions or instructions
a facilitator or leader
GDSS tools such as those embedded within existing group
systems such a following commercially available systems:
GroupSystems, Electronic Information and Exchange System
2 (EIES2) or Software Aided Meeting Management (SAMM)

Asynchronous groups can also be coordinated through facilitation and through additional software tools such as collaborative document editing. The CyberCollaboratory contains the following software environments:

Group Discussion (Computer mediated conferencing)
Collaborative Document Editing
Chat
Group Decision Support including Electronic Brainstorming, Idea
Organizing and Voting Intelligent Project Management (Project
Management Advisor)

TRADITIONAL MEETINGS PRESENT MANY PROBLEMS FOR GROUP MEMBERS:

Traditional meetings pose many problems for group members, especially if the groups are spread as thinly as are the DOH staff. The limitations of a traditional meeting are listed in Table 3.

Group support systems may provide the following benefits (Turban, 1998):

Bring more information, knowledge, and skills
Support parallel processing of idea generation
Provide instant anonymous voting results
Record all information automatically
Provide organization memory
Motivate members' participation
Give members more satisfaction

The CyberCollaboratory is expected to provide the following additional benefits:

Economic: save time and money by minimizing travel

Availability: does not require its own expensive telecommunication
infrastructure. It is readily available through a phone line.

Scheduling: scheduling a meeting for a group or groups of people is
a very difficult task. Through asynchronous communication and task
control, we could eliminate this problem. Group members can
participate in their meetings from anywhere at any time of their
choice.

THE CYBERCOLLABORATORY ARCHITECTURE

The CyberCollaboratory Architecture is shown in Figure 4. It is web enabled through the use of Domino and built upon the foundation of Lotus Notes, both of which are supported by IBM. This insures a secure and relatively trouble-free and maintenance-free environment.

[FIGURE 4 OMITTED]

DEVELOPMENT OF THE MEASURES OF SATISFACTION AND PRODUCTIVITY FOR ASYNCHRONOUS TECHNOLOGIES

For this research, the survey instruments used to measure the efficiency and effectiveness of asynchronous technologies such as the CyberCollaboratory evolved and were validated over a ten year period through the research efforts of Hiltz and Turoff at the New Jersey Institute of Technology (NJIT) and their graduate students, and the efforts of other researchers at the University of Minnesota (Watson, 1987). In 1998 Dufner and Kwon modified the survey instruments to fit this and other similar studies underway.

The instruments include a Consent Form, a Pre-test Questionnaire (to obtain demographic information), a Training Evaluation Questionnaire (to assess the participants experience of the training to use the environment), a Task Expectations Questionnaire (designed to measure the subjects expectations regarding performance of the forthcoming task), and a Post-test Questionnaire (administered after the task was completed).

THE RESEARCH METHODOLOGY

The DOH group attended a face-to-face (synchronous) training session where the Consent Form and Pretest Questionnaire were administered, and then training was begun. The hands-on training in the use of the CyberCollaboratory followed. The Systems Expectations and Task Expectations questionnaires were administered directly following completion of the training session. The subjects were trained in the use of the entire CyberCollaboratory and were given a paper copy of the user's guide. This user's guide was also emailed to all group members because in its electronic form, the table of contents, and the figures were hyper-linked for easy reference.

The group member with the greatest technical skill and experience was selected to serve as the group's facilitator for performance of the asynchronous task. The facilitator establishes the task within the CyberCollaboratory as a TOPIC, shown in Figure 5, and moves the group through the various stages of the decision-making process by using the tools and agents (such as Brainstorming and Idea Organizing) that are available within the CyberCollaboratory environment.

[FIGURE 5 OMITTED]

After the training session was completed, the subjects were instructed to decide on a suitable task over the next week to ten-day period for performance in the asynchronous mode of communication (distributed in time and place), using the CyberCollaboratory.

After the group had decided on an appropriate task, the subjects using the asynchronous mode of communication as shown in Figure 3 began performance of the task. The Post-Test Questionnaire was administered immediately following completion of performance of the real-world task by the subjects. Analysis of the CyberCollaboratory transcripts and questionnaire data followed collection of the Post-tests. The entire process spanned approximately two years, from January 1998 through November 1999, and included the initial presentation of the ideas to the administration of the DOH, coordination, obtaining appropriate equipment, training, and administration of the questionnaires, and performance of the task by the DOH team. One of the problems that caused to take the entire process so long was obtaining appropriate computers for DOH employees. Even though the CyberCollaboratory can be accessed using a PC with any standard Web browser and Internet access, the DOH had many computers (386) that would have been very slow when using the Web and thus might have served as an intervening variable in the study. Upgrades for these machines were deemed necessary before the study began.

THE TASK

The staff members of the DOH chose to select a color for their midwinter display, and this task was identified within the VMS system as, "What color borders/background for midwinter display?" The facilitator set up the task as a TOPIC within the CyberCollaboratory as shown in Figure 5.

This task worked very well for the group's pilot project. Indeed, responses from the group members indicated that (1) the task was neither too simple nor too difficult; (2) the task required somewhat tightly integrated action from the group to complete; and (3) the task was an important and urgent task but not too critical to the group. The task seemed to be an excellent choice as the first CyberCollaboratory effort because a task of very high urgency or one that was critical to the group might have caused the group to abandon the CyberCollaboratory and use alternative methods of communication. Also, a task that required low integration from the group might have been performed by one individual for the group.

FINDINGS

The sample size (N = 7) is small due to the small size of the agency staff, as discussed above in the introduction. The Pre-test responses reveals that 100% of the participants reported they are "White" and 86% reported that English is their native language. The group consists of six females and one male. This group is mature (no teens or twenties), with a mean age of 45.3 years. All of the subjects have many years of full-time employment experience with a mean of 23.3 years. Their decision-making experience is distributed from "low" to "very high."

The mean responses to the "Communications Medium," the CyberCollaboratory were generally more positive or neutral than negative. The high standard deviations for the variables presented shown in Table 4 is probably based on the relatively wide variance in the group members' previous training and experience with computers, even though all participants reported having good computer access, frequently using various types of computer software, and having easy access to the Internet from their office.

The respondents also reported that the overall system is more good than bad with a mean of 3.0 (scale 1-7, extremely good/extremely bad). On the pre-test, 57% of the participants reported feeling a bit tense when participating in group discussions. On the post-test questionnaire we see a shift away from feeling tense in the direction of feeling less tense with only 43% of the respondents reporting that they get frustrated or get nervous when participating in group meetings. The CyberCollaboratory might help shy people to feel more comfortable when participating in meetings. Table 5 indicates that respondents thought using the system is somewhat easy (the average of the mean numbers is 4.8). Table 6 indicates that the respondents perceived the GDSS tools (i.e. EBS, IO, Voting, etc.) are as somewhat easy to use (the average of the mean numbers is 2.9) and somewhat valuable (the average of the mean numbers is 4.9).

The respondents reported that the time spent for traveling to actual meetings and coordinating meetings was less after having used the CyberCollaboratory. With the CyberCollaboratory, the group reported they gained more time for substantive preparation and for the meetings themselves.

Of the participants, 71% reported that use of the system helped the group save time. 80% of the people who reported that use of the system helped the group save time also indicated they saved between 1-2 hours per week, while 20% reported they saved between 5-10 hours per week.

Only 43% of the participants reported that the system helped reduce travel. This finding may simply reflect the fact that the secretaries in the group are not required to travel for their jobs, or that the task would not have required as much travel as a more critical task. 100% of the people who said 'Yes' to less travel as a result of using the CyberCollaboratory for this project indicated that they saved 1-2 trips per month. Overall they reported spending less time traveling.

CONCLUSION

These very preliminary findings seem to indicate that the CyberCollaboratory is a useful tool for the DOH. Further studies would be needed to fully confirm these findings. However, the results of the pilot study are very encouraging and it is entirely possible that more savings in time and travel could be realized by the agency as members gain more familiarity and comfort with the VMS system.

Asynchronous Groupware such as the CyberCollaboratory seem to hold much promise for group work and decision-making because artificial pressures to closure are greatly reduced (Hoffman, 1976). These systems can support an on-going group decision process where the group's members are distributed both in time and geographically, as is the DOH staff. The Group members can research and analyze alternatives and explore ideas free from the pressures of having to coordinate and attend a face-to-face meetings or, conversely, from having to end meetings prematurely as a result of artificial constraints such as the end of the workday.

The group at the DOH used the CyberCollaboratory to perform a task asynchronously (distributed in space and time) rather than traveling to meet to perform the task as a face-to-face group. Even though the sample size was small (due to the small size of the department after positions were lost due to drastic budget cuts), we gained important insights and extremely encouraging results from the pilot study. Asynchronous technologies such as the CyberCollaboratory can be used to contribute to productivity by reducing the amount of time needed to coordinate and travel to meetings when staff are distributed over great distances.

REFERENCES

Barua, A., R. Chellappa, & A.B. Whinston (1995) Creating a Collaboratory in Cyberspace: Theoretical Foundation and Implementation. Journal of Organizational Computing, 5(4), 417-442.

Biond, A. (1997) Implications that Downsizing Could Have for Public Sector Unions. Journal of Collective Negotiations in the Public Sector, 26(4), 295-301.

Bajjaly, S. (1999) Managing Emerging Information Systems in the Public Sector. Public Productivity & Management Review, 23(1), 40-47.

Boynton, A. C. & R.W. Zmud (1987) Information Technology Planning in the 1990's: Directions for Practice and Research. MIS Quarterly.

Champy, J. (1996) Better Government, Not Necessarily Smaller. Government Executive Magazine, September, 7A-10A.

DeSanctis, G. & R.B. Gallupe (1987) A foundation for the Study of Group Decision Support Systems. Management Science, 33 (5), 589-609.

Dufner, D., O. Kwon & R. Hadidi (1999) Web-CCAT: a Collaborative Learning Environment for Geographically Distributed Information Technology Students and Working Professionals. Communications of The Association of Information Systems, 1(12).

Faucett, A. & B.N. Kleiner (1994) Developments in Performance Measures of Public Programmes. International Journal pf Public Sector Management, 7(3), 63-70.

Fjermestad, J. & S. Hiltz (1998) An Assessment of Group Support Systems Research: Results. Proceedings of the 32nd Annual Hawaii International Conference on System Science (HICSS-32), Los Alamitos, CA: IEEE Computer Society Press, January.

Hammer, M. (1996) The Soul of The New Organization. Government Executive Magazine, September, 2A-6A.

Hiltz, S. (1988) Productivity Enhancement from Computer-Mediated Communication: a Systems Contingency Approach. Communications of the ACM, December, Vol. 31, 1438-1454.

Hiltz, S. & M. Turoff (1978) The Network Nation: Human Communication via Computer, Addison-Wesley.

Hoffman, L.R. (1979) Applying Experimental Research on Group Problem Solving to Organizations. The Journal of Applied Behavioral Science, 15(3), 375-390.

Hornestay, D. (1996) Downsizing Dilemma. Government Executive Magazine, September, 27-30.

Huang, W., K. Wei, B. Bostrom, L. Lim, & R. Watson (1998) Supporting Distributed Team-Building Using GSS: A Dialogue Theory-Based Framework. Proceedings of the 31st Annual Hawaii International conference on System Sciences (HICSS 31), Los Alamitos, CA: IEEE Computer Society Press, January, 98-107.

Illinois Department of Public Health: Division of Oral Health, P.O. #347019, January 1997.

Kouzes, R.T., J.D. Meyers & W.A. Wulf (1996) "Collaboratories: Doing Science on the Internet. Computer 29(8), IEEE Computer Society Press, August, 40-46.

Nunamaker, J.F., R. Briggs, D. Mittleman & P. Balthazard (1997) Lessons from a Dozen Years of Group Support Systems Research: A Discussion of Lab and Findings. Journal of Management Information Systems, (Winter), 163-207.

Nunamaker, J.F., A.R. Dennis, J.S. Valacich, D.R. Vogel, & J.F. George (1991) Electronic Meeting Systems to Support Group Work. Communications of the ACM, 34 (7), 40-61.

Popovich, M.G. (1998) Creating High-performance Government Organizations, (Editor). Alliance for Redesigning Government, Joeeey-Bass Publishers, San Francisco.

Roach, S.S. (1993) A Special Economic Study: Making Technology Work, New York, Morgan Stanley.

Roach, S.S. (1994) A Special Economic Study: The Perils of America's Productivity-Led Recovery, January, New York, Morgan Stanley.

State J. (1991) Edgar: Cut Budget $509 Million. State Journal-Register, Mar 7, 1-2.

State J. (1991) State Crisis Worse Than Expected. State Journal-Register, Dec 18, 1-6.

State J. (1992) Budget Cuts Elude Edgar Administration. State Journal-Register, Jan 15, 1-5.

Trimmer, J.J., C.V. Slyke & C. Conca (1997) Transforming Student Groups into Teams. Proceedings of the Association for Information Systems (AIS) Americas Conference, Indianapolis, Indiana, 971-973.

Tyran, C.K., A.R. Dennis, D.R. Vogel, & J.F. Nunamaker (1992) The Application of Electronic Meeting Technology to Support Strategic Management. MIS Quarterly, September, 313-334.

Wulf, W. & R. Kouzes (1996) Collaboraties: Doing Science on the Internet. Computer, IEEE Computer Society Press, August, 40-46.

Ojoung Kwon, California State University--Fresno

Donna Dufner, University of Nebraska at Omaha

Maria Kwon, Southern Illinois University--School of Medicine

Table 1. Development History and Future Direction

Phase I(1993-1995) Face-to-Face (Same Time & Same Place)
 Meeting Environment
 Level 1 GDSS
 Decision Room
 GDSS, E-mail

Phase II(1995-1996) Distributed Synchronous (Same Time but
 Different Place)
 Level 1 GDSS
 LAN-based
 GDSS, E-mail

Phase III(1996-present) Distributed Asynchronous (Different Time
 & Different Place)
 Level 2 GDSS
 Web-based
 CyberCollaboratory including GDSS,
 E-mail, Discussion Group,
 Project Management, ASADE

Phase IV Intelligent Distributed Asynchronous
(Future Direction) Level 3 GDSS
 Web-based
 Intelligent Facilitation Agent
 (Self-Facilitation)

Table 2. Four types of collaborative interaction
(Hsu & Hockwood, 1993)

Place of Access Modes of Communication

 Same Time (Synchronous)

Same Place Face-to-Face
 Occurs at the same time and place
 Decision Room

Different Place Distributed Synchronous
 Occurs at the same time but at different places
 Audio/Video conferencing, LAN based GDSS

Place of Access Modes of Communication

 Different Time (Asynchronous)

Same Place Asynchronous
 Occurs at different times but at the same place
 Shared offices, Working in shifts

Different Place Distributed Asynchronous
 Occurs at different times and at different places
 E-mail, Discussion Groups Web-based GDSS

Table 3. Limitations of a Traditional Meeting
(Palmer and Palmer, 1983)

1. Subject and purpose are not clearly defined

2. The meeting is held at a bad time for those who are
 expected to attend

3. Physical meeting facilities are poor

4. Participants are not prepared with ideas or materials

5. The meeting does not start on time

6. The wrong people are in attendance; the right people are not

7. Participants are allowed to get off track

8. The leader gets off track

9. One participant dominates

10. Participants argue uncontrollably among themselves or with
 the leader

11. Participants do not express their true feelings or opinions

12. Interruptions do not express their true feelings or opinions

13. Participants are ridiculed or embarrassed by the leader or
 others in attendance

14. Participants are unclear or confused about the issues or
 information under discussion or being presented

15. Audiovisual equipment does not work

16. Participants can not see, read, hear, or understand audiovisual
 aids

17. The meeting lasts longer than planned

18. When the meeting is concluded, participants are not sure
 what was accomplished nor do they know what is to
 happen next

19. Participants are dissatisfied and unmotivated

Table 4. Reactions to the CyberCollaboratory

Means and Standard Deviations for Participants' Reactions to the
Communications Medium

Semantic Differential Scale 1 through 7 for all items N= 7

 Mean SD

Dependable Not dependable 3.0 1.4
Technical Non-technical 3.4 1.1
Delayed feedback Immediate feedback 4.0 0.8
Simple Complex 3.1 0.9
Urgent Not urgent 4.3 1.5
Inconvenient Convenient 4.1 1.3
Easy to use Hard to use 3.3 1.4
Always available Not always available 3.1 1.2
Flexible Restricted 3.6 1.3
Confidential Public 3.4 1.1
Ambiguous Clear 4.3 0.5
Personal Impersonal 4.1 0.7
Rich Impoverished 3.7 0.8
Much feedback Little feedback 4.0 1.0
Wide-ranging Narrow-focused 4.4 1.0
Subjective Objective 4.0 0.6
Sensitive Not sensitive 4.0 0.8

Table 5. User Perceptions of the System

Semantic differential scale variables
(differential scale range: 1 - 7), N = 7

Perceptions of the System Mean

Hard to learn Easy to learn 4.9

Impersonal Friendly 5.0

Frustrating Not frustrating 4.3

Time wasting Time Saving 5.1

Unproductive Productive 4.9

Table 6. User Perceptions of the GDSS Tools
Semantic differential scale variables N = 7

GDSS Tool User Perception of the GDSS Tools Mean

EBS Easy Hard 3.1

EBS Useless Valuable 4.9

IO Easy Hard 2.9

IO Useless Valuable 4.8

Voting Easy Hard 2.8

Voting Useless Valuable 5.1

Chat Easy Hard 2.9

Chat Useless Valuable 4.9

GD & DP Easy Hard 3.1

GD & DP Useless Valuable 4.9