Hierarchical Linear Modeling of the Effect of Experience on Decision-Making in Outdoor Leaders

This research measures the effect that experience has on the decision-making of outdoor leaders from the perspective of social judgment theory. It investigates the following five research questions:

1. Are there significant differences between experience levels in outdoor leaders?

2. Do subgroups of outdoor leaders (by level of experience) vary in levels of perceptual threshold for particular factors in the decision-making environment?

3. Do subgroups of outdoor leaders vary significantly differently in their judgments?

4. Do subgroups of outdoor leaders vary in the extent of stochastic error displayed?

5. Do subgroups of outdoor leaders vary significantly in the weights they give to different dimensions in the decision vignettes?

Two instruments were developed for this research: decision-making was measured utilizing vignettes developed using a factorial survey approach. Experience was assessed using experience-use history methodology. The sample for the current study includes staff from North Carolina Outward Bound School and Voyageur Outward Bound School (NCOBS, N = 61; VOBS, N = 42). Data was collected in June 2002. Hierarchical linear modeling identified significant fixed effects for degree of isolation, level of student injury, and group cohesion. Significant random effects were identified for degree of isolation and level of student injury. No significant within or between-instructor effects were identified in this data.

In order to develop ecological validity for the factorial decision vignettes, a qualitative methodology was utilized in determining the factors, dimensions, and levels relevant to the decision-making process for outdoor leaders. A semi-structured interview format was developed and interviews were conducted with Voyageur Outward Bound School (VOBS) field staff members (N= 20) in late June 2001. The sample constituted a range of experience within this organization, as well as across the outdoor leadership field as a whole (intern, assistant instructor, instructor, etc.). Recorded interviews were transcribed and the text was coded for factors affecting decision-making in field situations utilizing a selective coding process (Strauss & Corbin, 1998).

Internal consistency reliability for the OLEUH was estimated by computing Cronbach's alpha for the eight subscales, resulting in an overall reliability coefficient of .78. This finding indicates some improvement in instrument reliability over the results from the analysis conducted during the instrument development phase, which indicated a Chronbach's alpha of .71. In both cases, this coefficient meets the accepted standard for reliability coefficients of .70 (Nunnally, 1978). In order to estimate construct validity, a factor analysis was conducted. Principal components analysis using Varimax rotation resulted in a two-factor model that explained 55.43% of the variance in this data set. Variance explained during instrument development equaled 44.62, indicating some improvement in the current data set over previous data.

Subjects were asked to complete an OLEUH and a set of 20 "Decision Vignettes" that were randomly selected from the factorial object universe. Given the complexity of the instruments involved, data collection took place during staff training (June 4-5, 2003). The majority of the staff for each base is only on-site during particular periods of time, and all-staff training provided the highest concentration of instructors for data collection. An on-site data collection approach was selected in order to gather most of the data at one time and to provide clarification on instrument questions, if needed. Duo to the fact that both VOBS' and NCOBS' All-Staff Trainings occur on the same days, a research assistant was recruited and trained to collect data at VOBS, while the principle researcher executed the NCOBS data collection. Subjects for this examination consisted of field instructor staff from Voyageur Outward Bound School (Field staff, N = 80) and North Carolina Outward Bound School (Field staff, N = 90). These organizations were selected due to the broad range of experience (e.g., intern, assistant instructor, instructor, course director) and professional development of outdoor leaders within each organization, as well as the fact that instrumentation developed for this study was refined using data from these individuals.

The instructors varied significantly in terms of their experience levels. Significant variance was also present in both the judgment threshold and judgment processes of the outdoor leaders for the environmental factors of the degree of isolation, and the level of the student's injury. The instructors also displayed significant variance in their judgments according to their level of experience for the decision environment factors of degree of isolation and level of student injury. The complicated nature of the error structure generated in hierarchical linear models renders the response to the question of stochastic error and outdoor instructor experience more difficult to answer.

Analysis of the fixed effects leads to several conclusions. The interaction of isolation, injury, and experience on decisions to evacuate indicates a higher level of efficiency for outdoor instructors with the highest levels of experience in response to the higher degree of risk. It is reasonable to conclude that, in terms of degree of isolation and level of injury, more experienced instructors display a higher degree of risk aversion. The relationship between the degree of isolation and experience indicates that the novice group becomes less likely to evacuate as the degree of isolation increases. The interaction of level of student injury and level of experience in outdoor leaders also varies across the range of experience. The interaction of group cohesion and level of experience indicates different levels of the ability to utilize the group in problem-solving.

Implications of the study include indications for training and development of outdoor leaders. Medical training may be augmented with isolated settings and/or social components to better simulate naturalistic decisionmaking environments. The decision-making of less experienced outdoor leaders may benefit from additional training for coping in isolated environments, as well as methods for using the group to help problem-solve.

Information gained from this study allows for an empirical understanding of critical factors in the decision-making process of outdoor leaders. Understanding the weights that outdoor instructors of different experience levels place on factors in their decision-making, as well as variance in assignment of those weights and stochastic error rates, provides an empirical foundation for the training and professional development of outdoor leader decision-making at the staff and individual levels. Furthermore, access to such information reveals important risk management practices or provides justification for existing practices. For example, training program curricula designed specifically for novice decision-makers could integrate information and practice with problems adapted to empirically observed novice decision policies. Development of an empirical understanding of the decision-making of outdoor leaders allows for the development of this occupation as a profession, enhancing both critical abilities of outdoor leaders, and the credibility of the field as a whole.

Shayne Galloway, Ph.D., Assistant Professor, Physical Education & Recreation, Utah Valley State College, Orem, UT 84058-5999. E-mail: gallowash@uvsc.edu

Copyright Association for Experiential Education 2005
Provided by ProQuest Information and Learning Company. All rights Reserved