An ecological approach to evaluating a special education program.

DeSouza, Eros Ramos ; Sivewright, David

About fifteen percent of all children show mild behavioral problems, and about seven percent show moderate to severe disorders (Cotler, 1986). However, not enough has been done to treat or prevent these problems (Levine & Perkins, 1987). Even less has been done to evaluate existing treatment programs (Felner et al., 1983).

Historically, evaluations of special education programs have examined deficits in student performance or capability, focusing on organismic, cognitive, or behavioral problems such as brain damage, perception difficulties, learning disabilities, and lack of motivation (Greenwood & Carta, 1987). Although these individual-level factors are important, they may lead to premature labeling and victimization. For example, young children have been disproportionately labeled as socially, cognitively, or behaviorally "deficient" (Drabman, Tarnowski, & Kelly, 1987). Hence, teachers may interact with students as if they were a bundle of deficiencies (labels), rather than considering settings and systems as part of the problem or solution.

Further, children who live in economically and educationally impoverished environments are at higher risk for a number of negative educational and social-emotional outcomes (Brand, Dubois, & Felner, 1990). Thus, ecological variables need to be investigated because they can either influence the onset of behavioral problems (Heller, 1990) or positively affect learning behaviors (Greenwood & Carta, 1987).

The interaction of academic materials (static features of the environment) with students and teachers (dynamic features of the environment) forms classroom ecologies. In an effort to describe these environments, the ecobehavioral approach to program evaluation has emerged in the field of applied behavior analysis. This approach links ecological factors, which may facilitate or hinder students' academic performance, with program outcomes (Greenwood & Carta, 1987). Thus, the ecobehavioral approach is useful for providing important information about the classroom climate and the effects of interventions (Greenwood et al., 1985).

One indication of positive effects of special education programs is that students engage more often in academic-related behaviors than in disruptive ones. For example, there is evidence that active classroom behaviors (e.g., asking or answering questions, reading, writing) are correlates of academic achievement, whereas disruptive behaviors are not (Rosenshine & Stevens, 1986). As Becker (1977) states, "at risk" students need instruction that enables them to perform academically at accelerated rates. In fact, these students need to learn more and faster just to obtain achievement comparable to more advantaged students. Therefore, the purpose of the present study was to examine ecological and student behavior variables that may be linked to the objectives of the program--increasing appropriate classroom behaviors and decreasing inappropriate ones. Here, the ecobehavioral approach was modified in order to describe patterns in the whole classroom rather than patterns in one student, as originally proposed by Greenwood and Carta (1987).

METHOD

Setting and Program Description

The setting was a private residential treatment center located in a large midwestern city. The center serves behaviorally disturbed adolescent males between the ages of 12 and 18 years. For those whose educational needs cannot be met in a regular school setting, the center offers a special education program. This program aims to alleviate behavioral difficulties that impede academic performance, such as chronic task incompletion, acting out, behavior problems, or social interaction conflicts. Two important objectives are to increase academic-related performance and decrease disruptive behaviors in the classroom.

Participants

At the time the study was conducted, there were 84 emotionally disturbed adolescent males in the special education program. Their average age was 15 years, and most of them were Caucasian (90%). The majority of the boys (71%) were diagnosed with disruptive disorders (American Psychiatric Association, 1987).

Materials

Ecobehavioral data were collected using the Code for Instructional Structure and Student Academic Response (CISSAR; Stanley & Greenwood, 1981). CISSAR assesses the occurrence of ecological events and student behavior variables that are closely associated in time. A brief description of the CISSAR codes used in this study appears in Table 1.

In order to permit additive frequencies, the CISSAR codes were designed to be mutually exclusive. Observers concurrently assessed the occurrence of one ecological event and one student behavior. This type of assessment allowed for the creation of three hierarchical composites from the student behavior variables: (1) active academic responses, including writing, engaging in task participation, reading aloud, talking about academic topics, and reading silently; (2) passive responses, including attending to task (not actively engaged) and other appropriate behaviors (not academically related); and (3) competing responses (behaviors that compete with academic ones), including looking around, being disruptive, or exhibiting other inappropriate behaviors.

Procedure

Observational data were collected using a momentary time sampling procedure. This type of observation is analogous to taking a photograph of the instructional process at that instant. A 15-second time interval was used.

Before collecting usable data, observers had to become thoroughly familiar with the CISSAR codes during a three-week training period. At its completion, these observers achieved 98% agreement in three consecutive 20-minute observations. Interrater reliability was computed by comparing the records of two observers, who simultaneously, but independently, recorded the same phenomena. If the accounts agreed exactly or disagreed by no more than plus or minus one tally, agreement was recorded for the cell.

Classroom observations were then randomly scheduled in math, English, and social studies courses ("core classes" that all students were required to take) during the academic semester. For each observation, TABULAR DATA OMITTED one observer recorded the instructional process for a 20-minute block, unless there was a prolonged interruption, which resulted in a new observational block. At the end of each 15-second interval, the observer targeted one student and coded both the ecological event and student behavior occurring at that precise moment. At the end of the next interval, the observer targeted another student and recorded the ecological event and behavior he was engaged in. Each student was targeted in a clockwise manner until all students in the classroom had been observed (classroom size ranged from 4 to 12 students, with an average of 10 per room). The observer then repeated the cycle. Thus, inferences could be made about the relative frequency of occurrence of each variable for the entire class.

In order to minimize observer bias or drift, reliability checks were conducted every three weeks for the duration of the semester. Observers maintained an excellent level of interrater reliability (90% or above).

RESULTS

The results of 4,612 samples of behavior from nearly 20 hours of direct classroom observation are presented in Tables 2 and 3. Percentages for the response composites and the student behavior variables appear in Table 2. Active academic responses comprised 56% of observed time, with reading silently (29%) and writing (16%) being the most frequent behaviors. Passive responses accounted for 36% of observed time, with attending to task (29%) being the most frequent behavior. Competing responses made up only 9% of observed time, with looking around (6%) being the most frequent behavior in this category.

The ecobehavioral data also allowed for determination of conditional probabilities, which are estimations of the probability of particular behaviors given the presence of specific materials (physical aspects) or groups (types of relationships between persons in groups). The conditional probabilities are shown in Table 3. The most interesting of these conditional probabilities concerns competing responses. Although student-student discussion (Ssd) occurred at a very low rate (5%), it was during this time that students engaged in the highest level of other inappropriate (Oi) behaviors (27%). Another interesting finding is that writing and reading occurred almost exclusively during independent tasks (e.g., when using readers, workbooks, worksheets, paper and pencil, and other media), rather than during more interactive lessons, such as discussions with the teacher or peers.

Table 2

RESPONSE COMPOSITES AND PERCENTAGES

Composite Student Behaviors %

Active Responses Writing 16%
 Task participation 1%
 Reading aloud 1%
 Talking academically 9%
 Reading silently 29%

 56%

Passive Responses Attending to task 29%
 Other appropriate 7%

 36%

Competing Responses Looking around 6%
 Disruptive 0%
 Other inappropriate 3%

 9%

TABULAR DATA OMITTED

DISCUSSION

The ecobehavioral data reveal that these students spent over half of the observed time engaged in active academic responses (56%) and only 9% of the time in competing responses (inappropriate behaviors). Although these findings indicate that the program apparently accomplished its objectives (i.e., to increase appropriate and decrease competing behaviors), there are intriguing ecological implications. First, most of the active academic behaviors observed reflect engagement in busywork (static aspects of the classroom environment). For example, although reading silently (29%) and writing (16%) comprised the most common forms of active responding, students were working by themselves rather than engaging in interactive lessons, such as discussions with the teacher and peers (dynamic features of the environment).

Second, these students engaged in passive responding over one-third of the time (36%). They spent a considerable amount of time (29%) appearing to be paying attention in class (i.e., attending to task). During that time, they spent 87% of the time appearing to be listening to their teacher lecturing. That is, they were oriented toward the teacher, but made no active responses (e.g., asking questions) that would demonstrate comprehension or mastery. Thus, the data suggest that these students may act as passive responders to external, perhaps uninteresting, stimuli.

Third, these students were often restricted to highly structured classroom activities. Such activities may provide few opportunities for students to engage in challenging work. The literature indicates that highly structured activities do little to enhance students' abilities to think critically, weigh evidence, and develop independent judgment (Dreeben & Gamoran, 1986; Greenwood, Delquadri, & Hall, 1984; Longshore & Prager, 1985). In a similar vein, there is evidence that controlling environments often have a deleterious effect on motivation and learning (Deci & Ryan, 1985).

In summary, these students may benefit from challenging interactions that promote learning, such as discussions with teachers and peers (Deci & Ryan, 1985). Specifically, they may advance faster if they receive immediate feedback on their tasks (e.g., by asking questions or talking about academic matters, rather than by engaging in busywork).

CONCLUSION

After disseminating these findings to administrators and teachers, those in charge of the program have made attempts to improve it. For example, teachers have been instructed to increase active learning responses by providing students with more challenging tasks and greater opportunities for immediate feedback.

There are costs as well as benefits involved in this modified version of the ecobehavioral approach. Although generalization is increased by assessing all students in the classroom, it is difficult to determine the validity of this approach from a single study. Thus, more experimental validation is warranted.

REFERENCES

American Psychiatric Association (1987). Diagnostic and statistical manual of mental disorders (3rd ed., rev.). Washington, DC: Author.

Becker, W. C. (1977). Teaching reading and language to the disadvantaged--What we have learned from field research. Harvard Educational Review, 47, 518-543.

Brand, S., Dubois, D., & Felner, R. (1990, August). Classroom environment and adaptation among disadvantaged elementary school students. Paper presented at a meeting of the American Psychological Association, Boston.

Cotler, S. (1986). Epidemiology and outcome. In J.M. Reisman (Ed.), Behavior disorders in infants, children, and adolescents. New York: Random House.

Deci, E. L., & Ryan, R. M. (1985). Intrinsic motivation and self-determination in human behavior. New York: Plenum Press.

Drabman, R. S., Tarnowski, K. J., & Kelly, P. A. (1987). Are younger classroom children disproportionately referred for childhood academic and behavior problems? Journal of Consulting and Clinical Psychology, 55, 907-909.

Dreeben, R., & Gamoran, A. (1986). Race, instruction, and learning. American Sociological Review, 51, 660-669.

Felner, R. D., Jason, L. A., Moritsugu, J. N., & Farber, S. S. (1983). Preventive psychology: Theory, research and practice. New York: Pergamon Press.

Greenwood, C. R., & Carta, J. J. (1987). An ecobehavioral interaction analysis of instruction within special education. Focus on Exceptional Children, 19, 1-12.

Greenwood, C. R., Delquadri, J. C., & Hall, R. V. (1984). Opportunity to respond and student academic performance. In W. L. Heward, T. E. Herron, J. Trap-Porter, & D. S. Hill (Eds.), Focus on behavior analysis in education. Columbus, OH: Charles Merrill.

Greenwood, C. R., Delquadri, J. C., Stanley, S. O., Terry, T., & Hall, R. V. (1985). Assessment of eco-behavioral interaction in school settings. Behavioral Assessment, 7, 331-347.

Heller, K. (1990). Social and community intervention. Annual Review of Psychology, 41, 141-168.

Levine, M., & Perkins, D. V. (1987). Principles of community psychology: Perspectives and applications. New York: Oxford University Press.

Longshore, D., & Prager, J. (1985). The impact of school desegregation: A situational analysis. Annual Review of Sociology, 11, 75-91.

Rosenshine, B., & Stevens, R. (1986). Teaching functions. In M. C. Wittrock (Ed.), Handbook of research on teaching (3rd ed., pp. 376-391). New York: Macmillan.

Stanley, S. O., & Greenwood, C. R. (1981). CISSAR: Code for instructional structure and student academic response. Observer's manual. Kansas City, KS: University of Kansas, Bureau of Child Research, Juniper Gardens Children's Project.