Fair and Unfair Testing Accommodations

What's considered appropriate when assessing the academic performance of students with disabilities?

Is it fair for an individual with a disability to take a nonstandard administration of a standardized achievement test when students without disabilities don't have this option?

This is just one of the concerns facing schools as they work to assess students with disabilities. Among others: Should all students with the same disability receive the same accommodation? How do we know we are choosing the right accommodation for any one student with a disability?

Two prominent policy initiatives lend critical significance to these questions. First, strengthening academic performance and accountability have become bywords at the federal, state and district levels. At the same time, the 1997 amendments to the Individuals with Disabilities Education Act require states and districts to include students with disabilities in accountability programs.

The rationale for the IDEA amendments is based on the fact that performance of students with disabilities has not always counted in accountability programs. "Too often, in the past," explain Judith Heumann and Norma Cantu, both assistant secretaries in the U. S. Department of Education, "students with disabilities have not fully participated in state and district assessments, only to be shortchanged by the low expectations and less challenging curriculum that may result from exclusion."

Jim Ysseldyke and Martha Thurlow of the National Center on Educational Outcomes document similar problems. These researchers found that statewide accountability programs prior to 1998 excluded many students with disabilities. According to a 1997 National Academy of Sciences panel, even when students with disabilities completed assessments, some states or districts removed their scores from public reports.

What Accommodations?

Test accommodations are changes in standardized test conditions designed to level the playing field between students with and without disabilities. The purpose of identifying appropriate accommodations is to achieve valid, not optimal, scores. On one hand, disallowing appropriate, or valid, accommodations prevents students with disabilities from demonstrating their competence. On the other hand, overly permissive accommodation policies inflate scores and thereby reduce pressure on schools to increase learning opportunities for students with disabilities.

For example, if an adult reads a reading test to a student with disabilities, the student will score unrealistically high. With this high score, the school will experience less pressure to provide appropriate, quality reading instruction to the student.

Unfortunately, policymakers and test experts do not agree on methods for determining which accommodations are indeed valid. According to the Council of Chief State School Officers, some states prohibit the same accommodations that other states recommend. Without consensus on appropriate criteria, comparisons among states or districts can become unfair and meaningless. This is one reason why states and districts have excluded students with disabilities from their accountability programs.

The IDEA amendments, however, deny states and districts that latitude. Although a small proportion of students (for example, some with severe cognitive deficits) may participate in alternate assessments, districts now must test all students with disabilities. To accomplish this, they must develop test accommodation policies.

Key Assumptions

For some disabilities, accommodations that level the playing field can be identified easily using logical analysis. For example, a large-print version of a mathematics applications test permits many students with visual disabilities to demonstrate their competence, while the meaningfulness of the test construct is preserved.

Moreover, although this accommodation makes the test more accessible to many students with disabilities, it would not increase the scores of children without disabilities. This notion of "differential boost," which Susan Phillips of Michigan State University's Institute for Social Research describes as an accommodation that increases the performance of students with disabilities more than it increases the scores of students without disabilities, is critical to the concept of leveling the playing field.

Of course, logical analysis of test accommodations is not always so straightforward. For example, students with learning disabilities (who constitute more than half the population of students in special education) present a particular challenge for two reasons. First, this group is dramatically heterogeneous. As shown by researchers Deborah Speece and David Cooper of the University of Maryland, students with learning disabilities can be subtyped into clusters with varying underlying problems, thus making conceptual analysis of meaningful test accommodations impossible.

A second concern is the nature of the cognitive problems that students with LD present. Their most distinguishing characteristics are reading and math deficits, while most large-scale assessments directly measure or rely heavily on these same skills.

Under these circumstances, many accommodations currently used (for instance, extending test taking time, decoding questions or encoding responses) may distort the meaning and interpretation of scores. In essence, because the disability is intertwined with the constructs measured, accommodations may exempt students with LD from demonstrating he skills the test measures.

These consideration support two major assumptions about the validity of test accommodations. First, as illustrated with the large-print accommodation for students with visual disabilities, accommodations must produce a differential boost. Second, for disabilities such as LD, where t e population and its underlying problems are more disparate, appropriate accommodation may need to be determined on a case-by-case basis.

Practical Implications

With these assumptions in mind, researchers have been co ducting controlled studies to compare the effects of test accommodations for students with and without learning disabilities.

One broad implication from this emerging database is that some of the most popular accommodations do not differentially benefit students with LD. Extended test time, for example, increases scores for students with LD no more than for students without LD. Similarly, directing students with LD to mark responses directly on tests, instead of bubbling in answer sheets, may not be useful.

By contrast, some less frequently used accommodations produce a differential boost for learning disabled students. Permitting students to read aloud for reading assessments or providing an adult reader (or writer) for math tests requiring extensive reading or writing demands appear to produce a differential boost.

But what about the marked heterogeneity of the LD population? It is important to remember that accommodations that do not yield average differential boosts for groups of these students do produce substantial boosts for some individuals. The question becomes this: How do we identify appropriate accommodations for each student with LD?

Teacher Decisions

The new IDEA amendments require school personnel to make judgments about which accommodations are appropriate for which children with disabilities. This responsibility typically falls to the individual education plan team, which tends to rely heavily on the child's teacher. Research we have conducted at Vanderbilt University, however, suggests a need for caution when relying on teacher judgments about test accommodations.

In studies we have completed in reading and math, teacher decisions about which students with a learning disability required which test accommodations did not correspond to the benefits children actually derived. For example, the group of students to whom the teachers awarded extended time did not profit from this accommodation more than the group to whom the teachers had denied the extra time.

In addition, teachers awarded accommodations to large numbers of students who benefited from the accommodations no more than students without a disability.

Objective Methods

Because of difficulties associated with relying solely on teacher judgment, we have developed an objective diagnostic procedure that we call the Dynamic Assessment of Testing Accommodations, or DATA. With DATA, teachers administer brief tests, with and without accommodations. Then for each student they calculate the performance-boost (score on the accommodated test minus the score on the standard administration) associate with each accommodation.

Teachers compare the size of these boosts to normative information about how these accommodations typically affect nondisabled students. In this way, teachers identify individual children whose performance boosts substantially exceed those of nondisabled students.

Take Bobby, a 4th-grade student with a learning disability. Using DATA to assess his response to various accommodations on concepts and applications on math tests, his teacher administered four different but equivalent 25-item tests: a standard administration (six minutes; no calculator; no reader); a test with extended time (20 minutes); a test with a calculator; and a test with an adult reading nonmathematical content.

Bobby's performance boost varied widely under the accommodations. With extended time, his score remained the same; with a calculator, his score increased by 5 problems; and with an adult reader, his score increased by 19 problems.

Bobby's teacher then compared these boosts to 4th-grade norms. Bobby's boost exceeded those of the nondisabled students when an adult reader was provided as an accommodation. Based on these comparisons, in combination with her observations of Bobby's classroom performance, the teacher recommended that Bobby be provided with a reader for math tests that are administered as part of the regular instructional program as well as those that are used in the statewide assessment.

Our research shows that DATA-based methods predict students' actual performance boosts on standardized achievement tests better than teacher judgments about test accommodations. For example, in a study we conducted during the 1998-99 school year, we used the Iowa Test of Basic Skills reading comprehension subtest to monitor 400 students, including both children with and without learning disabilities in the area of reading.

The conclusion of that study was that, on the average, the accommodation boost on the Iowa test was larger for students to whom teachers had denied accommodations than it was for students to whom teachers had granted special accommodations. By contrast, accommodation boosts were significantly larger for students to whom DATA had awarded accommodations compared to students to whom DATA had denied accommodations. In other words, DATA dramatically strengthened teachers' judgments about which students with LD benefited from accommodations.

Improving Decisions

As schools comply with the 1997 IDEA amendments, they must make decisions about which students require which accommodations for state and district assessments. At the current time, many educators have relatively little understanding of the purpose of test accommodations. They also have little experience in observing the actual effects of accommodations for students with and without disabilities.

DATA not only enhances teacher judgment in this area, but also may help educators become more familiar with the effects of test accommodations.

By using DATA and other objective methods for identifying valid accommodations, teacher judgment should improve, decisions about test accommodations should become more sound, and the whole issue of test accommodations should become less controversial.

Lynn Fuchs and Douglas Fuchs are professors of special education at Vanderbil University, Box 328 Peabody, Nashville, Tenn. 37203.

More on Test Accommodations

Lynn Fuchs, a professor of special education at Vanderbilt University, and Steve Henry, a central-office administrator in the Topeka, Kan., schools, compiled the following resources on how to deal appropriately with test accommodations for students with disabilities.

Publications

Analyzing, Dissaggregating, Reporting and Interpreting Students' Achievement Test Results: A Guide to Practice for Title I and Beyond, by Richard M. Jaeger and Charlene G. Tucker, available from the Council of Chief State School Officers, Washington, D.C., 202-336-7005, www.ccsso.org

Educating One and All: Students with Disabilities and Standards-Based Reform, edited by Lorraine M. McDonnell, Margaret J. McLaughlin and Patricia Morison, available from National Academy Press, Washington, D.C., 888-624-8373, www.nap.edu

Testing Students With Disabilities: Practical Strategies for Complying With District and State Requirements, by Martha Thurlow, Judy Elliott and James Ysseldyke, available from Corwin Press, Thousand Oaks, Calif., 805-499-9774, www.corwinpress.com

"State Assessment Policies on Participation and Accommodations for Students with Disabilities" (Synthesis Report No. 29), by Martha Thurlow and others, University of Minnesota, National Center on Educational Outcomes, Minneapolis, Minn., 612-626-1530, www.coled.umn.edu/nceo

Journal Articles

"High-Stakes Testing Accommodations: Validity Versus Disabled Rights," by Susan Phillips, Applied Measurement in Education, Volume 7 (2), 93-120. 1994.

"Supplementing Teachers' Judgment of Test Accommodations with Objective Data Source by Lynn Fuchs and others, School Psychology Review, expected publication early 2000.

Web Resources

www.coled.umn.edu/nceo Web site for the National Center for Educational Outcomes at the University of Minnesota. NCEO is a federally funded center brokering research studies and providing technical assistance on participation of students with disabilities in national and state assessments, standards-setting efforts and graduation requirements. NCEO documents are available at this site.

www.natd.org Web site for the National Association of Test Directors, including examples of district testing policies, testing schedules, sample forms and other documents relating to all aspects of assessment in K-12 Schools.

ericae.net and ericae.net/intass.htm Both of these Web sites, maintained by the ERIC Clearinghouse on Assessment and Evaluation, contain a wide range of relevant information. See especially the disabilites category at the second Web site for information on accommodation and alternate assessments.

COPYRIGHT 1999 American Association of School Administrators
COPYRIGHT 2004 Gale Group