Assessing Listening and Speaking Skills

TEXT: WHY TEACH AND ASSESS LISTENING AND SPEAKING SKILLS?

Even though many students have mastered basic listening and speaking skills, some students are much more effective in their oral communication than others. And those who are more effective communicators experience more success in school and in other areas of their lives. The skills that can make the difference between minimal and effective communication can be taught, practiced, and improved.

The method used for assessing oral communication skills depends on the purpose of the assessment. A method that is appropriate for giving feedback to students who are learning a new skill is not appropriate for evaluating students at the end of a course. However, any assessment method should adhere to the measurement principles of reliability, validity, and fairness. The instrument must be accurate and consistent, it must represent the abilities we wish to measure, and it must operate in the same way with a wide range of students. The concerns of measurement, as they relate to oral communication, are highlighted below. Detailed discussions of speaking and listening assessment may be found in Powers (1984), Rubin and Mead (1984), and Stiggins (1981).

HOW ARE ORAL COMMUNICATION AND LISTENING DEFINED?

Defining the domain of knowledge, skills, or attitudes to be measured is at the core of any assessment. Most people define oral communication narrowly, focusing on speaking and listening skills separately. Traditionally, when people describe speaking skills, they do so in a context of public speaking. Recently, however, definitions of speaking have been expanded (Brown 1981). One trend has been to focus on communication activities that reflect a variety of settings: one-to-many, small group, one-to-one, and mass media. Another approach has been to focus on using communication to achieve specific purposes: to inform, to persuade, and to solve problems. A third trend has been to focus on basic competencies needed for everyday life -- for example, giving directions, asking for information, or providing basic information in an emergency situation. The latter approach has been taken in the Speech Communication Association's guidelines for elementary and secondary students. Many of these broader views stress that oral communication is an interactive process in which an individual alternately takes the roles of speaker and listener, and which includes both verbal and nonverbal components.

Listening, like reading comprehension, is usually defined as a receptive skill comprising both a physical process and an interpretive, analytical process. (See Lundsteen 1979 for a discussion of listening.) However, this definition is often expanded to include critical listening skills (higher-order skills such as analysis and synthesis) and nonverbal listening (comprehending the meaning of tone of voice, facial expressions, gestures, and other nonverbal cues.) The expanded definition of listening also emphasizes the relationship between listening and speaking.

HOW ARE SPEAKING SKILLS ASSESSED?

Two methods are used for assessing speaking skills. In the observational approach, the student's behavior is observed and assessed unobtrusively. In the structured approach, the student is asked to perform one or more specific oral communication tasks. His or her performance on the task is then evaluated. The task can be administered in a one-on-one setting -- with the test administrator and one student -- or in a group or class setting. In either setting, students should feel that they are communicating meaningful content to a real audience. Tasks should focus on topics that all students can easily talk about, or, if they do not include such a focus, students should be given an opportunity to collect information on the topic.

Both observational and structured approaches use a variety of rating systems. A holistic rating captures a general impression of the student's performance. A primary trait score assesses the student's ability to achieve a specific communication purpose -- for example, to persuade the listener to adopt a certain point of view. Analytic scales capture the student's performance on various aspects of communication, such as delivery, organization, content, and language. Rating systems may describe varying degrees of competence along a scale or may indicate the presence or absence of a characteristic.

A major aspect of any rating system is rater objectivity: Is the rater applying the scoring criteria accurately and consistently to all students across time? The reliability of raters should be established during their training and checked during administration or scoring of the assessment. If ratings are made on the spot, two raters will be required for some administrations. If ratings are recorded for later scoring, double scoring will be needed.

HOW ARE LISTENING SKILLS ASSESSED?

Listening tests typically resemble reading comprehension tests except that the student listens to a passage instead of reading it. The student then answers mulitiple-choice questions that address various levels of literal and inferential comprehension. Important elements in all listening tests are (1) the listening stimuli, (2) the questions, and (3) the test environment.

The listening stimuli should represent typical oral language, and not consist of simply the oral reading of passages designed to be written material. The material should model the language that students might typically be expected to hear in the classroom, in various media, or in conversations. Since listening performance is strongly influenced by motivation and memory, the passages should be interesting and relatively short. To ensure fairness, topics should be grounded in experience common to all students, irrespective of sex and geographic, socioeconomic, or racial/ethnic background.

In regard to questions, multiple-choice items should focus on the most important aspects of the passage -- not trivial details -- and should measure skills from a particular domain. Answers designated as correct should be derived from the passage, without reliance on the student's prior knowledge or experience. Questions and response choices should meet accepted psychometric standards for multiple-choice questions.

An alternative to the multiple-choice test is a performance test that requires students to select a picture or actually perform a task based on oral instruction. For example, students might hear a description of several geometric figures and choose pictures that match the description, or they might be given a map and instructed to trace a route that is described orally.

The testing environment for listening assessment should be free of external distractions. If stimuli are presented from a tape, the sound quality should be excellent. If stimuli are presented by a test administrator, the material should be presented clearly, with appropriate volume and rate of speaking.

HOW SHOULD ASSESSMENT INSTRUMENTS BE SELECTED OR DESIGNED?

Identifying an appropriate instrument depends upon the purpose for assessment and the availability of existing instruments. If the purpose is to assess a specific set of skills -- for instance, diagnosing strengths and weaknesses or assessing mastery of an objective -- the test should match those skills. If appropriate tests are not available, it makes sense to design an assessment instrument to reflect specific needs. If the purpose is to assess communication broadly, as in evaluating a new program or assessing district goals, the test should measure progress over time and, if possible, describe that progress in terms of external norms, such as national or state norms. In this case, it is useful to seek out a pertinent test that has undergone careful development, validation, and norming, even if it does not exactly match the local program.

Several reviews of oral communication tests are available (Rubin and Mead 1984). The Speech Communication Association has compiled a set of RESOURCES FOR ASSESSMENT IN COMMUNICATION, which includes standards for effective oral communication programs, criteria for evaluating instruments, procedures for assessing speaking and listening, an annotated bibliography, and a list of consultants.

CONCLUSIONS

The abilities to listen critically and to express oneself clearly and effectively contribute to a student's success in school and later in life. Teachers concerned with developing the speaking and listening communication skills of their students need methods for assessing their students' progress. These techniques range from observation and questioning to standardized testing. However, even the most informal methods should embrace the measurement principles of reliability, validity, and fairness. The methods used should be appropriate to the purpose of the assessment and make use of the best instruments and procedures available.

FOR MORE INFORMATION

Brown, Kenneth L. TEACHING, SPEAKING AND LISTENING SKILLS IN THE ELEMENTARY AND SECONDARY SCHOOL. Boston, MA: Massachusetts Department of Education, 1981. ED 234 440.

Lundsteen, Sara W. LISTENING: ITS IMPACT ON READING AND THE OTHER LANGUAGE ARTS. Revised ed. Urbana, IL: National Council of Teachers of English and the ERIC Clearinghouse on Reading and Communication Skills, 1979. ED 169 537.

Powers, Donald E. CONSIDERATIONS FOR DEVELOPING MEASURES OF SPEAKING AND LISTENING. New York: College Entrance Examination Board, 1984.

Rubin, Don. L., and Mead, Nancy A. LARGE SCALE ASSESSMENT OF ORAL COMMUNICATION SKILLS: KINDERGARTEN THROUGH GRADE 12. Annandale, VA: Speech Communication Association and the ERIC Clearinghouse on Reading and Communication Skills, 1984. ED 245 293.

Speech Communication Association. RESOURCES FOR ASSESSMENT IN COMMUNICATION. Annandale, VA.: Speech Communication Association, 1984.

SCA Guidelines: ESSENTIAL SPEAKING AND LISTENING SKILLS FOR ELEMENTARY SCHOOL STUDENTS (6th GRADE LEVEL). Annandale, VA.: Speech Communication Association. (Pamphlet, 1984).

SCA Guidelines: SPEAKING AND LISTENING COMPETENCIES FOR HIGH SCHOOL GRADUATES. Annandale, VA.: Speech Communication Association. (Pamphlet, 1984).

Stiggins, Richard J., ed. PERSPECTIVES ON THE ASSESSMENT OF SPEAKING AND LISTENING SKILLS FOR THE 1980s. Portland, OR: Clearinghouse for Applied Performance Testing, Northwest Regional Educational Laboratory, 1981. ED 210 748.