Evaluating School Guidance Programs

"Demonstrating accountability through the measured effectiveness of the delivery of the guidance program and the performance of the guidance staff helps ensure that students, parents, teachers, administrators, and the general public will continue to benefit from quality comprehensive guidance programs" (Gysbers & Henderson, 1994, p. 362). To achieve accountability, evaluation is needed concerning the nature, structure, organization and implementation of school district/building guidance programs; the school counselors and other personnel who are implementing the programs; and the impact the programs are having on students, the schools where they learn, and the communities in which they live. Thus, the overall evaluation of school district/building guidance programs needs to be approached from three perspectives: program evaluation, personnel evaluation, and results evaluation (Gysbers & Henderson, 1994).GUIDANCE PROGRAM EVALUATIONGuidance program evaluation asks two questions. First, is there a written guidance program in the school district? And second, is the written guidance program the actual implemented program in the buildings of the district? Discrepancies between the written program and the implemented program, if present, will come into sharp focus as the program evaluation process unfolds.

To conduct program evaluation, program standards are required. Program standards are acknowledged measures of comparison or the criteria used to make judgments about the adequacy of the nature and structure of the program as well as the degree to which the program is in place. For example, here is a program standard:

The school district is able to demonstrate that all students are

provided the opportunity to gain knowledge, skills, values, and

attitudes that lead to a self-sufficient, socially responsible life.

(Gysbers & Henderson, 1994, p. 481)

To make judgments about guidance programs using standards, evidence is needed concerning whether or not the standards are being met. In program evaluation such evidence is called documentation. Using the standard listed above, evidence that the standard is in place might include the following:

1. A developmentally appropriate guidance curriculum that teaches all students the knowledge and skills they need to be self-sufficient and lead socially responsible lives.

2. Yearly schedule that incorporates the classroom guidance plan (Gysbers & Henderson, 1994, p. 482).

Documentation of such evidence could include:

1. guidance curriculum guides

2. teachers' and counselors' unit and lesson plans

3. yearly master calendar for the guidance program

4. curriculum materials (Gysbers & Henderson, 1994, p. 482)

Sometimes the program evaluation process is called a program audit. The American School Counselor Association, for example, uses the "term" audit in its program evaluation materials. The Association has developed guidelines for a program audit for secondary schools (ASCA, 1986), for middle/junior high schools (ASCA, 1990b), and for elementary schools (ASCA, 1990a).GUIDANCE PROGRAM PERSONNEL EVALUATIONPersonnel evaluation begins with the organizational structure and activities of the guidance program in a school district. A major first step is the development of job descriptions that are based directly on the structure and activities of a school district's guidance program.

Using the Missouri Comprehensive Guidance Program framework for example, the job description of a school counselor would include the following key duties: implementing the guidance curriculum; counseling individuals and small groups concerning their educational and occupational plans; counseling individuals and small groups with immediate needs and specific problems; consulting with parents and teachers; referring students to appropriate community agencies; coordinating, conducting, and being involved with activities that improve the operation of the school; evaluating and updating the guidance program; and continuing professional development (Starr & Gysbers, 1993). (For examples of job descriptions of other guidance personnel including director of guidance, career guidance center technician, and high school registrar, see Gysbers & Henderson, 1994, 422-428).

Guidance program personnel evaluation is based directly on their job task descriptions and usually has two parts: a formative part (supervision) and a summative part (evaluation). The job task description identifies the performance areas to be supervised and evaluated. Gysbers and Henderson (1994) have developed an extensive listing of job task descriptors for school counselors grouped under the basic guidance program components of guidance curriculum, individual planning, responsive services, and system support plus the areas of professional relationships and professional responsibilities.PROGRAM RESULTS EVALUATIONHaving established that a guidance program is operating in a school district through program evaluation, and having established through personnel evaluation that school counselors and other guidance program personnel are carrying out the duties listed on their job descriptions 100% of the time, it now is possible to evaluate the results of the program. Johnson (1991) suggested that there are long-range, intermediate, immediate, and unplanned-for results that need consideration. According to Johnson, long-range results focus on how programs affect students after they have left school. Usually long-range results are gathered using follow-up studies. Intermediate results focus on the knowledge and skills all students may gain by graduation from participating in the guidance program. Immediate results are the knowledge and skills students may gain from participating in specific guidance activities. Finally, the possibility of unplanned-for results that may occur as a consequence of guidance activities conducted as a part of the guidance program also need to be taken into account.

For the purposes of this digest, illustrations of immediate and intermediate results evaluation using the structure of the Missouri Comprehensive Guidance Program Model (Starr & Gysbers, 1993) are presented in the form of two research questions. First, do students master guidance competencies as a result of their participation in the Guidance Curriculum Component of the Model (immediate evaluation)? Second, do students develop and use career plans as a result of their participation in the Individual Planning Component of the Model (intermediate evaluation)?IMMEDIATE EVALUATION--GUIDANCE COMPETENCY MASTERYDo students master guidance competencies? Johnson (1991) outlined the following procedures to answer this question for immediate results. First the competencies to be mastered need to be identified. Second what results (what students should be able to write, what they may be able to talk about, or what they may be able to do) are specified. Then who will conduct the evaluation is decided. This is followed by when the evaluation is done. Then criteria are established so that judgments can be made about students' mastery of guidance competencies. Finally, how all of this is done is specified.

Do students master guidance competencies? Another way to conduct immediate evaluation, to measure mastery of guidance competencies, is the use of a confidence survey. In this format, guidance competencies are listed and students are asked to rate how confident they are that they have mastered these competencies. The confidence survey can then be used as a pre-post measure. Gain scores can be obtained and related to such measures as academic achievement and vocational identity. (Gysbers, Hughey, Starr, & Lapan, 1992; Gysbers, Lapan, Multon, & Lukin, 1992; Lapan, Gysbers, Hughey, & Arni, 1993).INTERMEDIATE EVALUATION--CAREER PLANSDo students develop and use career plans? In making judgments concerning the career plans of students, criteria need to be identified as to what makes good plans. Four criteria are recommended; plans need to be comprehensive, developmental, student-centered and student-directed, and competency based.

Based on these criteria, one way to evaluate students' career plans is to judge the extent to which the activities included in the Individual Planning Component of the guidance program lead to the development of plans that meet these criteria. A second way is to make judgments about the adequacy of the plan contents. Finally, a third way is to judge their use. Do students actually use their career plans in planning for the future?SUMMARYIn order to fully evaluate comprehensive school guidance programs, three forms of evaluation are required. First, the program must be reviewed using program standards, evidence, and documentation to establish that there is a written guidance program in a school district and/or building and that the written program is the implemented program. Second, guidance program personnel need job descriptions derived directly from the program so that evaluation forms can be developed and used for formative and summative personnel evaluation. Third, results evaluation that focuses on the impact of the guidance and counseling activities in the guidance curriculum, individual planning, responsive services, and system support components of a comprehensive guidance program is mandatory.REFERENCESAmerican School Counselor Association (1986). Professional development guidelines for secondary school counselors: a self audit. Alexandria, VA: Author

American School Counselor Association (1990a). Professional development guidelines for elementary school counselors: a self audit. Alexandria, VA: Author

American School Counselor Association (1990b). Professional development guidelines for middle/junior high school counselors: a self audit. Alexandria, VA: Author

Gysbers, N.C., Lapan, R.T., Multon, K.D., & Lukin, L. (1992). Missouri guidance competency evaluation surveys, grades 4-6, 6-9, and 9-12. Columbia, MO: Center for Educational Assessment

Gysbers, N.C., & Henderson, P. (1994). Developing and managing your school guidance program (2nd ed.). Alexandria, VA: American Counseling Association

Johnson, C.D. (1991). Assessing results. In S.K. Johnson & E.A. Whitfield (Eds.), Evaluating guidance programs. Iowa City, IA: American College Testing Program

Lapan, R.T., Gysbers, N., Hughey, K., & Arni, T.J. (1993). Evaluating a guidance and language arts unit for high school juniors. Journal of Counseling and Development, 71, 444-451

Starr, M., & Gysbers, N.C. (1993). Missouri comprehensive guidance: a model for program development, implementation, and evaluation (Rev. ed.). Jefferson City, MO: Missouri Department of Elementary and Secondary Education

Norman C. Gysbers Ph.D. is a Professor in the Department of Educational and Counseling Psychology, University of Missouri-Columbia, Columbia, Missouri.

ERIC Digests are in the public domain and may be freely reproduced and disseminated. This publication was funded by the U.S. Department of Education, Office of Educational Research and Improvement, Contract No. RR93002004. Opinions expressed in this report do not necessarily reflect the positions of the U.S. Department of Education, OERI, or ERIC/CASS. Title: Evaluating School Guidance Programs. ERIC Digest. Document Type: Information Analyses---ERIC Information Analysis Products (IAPs) (071); Information Analyses---ERIC Digests (Selected) in Full Text (073); Descriptors: Counselor Evaluation, Educational Assessment, Educational Quality, Elementary Secondary Education, Evaluation Criteria, Formative Evaluation, Personnel Evaluation, Program Evaluation, School Counselors, School Guidance, Standards, Summative Evaluation Identifiers: ERIC Digests

###

[Return to ERIC Digest Search Page]