ORIGINS AND IMPACT OF CALIFORNIA STATE TESTING IN HISTORY-SOCIAL SCIENCE, THE

In October 1998 the California State Board of Education formally adopted rigorous academic content standards in history-social science for California public schools, K-12. Accompanying the adoption of similarly rigorous standards in the other three core content areas - language arts, mathematics and science - this Board action signaled the launch of a reform movement that has reverberated through the school districts of California. It was the beginning of an ambitious assessment and accountability program that would seek major upgrades in California student achievement and that promised to hold schools liable for inadequate progress in student performance. The purpose of state K-12 testing as it has evolved is to determine student achievement in relation to the rigorous state standards and to use that data to spur further improvement.

Prompting this massive effort that could alter curriculum and instruction practices on a wholesale scale, not only in California but nationwide, was a series of reports and studies in the past twenty years asserting that the performance of America's students was declining in comparison to those of other countries. Concurrently, business complaints that young adults entering the workforce lacked necessary fundamental skills, and university laments about the need to enroll a substantial number of freshmen in remedial classes fueled the fires of discontent. The popular impression grew that the nation's public schools were failing in their educational mission, and that the U. S. would suffer severe economic consequences as a result.

The critics of the educational system argued that schools and students needed tough, clear standards that precisely outlined what students were to know and be able to do, assessment programs that fairly and objectively measured student achievement, and accountability in the form of significant rewards and punishments for schools. Emphasis was also placed on ending social promotion and requiring a certain level of student performance to earn a high school diploma. As news reports about low performing schools became increasingly common and as people began to question what was being accomplished with the large amounts of money spent on public K-12 education, state and national politicians took notice. Standards-based educational reform became a prominent part of the political agenda.

Urged by the federal government and with strong bipartisan support, state legislatures instituted standards and assessment programs in virtually every state in the country. Though of varying quality and impact the programs were well received by a public that did not know how to evaluate public schools and welcomed the opportunity to see education held accountable. Increasingly subject to performance quotas, job evaluations, management reviews and the like in their work environments, people generally felt school administrators and teachers should be similarly treated. Although some education leaders questioned various facets of the standards and assessment programs, their analysis had little effect as the issue was no longer educational in nature - it had become an important public issue. Nonetheless, those leaders argued that low achievement by certain students was a function of factors outside school control, and that major additional resources would have to be devoted to help these students meet standards.

What occurred in California was similar to what was happening in other states, but with some distinctive elements peculiar to California. Superintendent of Public Instruction Delaine Eastin was an early advocate of standards-based reform, initiating what she called her "Challenge" program that encouraged school districts to work with the California Department of Education to create standards and apply them. However, Governor Pete Wilson, the State Board of Education (largely appointed by the governor), and the legislature believed stronger measures were necessary. Chapter 975, Statutes of 1995, became part of the public laws of California and went into effect in 1996. It established a state Commission for the Establishment of Academic and Performance Standards, and tasked it with developing and recommending to the State Board rigorous content standards in the four core academic areas. Wary of local efforts such as the Challenge standards, the Commission recruited a staff primarily from Virginia, where well-regarded, rigorous standards had been created earlier than in most other states.

The Commission began its work with the two disciplines regarded as the gateways to further educational attainment - language arts and mathematics. With the Board adopting standards recommended in those subjects in 1997, work then turned to history-social science and science. Science, like language arts and mathematics, generated substantial contention as the Commission heard testimony and proceeded with its deliberations, with passionate proponents and opponents of various educational practices assailing one another. Some people felt that similar rancor would arise as the Commission took up the development of history-social science standards. Fortunately, the acrimony that characterized debate in the other subject areas did not surface widely in history-social science.

What made history-social science different was the existence of a well-regarded curriculum framework, published as the History-Social Science Framework for California Public Schools, Kindergarten through Grade Twelve, that had been state policy since 1987. Unwilling to change the positive direction the framework represented, the State Board asked the Commission to refrain from any significant departures from the basic structure and content of that document. Although spirited discussion had taken place in the course of deliberations about the framework as it was being constructed in the mid-1980s, especially the decision to center the framework in the discipline of history and to proceed with a sequential curriculum, by 1998 much effort had been invested in adapting to the framework and many teachers and administrators resisted any substantial change. Moreover, the principal history-social science teacher organization, the California Council for the Social Studies (CCSS), took an active role in advising the Commission and advocated using the framework as the guiding element in developing the standards.

As a result, the standards that emerged from the Commission's staff work reflected the grade level designations, organization and outlook of the framework, though there were some additions and emphases in the standards that reflected the east coast orientation of the drafters. Efforts by CCSS members and others to imbue a stronger California tone into the standards and to cut some of the overly detailed requirements were partially successful. After adoption by the State Board of Education in October 1998, implementation efforts began. One national evaluating group, the Fordham Foundation, labeled the California standards among the best in the country. A series of publications called Course Models, originally designed to offer teachers model lessons in support of the framework, were expanded and aligned to the standards and converted to a strictly online format (available at: www.history.ctaponline.org). Over time, other web resources were developed to assist teachers with the standards. These include the California Learning Resource Network (www.clrn.org/hss/), the California Department of Education history-social science site (www.cde.ca.gov/statetests/history), and the Schools of California Online Resources for Education site (http:// score.rims.k12.ca.us).

By the time the history-social science standards were adopted in California, however, the state assessment program that was ostensibly linked to the standards was already underway. In actuality, the early stages of the testing program had no relationship to the standards. Unwilling to leave testing implementation to the next administration, Governor Pete Wilson wanted to use an off-the-shelf national norm-referenced test to implement the program while the specific standards-based tests (eventually known as California Standards Tests) were being developed. The State Board wanted to utilize only one contractor for both purposes, and chose Harcourt Educational Measurement with its national SAT-9 test.

Entitled the STAR, or "Standardized Testing And Reporting," Program, the first round of tests was administered in Spring 1998. In history-social science, grades 9, 10 and 11 were tested. Generally, the historysocial science scores in 9th and 10th grades on the SAT-9 reflected the same performance as that in the other subject areas - slightly below the national average. On the 11th grade SAT-9, though, a curious phenomenon occurred. California's 11th grade history-social science students bucked the trend, scoring approximately seven points above the national average, the only grade level and subject area in the state to score that high.

This 11th grade result may be a function of the many years of teaching to California's curriculum framework, which relies on a sequence of courses that do not overlap much (ancient history in 6th grade, medieval history in 7th grade, etc.), rather than a layered curriculum common in many other states where the same subjects are covered in multiple grades. Nationally, it is likely that more children were exposed to a broader surface of social studies material at earlier grades than in California, so they recognized more of the items on the 9th and 10th grade SAT-9. But by the end of 11th grade, it seems that California's students finally encountered the same range of material, but had studied it in greater depth, and that may account for the markedly improved performance exceeding the national average in 11th grade. It also suggests that California's framework and standards provide productive curriculum direction.

The norm-referenced test in history-social science was dropped after 2002 when the History-Social Science Standards Tests were fully in place. The SAT-9 and other norm-referenced tests in history-social science were so far removed from the state standards that they were not seen as offering much useful data. This change coincided with an overall reduction in the state emphasis on normreferenced tests, and a corresponding hike in the importance of the standards tests, with the source data for the Academic Performance Index (API), which ranks schools according to state test performance, moving in the direction of the standards tests. Initially based entirely on the SAT-9, history-social science was assigned 20% of the weighting in the high school API.

Beginning in 2003, only the standards tests were used to calculate the high school history-social science portion of the API. The history-social science standards tests were launched in 2001 and applied to the API base in 2002. Originally the tests were given in grades 9, 10 and 11, with the standards for grades 10 and 11 forming the basis for those examinations, respectively. In grade 9, however, there was a glitch in the law. It required a historysocial science standards test in grade 9 once standards for the grade were established. But, consistent with the framework, the State Board left the 9th grade an elective year and instituted no history-social science standards for it. A comprehensive test in 9th grade covering standards from grades 4-8 was begun, but in 2003 was converted to an 8th grade test covering only grade 6-8 standards to help establish middle school accountability. The middle school history-social science component of the API will be based entirely on the standards tests, with the amount of weighting still under consideration by the State Board. Base line data for the middle school history-social science API inclusion will be established with the 2004 test.

The California Standards Tests (CSTs) are the centerpiece of history-social science state assessment; in fact they are now the only state history-social science tests, since the Golden State Examinations (GSE) were eliminated in the deficit-driven budget of 2002-03. Unlike the former testing programs such as CLAS and GSE, where the state had a greater direct role, the Standards Tests are prepared and administered by a contractor working to state specifications. The first contractor was Harcourt Educational Measurement, followed in 2002 by the current contractor, Educational testing Service (ETS). The state monitors the work of the contractor to ensure that its specifications are met. Policy for the testing program is determined by the State Board of Education and announced in documents like the test blueprints that appear on the CDE web site (www.cde.ca.gov/statetests). Performance levels for each of the tests were adopted by the State Board upon recommendation of a special panel convened in 2001 that consisted largely of teachers.

Test items are developed by the contractor, who retains California teachers to write the items whenever possible. Thereby the tests are hooked in part to the realities of the classroom. Once the items are submitted, the contractor undertakes an extensive internal review and editorial process to ensure that the items are aligned with the standards and that the standards identification is correct, that the items are accurate and reflect modern scholarship, that the items are understandable and intellectually accessible to all students at the intended grade level, and that they do not discriminate against any group of students or contain any offensive properties.

The items are then brought to a state Content Review Panel (CRP) for their scrutiny. The panels, appointed by the State Board, exist in each subject area and are largely composed of higher education faculty and teachers appropriate to the grade level. They also contain a few public members and administrators, with north-south participation, gender/ethnic diversity, and subject matter specialization being taken into consideration. The CRP recommends an exam blueprint to the State Board, reviews all prospective test items and forms, and recommends those items to be used on the tests. Their evaluation is based on close alignment to the standards and other factors cited above. The 10th and 11th grade tests each are comprised of 60 items, plus six embedded items in different versions to pilot next year's items. The middle school grade 6-8 test, given in 8th grade but covering the standards for grades 6, 7 and 8, has 75 items (plus the six pilot items) because it is a multi-grade test.

Before any test can finally be administered, the items must also be reviewed by a statutory Statewide Pupil Assessment Review (SPAR) panel. This panel is required to contain a majority of parents whose children attend public schools and, according to the law, it must ensure that nothing on the exam contains "any questions or items that solicit or invite disclosure of a pupil's, or his or her parents' or guardians' personal beliefs or practices in sex, family life, morality, or religion nor shall it contain any question designed to evaluate personal behavioral characteristics, including, but not limited to, honesty, integrity, sociability, or self-esteem."

All of the California Standards Tests are administered each year within a 21-day window of 85% of the school year plus or minus 10 days. Effectively, this means that students may take the examination with only 80% of the year's instruction complete, or as much as 90%. For history-social science the difference might be significant since the tests include items from all of the standards, and greater instructional time implies more extensive standards coverage. However, some teachers feel that if the students take the history-social science test later in the window there could be a more pronounced burn-out effect. Although State Board policy is to test all of the standards, CRP members are deeply conscious of classroom pacing and they make every effort to approve only those items for use on the test that are likely to have been covered.

Scoring on the history-social science standards tests are entirely done by computer, since all of the items are multiple-choice. The STAR program contains only two writing samples, given in language arts at grades 4 and 7. With good reason, many history-social science teachers advocate essay examinations for students in order that they may demonstrate achievement in higher order thinking skills necessary to full civic literacy. Because of the cost and complexity of scoring such examinations, though, essays are not a part of the state history-social science tests. Nor in the state budget climate of the 200Os are they likely to be instituted. Historical and social science analysis skills are specified in the standards, and are not ignored in the tests. The contractor is required to include an element of these skills in 25% of the test questions, always conjoined with a content question. Often such items involve an analysis requirement, such as interpreting a map or a primary source excerpt.

The scores on the California Standards Tests to date do show improvement. Spanning the three years the tests have been given, scores in grades 10 and 11 history-social science have moved upward, with three percent more students attaining the category of "proficient" in 2003 as opposed to 2001 (mobility in grade 8 scores cannot yet be measured since the grade 8 test was not given before 2003). The State Board has established five performance levels: advanced, proficient, basic, below basic and far below basic. As of the 2003 results, 34% of students in grade 11 scored at "proficient" or above, with 27% of grade 10 students and 28% of grade 9 falling into that category. Scores are reported on a 150-600 scale, with 350 being required to attain a "proficient" score. In general, a student must choose the correct response on approximately 60% of the items to attain the "proficient" level. Each test varies slightly, and the exams are equated for difficulty year by year so that true improvement can be ascertained. The performance of specific demographic groups can also be studied. State, county, district and school score totals are posted for each year on the web site: www.cde.ca.gov/statetests.

Teachers and administrators are understandably concerned about student readiness for the STAR tests. Specific test preparation, such as trying to coach students only on material they think will comprise test items is barred by State Board rule. But there are appropriate measures that can be undertaken. In addition to reports to parents for each student and the state reports on the web site, individual student reports and student master list summaries are sent to each school. The reporting cluster totals on these reports can reveal relative strengths and weaknesses in instruction. Each of the history-social science tests is divided into five clusters representing a standards category, such as, in grade 10, "Industrial Expansion and Imperialism," or "Causes and Effects of the First World War." Teachers can accumulate totals for their classes and find out where student performance was better, thereby helping them to determine where greater instructional emphasis might be placed. Administrators can do this for the school as a whole.

The most effective way to address the state test is to organize instruction to meet the standards, and to apply thorough and consistent multiple measures of assessment in the classroom so teachers can determine along the way if students are actually learning what they are covering. In that way, the outcome rather than the input drives instructional practice. This cannot be overemphasized. As a study on reading in 32 Bay Area schools suggests, "researchers found that schools where black and Latino students' test scores were rising did many things differently from the lower-achieving schools. Most notably, teachers diagnosed students' needs a few times each week then changed how they worked with the kids based on what the data revealed." The article, appearing in the San Francisco Chronicle on December 18, 2003 also reported that "at the higher-achieving schools, teachers were more likely to learn how to analyze data and apply it to teaching ... principals usually considered closing the achievement gap a primary goal, more people of color held leadership positions, and school goals were usually clear and focused."

Extensive assessment opportunities in the classroom, soundly grounded in the state standards, results in students being more ready for the state test. It is difficult for individual teachers to do this on a solitary basis. This approach happens best when undertaken by groups of teachers in a school, working with support of the school administrators. In that way teachers can jointly examine the standards and bring their minds collectively to bear on standards' emphases, classroom pacing, teaching resources and, especially, collaborating to devise assessments for the classroom that will provide the data they need to determine what the students are learning and what they are not, and thereby make instructional remedies at the time they are most needed. Among the resources teachers can draw upon for sample assessments are those provided by publishers, county offices and others, including newer, more easily useable electronic test generators.

The state tests in history-social science are but a snapshot of student performance. Yet their impact already extends far beyond the few hours they take each year. The tests focus keen attention on the standards and provide the foundation for accountability. They have compelled textbook publishers to orient their offerings to the standards, thus providing better resources for both students and teachers. They can encourage teachers to extend their professional knowledge, to form cooperative relationships with their colleagues, and to articulate connections with those at other school levels. They provide data that can lead to improvements in instruction and, most critically, they can impel the more effective use of classroom assessments. They have driven the creation of many teacher resources now aligned to the standards, not only from publishers but also from museums, archives, libraries, education institutes and web site developers. They are slowly working their way into teacher preparation programs, spurred in part by the California Commission on Teacher Credentialing's creation of new program standards based on the state student standards.

Unfortunately, there are always some individuals who attempt to "gimmick" the system. In order to try to boost language arts or math scores on the state tests, administrators in some elementary and even middle schools have reportedly eliminated history-social science entirely, even refusing the readily available opportunity to have the students learn reading partially through history-social science content (voluminous reading lists, called Pages of the Past, Grades K-6 and Tales of Time, Grades 6-8 have been produced by the Curriculum and Instruction Steering Committee of the California County Superintendent Educational Services Association). Students in these schools will be woefully unprepared for the subject matter in history-social science they will later be required to master. Moreover, such shortsighted maneuvering may condemn some of these students to such an incomplete knowledge of history, government, geography and economics that their ability to function as fully capable adult citizens will be severely jeopardized.

There are also some people who claim to be able to identify only "essential" history-social science standards to be covered on the test while eliminating teaching of the rest. This recipe for selective ignorance will not work. By State Board rule, all of the standards are tested over time and no one can predict any sequence. In addition, "cherry-picking" certain standards to teach can constrict the development of the full spectrum of civic knowledge and skills a student will need as an adult to be successful. It is certainly permissible and even advisable for teachers to structure the standards for instruction in a way that best suits their style, and to emphasize selected standards that provide the backbone for central themes. But all of the standards should, in one fashion or another, be treated; none should be totally disregarded.

Government at both federal and state levels continues to confront the pressing need to provide adequate resources to ensure that students from backgrounds that are less educationally advantaged can become proficient in the standards. This looms as a significant political issue in the years ahead. But the commitment to standards, assessment and accountability is not likely to evaporate, given its broad political and public appeal. Teachers, administrators and students all benefit when data drives instruction, and strong assessment programs provide that data. The September 2003 newsletter of the Center for Performance Assessment sums it up: "The most effective schools do not engage in data analysis as an annual review of test scores, but as a continuing endeavor. It is simply the way they do business, referring to external and internal data, finding their most effective instructional practices, and using data to provide accurate and effective feedback for teachers .... We have nothing to fear from educational accountability that is accurate and constructive."

John F. Burns, a frequent contributor to Social Studies Review, has authored and edited numerous publications including Taming the Elephant: Politics, Government and Law in Pioneer California (co-editor, 2003), the history chapter in Courthouses of California (2001), and Sacramento: Gold Rush Legacy, Metropolitan Destiny (editor, 1999). Forthcoming work includes a book chapter on veterans of the Vietnam War and a modern history of Sacramento, both expected in 2004. He is history-social science consultant and coordinator of STAR test development for the California Department of Education. Previously he served as State Archivist of California, museum director, teacher and adjunct professor, consultant for a variety of media, and a U. S. naval officer. He was recognized with the Diane L. Brooks Award from the California Council for the Social Studies and the California Military History Medal among other awards.

Copyright California Council for the Social Studies Spring 2004
Provided by ProQuest Information and Learning Company. All rights Reserved