Examining the influence of a curriculum-based elementary mathematics professional development program.
Polly, Drew ; Wang, Chuang ; McGee, Jennifer 等
This study presents findings from the first cohort of teachers in a
U.S. Department of Education Mathematics Science Partnership (MSP) grant
designed to support the use of a standards-based elementary school
mathematics curriculum, Investigations in Number, Data, and Space
(Investigations). In line with the goals of the MSP program, the 84-hour
professional development program focused on building teachers'
knowledge of mathematics content, examining how the mathematics content
is embedded into curriculum, and supporting teachers' enactment of
reform-based pedagogies. Teacher participants had a positive gain in
their content knowledge, but this increase did not have any
statistically significant impact on student gains in the assessment of
mathematics proficiency. Results about teacher beliefs were
inconclusive, as more time is needed to change teacher beliefs. Teachers
who changed their practices from teacher centered to student centered
found their students with statistically more gains in their performance
in curriculum-based mathematics assessments. Discussions and
implications of these findings were also presented.
Keywords: professional development, mathematics, curriculum,
elementary education
**********
Numerous studies have provided empirical evidence related to the
influence of mathematics instruction on student achievement measures and
students' understanding of mathematics concepts (e.g., Heck,
Banilower, Weiss, & Rosenberg, 2008; Hiebert & Stigler, 2000;
Stigler & Heibert, 1997; U.S. Department of Education [USDE], 2008).
Students demonstrate deeper mathematical understanding when their
teachers pose mathematically rich tasks, allow students to explore
concepts, facilitate students' connection between mathematical
ideas, and provide opportunities for students to communicate their
mathematical thinking (e.g., Heck et al., 2008; Smith & Smith, 2006;
Tarr, Reys, Reys, Chavez, & Shih, 2008). These pedagogies are
frequently referred to as standards-based pedagogies (e.g., Bailey,
2010; National Council for Teachers of Mathematics [NCTM], 2000).
In the 1990s, the National Science Foundation funded the
development of standards-based curriculum to provide students with
greater access to standards-based pedagogies (Goldsmith, Mark, &
Kantrov, 2000). These curricula were developed with the intent to build
students' conceptual understanding of mathematics concepts by
providing ample opportunities to explore content by completing rich
mathematical tasks and communicating about the mathematics in the tasks.
Although access to standards-based curricula supports teachers'
instruction and may influence student achievement, curricula enactment
is influenced by various teacher factors (Drake & Sherin, 2006;
Remillard, 1999); and the curricula alone cannot cause gains in student
achievement (National Research Council, 2004).
Various researchers have found issues during the implementation of
standards-based curricula. Frequently, teachers modify standards-based
curricula in ways that decrease the rigor or difficulty of the tasks
(Stein & Kim, 2009; Stein, Remillard, & Smith, 2007; Sutherland
et al., 2006). When tasks are modified, they resemble computational
exercises found in traditional curricular materials that are more
teacher centered (Stein et al., 2007). This dilemma of decreasing the
rigor of tasks also existed before standards-based curricula. Teachers
modified standards-based tasks and enacted either skill-based tasks or
guided students so much that students were required to only follow along
with the teachers' work (Cognition and Technology Group at
Vanderbilt [CTGVJ, 1997; Doyle, 1988; Henningsen & Stein, 1997).
Remillard's (2005) large-scale research synthesis found that
teachers' curricula implementation were linked to teacher
characteristics, such as mathematics content knowledge, beliefs about
mathematics teaching and learning, and support from their administrators
and colleagues. Further, Drake and Sherin (2002) found that
teachers' identities as learners of mathematics help to explain
teachers' instructional practices while implementing
standards-based curricula. For teachers to effectively enact
standards-based pedagogies, there is a need to provide ongoing learning
opportunities that influence teachers' knowledge, beliefs,
instructional practices, and also their students' achievement
(Darling-Hammond, Wei, Andree, Richardson, & Orphanos, 2009).
INFLUENCE OF PROFESSIONAL DEVELOPMENT
Characteristics of Learner-Centered Professional Development
Large-scale syntheses of professional development research (e.g.,
Darling-Hammond et al., 2009; Loucks-Horsley, Stiles, Mundry, Love,
& Hewson, 2009; Yoon, Duncan, Lee, Scarloss, & Shapley, 2007)
have identified key components of effective professional development
projects. These include addressing deficits in student learning outcomes
(Loucks-Horsley, Stiles, Mundry, Love, & Hewson, 2010; Wilson &
Berne, 1999); providing teachers with ownership of their professional
development activities (Loucks-Horsley et al., 2010; Polly &
Hannafin, 2010); promoting collaboration among teachers, administrators,
and others (DuFour & Eaker, 1998; Penuel, Fishman, Yamaguchi, &
Gallagher, 2007); developing teachers' knowledge of content and
pedagogy (Garet, Porter, Desimone, Birman, & Yoon, 2001; Heck et
al., 2008; Supovitz, 2003); supporting reflection and connections to
teachers' classroom practices (Desimone, 2009; Guskey, 2003); and
including ongoing support through workshops and classroom-embedded
experiences (e.g., Loucks-Horsley et al., 2010; Polly & Hannafin,
2010). In essence, these reforms embody a learner-centered approach to
professional development, which is grounded in the American
Psychological Associations' Learner-Centered Principles (Alexander
& Murphy, 1998; APA Work Group, 1997) and addresses teachers'
individual needs and learning goals (National Partnership for Excellence
and Accountability in Teaching [NPEAT], 2000; Polly & Hannafin,
2010).
Mathematics Professional Development Projects
Numerous mathematics professional development projects have been
implemented that embody characteristics of learner-centered professional
development (Borko, 2004; Darling-Hammond et al., 2009). The Cognitively
Guided Instruction (CGI) project provided elementary schoolteachers with
professional development about how children learn mathematics, how to
examine students' mathematical thinking, and how to pose specific
types of tasks to support children's development of mathematical
concepts (Carpenter, Fennema, Peterson, Chiang, & Loef, 1989;
Fennema et al., 1996). Researchers found that students whose teachers
participated in the CGI project outperformed their peers on a
problem-solving assessment. Students in CGI classrooms also reported
that they were more confident in doing mathematics than their peers in
non-CGI classrooms. The New Zealand Number Development Project found
that intensive, multiyear professional development about number sense
and how to address students' misconceptions of number and place
value led to teachers' improved instructional practices and
moderate gains in student learning outcomes (Higgins & Parsons,
2010). Another professional development project provided opportunities
for teachers to videotape their teaching and then meet with their peers
and a professional developer to discuss their teaching, their
students' learning, and plan future lessons. Research about these
"video clubs" indicates that teachers' instruction
includes more standards-based pedagogies (van Es & Sherin, 2008).
Although a myriad of other professional development projects have
examined the influence of teacher learning opportunities on
teachers' instruction and student learning outcomes, the literature
lacks studies focused on professional development that is curriculum
based and focused on supporting curriculum implementation. The
professional development project, funded by the U.S. Department of
Education Mathematics Science Partnership initiative, was designed to
support kindergarten through 5th-grade teachers' implementation of
standards-based mathematics curriculum, Investigations in Number, Data,
and Space (Russell & Economopolous, 2007). This study shares the
findings from the first year of the 3-year project.
This study was driven by the following research questions:
* To what extent did professional development influence
teacher-participants' beliefs about mathematics teaching and
learning?
* To what extent did professional development influence
teacher-participants' reported instructional practices?
* To what extent did professional development influence student
learning outcomes on curriculum-based assessments?
METHOD
Professional Development Context
The professional development project was part of a Mathematics
Science Partnership (MSP) grant funded by the state Department of
Education. Teachers from two school districts participated in the
professional development. District A is a large urban district that
contains 102 elementary schools (Grades K-5) and employs more than 3,600
elementary schoolteachers. District B is a small suburban district that
contains five elementary schools (Grades K-4) and one intermediate
school (Grades 5-6). Both districts are considered impoverished by the
state's guidelines; 75% of the schools in District A qualify for
Title I funding, whereas all of the schools in District B qualify for
Title I funding.
Although the professional development materials were identical for
both districts, each district conducted their own workshops. For all
professional development, kindergarten through 5th-grade teachers met
together, though time was given for teachers to work in small groups to
examine state standards and the Investigations materials for their grade
level. Both school districts were new to Investigations, though a few
teachers had used individual units or lessons in previous years.
Teachers selected for participation in the MSP grant were given 84
hours of professional development between July 2009 and April 2010. The
professional development was designed to include components of
learner-centered professional development (NPEAT, 2000; Polly &
Hannafin, 2010). During the summer of 2009, teachers completed a 60-hour
summer institute, in which they completed mathematical tasks focused on
number sense and algebra, examined how mathematical concepts were taught
in the curriculum, and developed skills related to posing questions to
examine students' mathematical thinking. Teachers also shared ideas
and concerns about teaching with the Investigations curriculum. During
the school year, teachers attended four 6-hour follow-up sessions
focused on issues related to teaching with Investigations. The primary
focus of the follow-up sessions were pedagogical issues, such as setting
up and teaching small groups, supporting struggling students while they
were exploring tasks, and facilitating class discussions about
mathematics concepts.
Participants
Fifty-three K-5 teachers from two school districts in a
southeastern state of the United States participated in a 3-year MSP
grant designed to support the use of a standards-based mathematics
curriculum. All teachers were certified to teach elementary schools.
Thirty-two teachers were from a large urban school district (District A)
and the remaining 21 teachers were from a neighboring suburban school
district (District B). Thirty-seven percent (n = 20) held a
bachelor's degree, 30% (n = 16) held a master's degree, and
one teacher held a bachelor's degree and certification specific to
the content area. Eighty-seven percent (n = 46) identified as White,
whereas 13% (n = 7) identified as African American.
Participants also included 688 students. Fifty percent (n = 344) of
the students were female and 50% (n = 344) were male. Thirty-nine
percent (n = 268) of the students were White, 34% (n = 234) were African
American, 20% (n = 138) were Hispanic, 4% (n = 28) were Asian, and 3% (n
= 21) were identified by their teachers as "Other." Fourteen
percent in = 96) were identified as Limited English Proficient (LEP) and
10% (n = 69) were identified as having Individualized Education Plans
(IEPs). Due to attrition and missing data, the actual sample sizes for
student participants were 629, 542, and 450, respectively, for the three
rounds of assessments.
Procedures
Teacher beliefs instrument. All teacher-participants completed
three pre- and post-instruments: a Teacher Beliefs Questionnaire (TBQ;
Figure 1), a Teacher Practices Questionnaire (TPQ; Figure 2), and a
Content Knowledge for Teaching Test. The TBQ examined teachers'
espoused beliefs about mathematics as a subject, how mathematics should
be taught, and how students learn mathematics (Swan, 2006). For each of
those three dimensions, teachers reported the percentage to which their
views align to each of the transmission, discovery, and connectionist
views. The sum of the three percentages in each section is 100. Teachers
were coded as discovery/connectionist if they indicated at least 45% in
either discovery or connectionist. The categories of discovery and
connectionist were merged because both of these two teacher orientations
were associated with student-centered practices, whereas the
transmission orientation was associated with teacher-centered practices
(Swan, 2007). The difference between postsurvey and presurvey was used
to indicate the change of teacher beliefs after participation of the
(professional development) program. Participants who expressed no
preferences (e.g., 30%, 30%, & 40%) and those who did not show a
change in their beliefs were treated as the reference group in data
analysis.
Teacher practices instrument. The TPQ examined participants'
self-report about instructional practices related to their mathematics
teaching (Swan, 2007). Each of the 25 items reflects either
student-centered or teacher-centered pedagogies. Teachers identified the
extent they used those instructional practices by rating each item on a
5-point Likert-type scale, where 0 represents none of the time and 4
represents all of the time. Of the 25 items, 13 (Items 1, 2, 3, 4, 8, 9,
10, 13, 14, 18, 19, 22, & 23) were indicators of teacher-centered
practices, whereas 12 (Items 5, 6, 7, 11, 12, 15, 16, 17, 20, 21, 24,
& 25) were indicators of student-centered practices. The 13 items
related to student-centered practices were reverse-coded so that
participants whose average response to all 25 items of 2.00 or less were
coded as "student centered" and participants with a mean score
of 2.01 or more were coded as "teacher centered." The
difference between postsurvey and presurvey was used to indicate the
change of teacher practices after participation of the PD program. The
Cronbach's alpha reliability coefficient was .79.
Content knowledge for teaching mathematics. The Content Knowledge
for Teaching Mathematics assessments were developed as part of the
Survey of Instructional Improvement at the University of Michigan. This
project used the Elementary Number and Operations assessments that
measure teachers' knowledge of mathematics content and knowledge of
students and content (e.g., knowledge related to pedagogy). There are 30
questions designed to measure content knowledge in number and
operations, patterns, functions and algebra, and geometry, but a total
of 62 items. Five forms of the teacher content knowledge test were
piloted in Mathematics Professional Development Institutes in California
from 2001 to 2003, and new forms were built in 2004. The latest version
of Learning Mathematics for Teaching Assessment (LMT; Form A04) was used
in this project. The reliabilities for subtests for each content
knowledge ranged from .72 to .83, and the content validity of the
instrument was guaranteed by piloting, revising, deleting, and adding
new items with consultation from experts in mathematics education and
practitioners. One point was given to each item, so the total possible
score for this test ranges from 0 to 62. These scales have been tested
for validity and reliability and have been employed to empirically link
teachers' knowledge to student achievement (Hill, Rowan, &
Ball, 2005).
Student achievement assessments. The student achievement measures
used in this study were end-of-unit assessments from the Investigations
in Number, Data, and Space (Russell & Economopolous, 2007)
elementary mathematics curricula. Three units were assessed from each
grade level, and each unit lasted between 3 and 5 weeks. Teachers
administered these assessments before teaching the unit (pretests) and
immediately after completing the unit (posttests). One of the project
evaluators used a standard rubric to score each assessment and converted
the score to a percentage that reflects the percent of problems a
student solved correctly. Gain scores were used in the analyses.
Descriptive statistics were used to report the number of teachers
with each category of teacher beliefs and practices at the beginning and
end of the first year. Comparison of the frequency of these categories
was used to look for the change of teacher beliefs and practices.
Independent sample t test was employed to examine the difference in
teacher's content knowledge between the two school districts. The
impact of the change of teacher content knowledge, beliefs, and
practices on the change of student learning outcomes in mathematics was
explored with hierarchical linear modeling (HLM). Two-level HLM was used
because the student-level variable (gain scores in mathematics
assessment) was nested within teacher-level variables (change in content
knowledge, beliefs, and practices). The average number of students for
each classroom is 20. Unconditional models were run to see if there was
enough variance to be explained with additional variables of interests
and to help calculate intraclass correlation coefficients (Raudenbush
& Bryk, 2002). Because we were interested in the impact of
teacher-level variables on student-level variables at each stage of the
program (three rounds of assessments), an HLM was tested for each round
of student assessment. Bonferroni correction ([[alpha].sub.i] =
[[alpha].sub.fw]/p where [[alpha].sub.fw] is the family-wise error rate
(.05), [[alpha].sub.i] is the individual-test error rate, and p is the
number of tests) was applied to control possible inflation of Type-I
error with multiple HLMs. For the sake of space, the conditional model
for the first round of student assessment is presented as follows (the
conditional HLM for the second and third rounds of student assessments
is the same except that the dependent variable becomes the gain scores
during the second and third rounds, respectively):
Level 1 (no student-level variables were considered as level-1
predictors):
[Y.sub.ij] = [[beta].sub.0j] + [r.sub.ij]
where,
[Y.sub.ij] = the dependent variable that represents the gain in
mathematics skills during the first round of assessment for an
individual student i in the classroom of teacher j
[[beta].sub.0j] = the true mean student gain in mathematics skills
during the first round of assessment in the classroom of teacher j
[e.sub.i] = a random error.
Level 2:
[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]
where,
[[gamma].sub.00] = the grand mean gain of all students in the first
round of assessment
[[gamma].sub.01] = the unique effect on student gains in
mathematics skills associated with teacher beliefs in teaching
mathematics
[[gamma].sub.02] = the unique effect on student gains in
mathematics skills associated with teacher beliefs in learning
mathematics
[[gamma].sub.03] = the unique effect on student gains in
mathematics skills associated with teacher beliefs in mathematics
[[gamma].sub.04] = the unique effect on student gains in
mathematics skills associated with teacher practices in the classroom
[[mu].sub.0j] = the residual teacher-specific effects and was
assumed normally distributed with mean 0 and variance [[tau].sub.00]
RESULTS
Change of Teacher Beliefs and Practices
Thirty-eight teachers completed TBQ and TPQ at the beginning and
end of the year. Results from the TBQ showed that from the beginning to
the end of the first year, seven teachers changed their beliefs about
teaching mathematics from disco very/connectionist orientation to
transmission orientation, whereas 22 remained unchanged. Of those 22
that were unchanged, 19 of the 22 were discovery/connectionist prior to
the project. Meanwhile, nine teachers changed their beliefs about
teaching mathematics from transmission orientation to
discovery/connectionist orientation. As for their beliefs toward
learning mathematics, 26 teachers remained unchanged, whereas eight
teachers changed from discovery/connectionist orientation to
transmission orientation and four teachers changed from transmission
orientation to discovery/connectionist orientation. Of the 26 that
remained unchanged in their views of learning mathematics, 20 of the 26
were already discovery/connectionist. With respect to teachers'
beliefs about mathematics, 25 teachers remained unchanged, whereas eight
teachers changed from discovery/connectionist orientation to
transmission orientation and five teachers changed from transmission
orientation to discovery/connectionist orientation. Of the 25 teachers
who remained unchanged, 22 were already discovery/connectionist.
As for teacher practices measured by TPQ, 40 teachers were
identified with student-centered classroom practices and 12 teachers
were identified with teacher-centered classroom practices at the
beginning of the year. Only 39 out of the 52 teachers completed the
postsurvey at the end of the year. Out of these 39 teachers, 34 were
identified with student-centered classroom practice, whereas five were
identified with teacher-centered classroom practice. Comparisons between
the pre- and postsurvey indicated that 29 teacher-participants remained
student centered, five changed from teacher-centered to student-centered
classroom practices, and five remained teacher centered.
Change of Mathematical Content Knowledge for Teaching
The Content Knowledge for Teaching Mathematics test was completed
by 35 teacher-participants at the beginning and end of the year. Gain
scores were computed by subtracting pretest scores from posttest scores.
The mean of the gain scores was 2.34 with a standard deviation of 5.37.
The minimum gain score was -12.00, and the maximum gain score was 11.00.
When the gain scores were compared between the two school districts, the
mean gain score for teachers in School District A (n = 17, M = 3.82, SD
= 4.20) was not statistically significantly different from that for
teachers in School District B (n = 18, M = .94, SD = 6.07), t(33) =
1.62, p = .11. The effect size measured by Cohen's (1988) d was
medium (d = .55).
Influence of Professional Development on Student Achievement
Student assessment gain scores (posttest minus pretest) are
presented in Table 1. Although there is a general trend of increasing
average gains from the first round to the third round as the
teacher-participants had spent more time in the professional
development, some negative gains were also observed.
The intraclass correlation coefficients were .41 for the first
round of student assessment, .37 for the second round of student
assessment, and .46 for the third round of student assessment,
justifying the need to use HLM because nearly 40% of the variability of
student gains in mathematics skills can be explained by teacher-level
variables. The estimated reliability for classroom mean gains on
mathematics skills were .93 for the first round, .89 for the second
round, and .91 for the third round of student assessment. Parameter
estimates of HLMs are presented in Table 2. The gain of teacher content
knowledge in mathematics was not statistically significantly related to
student gains during the second or third round but was negatively
associated with student gains in the first round. This means that
students taught by teachers with relatively less content knowledge at
the beginning, who gained more content knowledge through the
professional development, had relatively less gains in curriculum-based
assessments at the beginning of the professional development. This
difference, however, diminished as teachers were more exposed to the
professional development. Similarly, teachers who changed their practice
from teacher-centered to student-centered at year-end found their
students had relatively less gains at the beginning of the year. This
difference also diminished as they were more exposed to the professional
development.
Teachers who changed their beliefs about teaching mathematics from
discovery/connectionist orientation to transmission orientation found
their students had relatively more gains than students taught by other
teachers during the third round of assessment, which occurred in March
of the school year. This difference was not statistically significant
during the first and second rounds of assessment. This finding is
contradictory to our predictions, as the professional development is
designed to help teachers develop more discovery/connectionist
orientation of beliefs in teaching. We were expecting more student gains
for teachers who changed from transmission orientation to
discovery/connectionist orientation. Consistent with this finding was
that students taught by teachers who changed their beliefs about
mathematics and learning mathematics from transmission to
discovery/connectionist orientation had relatively less gains than
students taught by other teachers during the second round of assessment.
DISCUSSION AND IMPLICATIONS
Discussion of Findings
Teacher beliefs and practices. In terms of reported instructional
practices, prior to the professional development, 40 of 52 teachers
(76.92%) were student centered. By the end of the project, 34 of the 39
teachers (87.19%) reported using primarily student-centered pedagogies.
A majority of the professional development (more than 60 of the 84
hours) focused on effective implementation of Investigations and related
practices, which may explain the shift toward student-centered
pedagogies. Prior pedagogical-specific professional development projects
have found slight increases in teachers' practices within the first
year (Garet et al., 2001; Heck et al., 2008; Penuel et al., 2007), but
more significant increases after 2 years of comprehensive professional
development (Fishman, Marx, Best, & Tal, 2003; Penuel et al., 2007).
The results were more mixed with teacher beliefs, as teachers
either remained constant, became more transmission (teacher centered),
or became more connectionist/discovery (student centered). At the
beginning of the project, most of the teacher-participants reported
having discovery/connectionist beliefs in each of the three constructs
(teaching math, learning math, and math as a subject). Most participants
still reported having discovery/connectionist beliefs at the end of the
project. However, there is concern that teachers moved toward
transmission views of mathematics teaching and mathematics learning.
In our earlier work (McGee, Wang, & Polly, 2013), data analyses
indicated a lot of teacher apprehension about whether these pedagogies
will lead to student achievement, especially on standardized
assessments. Upon further examination of teachers who moved to
transmission views of mathematics, most of these participants taught
Grades 3, 4, or 5. One possible explanation for this was that
teacher-participants in Grades 3 through 5 were 3 weeks away from
administering the high-stakes state test, and both districts used a lot
of teacher-directed, transmission-like materials to prepare for this
test. For teaching mathematics, the number of Grades 3, 4, or 5 teachers
who moved to transmission views were 6 of 7 (84.3%) for teaching
mathematics, and 7 of 8 (87.5%) for learning mathematics and mathematics
as a subject. Perhaps, giving the postsurveys at a time farther away
from the state tests may have led to different results.
Previous studies noted that in some cases teachers required many
years to work on shifting their instructional practices before shifting
their beliefs (Fennema et al., 1996; Penuel et al., 2007). In the
seminal work of the CGI project, researchers (Fennema et al., 1996)
concluded that some teachers experienced a change in beliefs during
workshops and then implemented new pedagogies, whereas others
experimented with new pedagogies in their classroom before shifting
their beliefs. In our earlier work (McGee, Wang, & Polly, 2013),
data analysis of classroom observations shows a high degree of fidelity,
indicating that teachers fit into the second category that Fennema et
al. (1996) described. Teachers were more willing to use Investigations
and experiment teaching lessons before their beliefs shifted. The
project's focus on instructional practices and using the curriculum
might be a possible cause for this finding. Regardless, data from this
study support prior work (e.g., Banilower, Boyd, Pasley, & Weiss,
2006; Fishman et al., 2003; Heck et al., 2008; Orrill, 2001; Richardson,
1990) showing that influencing teachers' beliefs and practices
takes considerable amounts of time in workshops and multiple years of
teaching.
Student learning outcomes. Gains in student learning outcomes had
statistically significant links to some teacher-level variables.
Teacher-participants who reported shifting from teacher- to
student-centered practices had higher student learning outcomes on the
first assessment than their peers. This supports work from prior studies
that linked student-centered pedagogies with student learning outcomes
(Polly, 2008; Heck et al., 2008; Stigler & Hiebert, 1997; Tarr et
al., 2008).
In regard to the influence of teacher beliefs on student learning,
the results were mixed; students whose teachers shifted toward
transmission views of mathematics teaching outscored peers whose
teachers were discovery/connectionist. However, on the second round of
assessments, students whose teachers had shifted from transmission to
discovery/connectionist in regard to mathematics learning and
mathematics as a subject area outperformed their peers. Various studies
(e.g., Carpenter, Fennema, & Franke, 1996; Desimone, Porter, Garet,
Yoon, & Birman, 2002; Heck et al., 2008) have also found that
teachers who had embraced student-centered beliefs and practices saw
gains in student learning outcomes on problem-solving measures. However,
in this study there is not enough data to fully support that conclusion.
Limitations and Implications
Teachers were solicited to participate in this study as part of the
professional development activities and were told that they could
withdraw anytime during the study. Seventeen (33%) teachers did not
complete the pre- and postinstruments or did not provide their student
data and therefore were excluded from the study, although they completed
most of the professional development activities. Therefore, this study
is limited in the representativeness of the teachers who participated in
the professional development activities by including only 67%. Another
limitation is that possible measurement errors exist. The instruments
used in this study to measure teacher beliefs and practices were adapted
from Swan (2006, 2007). We followed the same procedures as Swan to code
the data; however, the validity information about these two instruments
was not available. The cut-off percentage (45%) in either discovery or
connectionist category is also arbitrary. The same is true for the use
of 2.00 as the cut-off point in the teacher practice survey. A
reexamination of the method to code the belief and practice surveys is
necessary. Although the practice survey is reliable (Cronbach's
alpha = .79), the use of reverse-keyed items is challenged and could
possibly influence the validity of the instrument (Kim, Wang, & Ng,
2010). Further study is needed to examine the validity of the
instruments and use structural equation modeling method to examine the
relationships while considering the measurement error.
With the limitations in mind, the findings in this study warrant
consideration for future studies. First, as indicated earlier, the time
that the postassessments are administered should be modified to avoid a
high-stress time, in which teachers in Grades 3,4, and 5 have modified
their instructional practices to drill for state tests. One possible
alternative is to collect data on practices and beliefs multiple times
during the school year, such as before the project, immediately after
the intensive summer workshop, and at a few points during the school
year. For example, collecting teacher data on practices and beliefs
periodically when we collect student assessment data would allow us to
track student performance and teacher change together. We also predict
that collecting data longitudinally will provide more valid data related
to teachers' beliefs and practices.
Further, there is a need to reconsider how students'
curriculum-based assessments are scored. For the purposes of this study,
the researchers scored the assessments numerically, assigning scores to
student answers based on rubrics developed by project staff and
teacher-leaders. The rubric scores were then converted to percentages.
There was a substantial difference in the weight of each item, as some
assessments had two multistep tasks, and others had as many as eight
tasks. It is possible that the present way of scoring the assessments
does not properly assess the level of student growth in mathematics
achievement. One alternative would be to collect alternative student
achievement data that are more formative in nature, such as student
interviews.
CONCLUDING THOUGHTS
Elementary schoolteachers participated in a year-long, 84-hour
professional development project focused on the implementation of
standards-based mathematics curriculum. Data analysis indicates moderate
growth in teachers' content knowledge, a shift toward
student-centered instructional practices, and gains on curriculum-based
assessments. Further, there was an empirical association between
student-centered instructional practices and gains in student learning
outcomes on the first round of curriculum-based assessments, as well as
a link between shifts toward discovery/connectionist views of how
students learn mathematics and student learning gains on the second
round of assessments. Teachers' change in beliefs was mixed,
indicating a lack of influence of the professional development on
teachers' beliefs, or the confounding context of high-stakes
testing.
FUNDING
This article is supported by the U.S. Department of Education
Mathematics Science Partnership Program. The thoughts and comments of
this article do not reflect those of the U.S. Department of Education.
DOI: 10.1080/02568543.2014.913276
REFERENCES
Alexander, P. A., & Murphy, P. K. (1998). The research base for
APA's learner-centered psychological principles. In N. M. Lambert
& B. L. McCombs (Eds.), Issues in school reform: A sampler of
psychological perspectives on learner- centered schools (pp. 33-60).
Washington, DC: American Psychological Association.
APA Work Group of the Board of Educational Affairs. (1997).
Learner-centered psychological principles: A framework for school reform
and redesign. Washington, DC: Author.
Bailey, L. B. (2010). The impact of sustained, standards-based
professional learning on second and third grade teachers' content
and pedagogical knowledge in integrated mathematics. Early Childhood
Education Journal, 38, 123-132. doi: 10.1007/s 10643-010-0389-x
Banilower, E. R., Boyd, S. E., Pasley, J. D., & Weiss, I. R.
(2006). Lessons from a decade of mathematics and science reform:
Capstone report for the local systemic change through teacher
enhancement initiative. Chapel Hill, NC: Horizon Research.
Borko, H. (2004). Professional development and teacher learning:
Mapping the terrain. Educational Researcher, 33,3-15.
Carpenter, T., Fennema, E., & Franke, M. L. (1996). Cognitively
guided instruction: A knowledge base for reform in primary mathematics
instruction. Elementary School Journal, 97(1), 3-20.
Carpenter, T. P., Fennema, E., Peterson, P. L., Chiang, C.-P.,
& Loef, M. (1989). Using knowledge of children's mathematics
thinking in classroom teaching: An experimental study. American
Educational Research Journal, 26, 499-531.
Cognition and Technology Group at Vanderbilt. (1997). The Jasper
project: Lessons in curriculum, instruction, assessment, and
professional development. Mahwah, NJ: Lawrence Erlbaum.
Darling-Hammond, L., Wei, R. C., Andree, A., Richardson, N., &
Orphanos, S. (2009). Professional learning in the learning profession: A
status report on teacher professional development in the United States
and abroad. Washington, DC: National Staff Development Council.
Desimone, L. M. (2009). Improving impact studies of teachers'
professional development: Toward better conceptualizations and measures.
Educational Researcher, 38(3), 181-199.
Desimone, L. M., Porter, A. C., Garet, M. S., Yoon, K. S., &
Birman, B. F. (2002). Effects of professional development on
teachers' instruction: Results from a three-year longitudinal
study. Educational Evaluation and Policy Analysis, 24(3), 81-112.
Doyle, W. (1988). Work in mathematics classes: The context of
students' thinking during instruction. Educational Psychologist,
23(2), 167-180.
Drake, C., & Sherin, M. G. (2002, April). Changing models of
curriculum use. Paper presented at the Annual Meeting of the American
Educational Research Association, New Orleans, LA.
Drake, C., & Sherin, M. G. (2006). Practicing change:
Curriculum adaptation and teacher narrative in the context of
mathematics education reform. Curriculum Inquiry, 36(2), 153-187.
DuFour, R., & Eaker, R. (1998). Professional learning
communities at work: Best practices for enhancing student achievement.
Bloomington, IN and Alexandria, VA: National Educational Service and:
Association of Supervision and Curriculum Development.
Fennema, L., Carpenter, T, Franke, M., Levi, M., Jacobs, V, &
Empson, S. (1996). A longitudinal study of learning to use
children's thinking in mathematics instruction. Journal for
Research in Mathematics Education, 27,403-134.
Fishman, B. J., Marx, R. W., Best, S., & Tal, R. T. (2003).
Linking teachers and student learning to improve professional
development in systemic reform. Teaching and Teacher Education, 19,
643-658.
Garet, M., Porter, A., Desimone, L., Briman, B., & Yoon, K.
(2001). What makes professional development effective? Analysis of a
national sample of teachers. American Educational Research Journal, 38,
915-945.
Goldsmith, L. T., Mark, J., & Kantrov, I. (2000). Choosing a
standards-based mathematics curriculum. Boston, MA: Educational
Development Center.
Guskey, T. R. (2003). What makes professional development
effective? Phi Delta Kappan, 84, 748-750.
Heck, D. J., Banilower, E. R., Weiss, I. R., & Rosenberg, S. L.
(2008). Studying the effects of professional development: The case of
the NSF's local systemic change through teacher enhancement
initiative. Journal for Research in Mathematics Education, 39(2),
113-152.
Henningsen, M., & Stein, M. K. (1997). Mathematical tasks and
student cognition: Classroom-based factors that support and inhibit
high-level mathematical thinking and reasoning. Journal for Research in
Mathematics Education, 28, 524-549.
Hiebert, L, & Stigler, J.W. (2000). A proposal for improving
classroom teaching: Lessons from the TIMSS Video Study. Elementary
School Journal, 101, 3-20.
Higgins, L, & Parsons, R. M. (2010, April). Designing a
successful system-wide professional development initiative in
mathematics: The New Zealand numeracy development project. Paper
presented at the Annual Meeting of the American Educational Research
Association. Denver, CO.
Hill, H., Rowan, B., & Ball, D. L. (2005). Effects of
teachers' mathematical knowledge for teaching on student
achievement. American Educational Research Journal, 42, 371-406.
Kim, D. H., Wang, C., & Ng, K. M. (2010). A Rasch rating scale
modeling of the Schutte self-report emotional intelligence scale in a
sample of international students. Assessment, 17, 484-496.
Loucks-Horsley, S., Stiles, K. E., Mundry, S., Love, N., &
Hewson, P. W. (2010). Designing professional development for teachers of
science and mathematics (3rd ed.). Thousand Oaks, CA: Corwin.
McGee, J. R., Wang, C., & Polly, D. (2013). Guiding teachers in
the use of a standards-based mathematics curriculum: Perceptions and
subsequent instructional practices after an intensive professional
development program. School Science and Mathematics, 113(1), 16-28.
doi:10.111 l/j,1949-8594.2012.00172.x
National Council for Teachers of Mathematics. (2000). Principles
and standards for school mathematics. Reston, VA: Author.
National Partnership for Excellence and Accountability in Teaching.
(2000). Revisioning professional development: What learner-centered
professional development looks like. Oxford, OH: Author. Retrieved from
www.nsdc.org/library/ policy/npeat213.pdf
National Research Council. (2004). On evaluating curricular
effectiveness: Judging the quality of K-12 mathematics evaluations.
Washington, DC: National Academy Press.
Orrill, C. H. (2001). Building learner-centered classrooms: A
professional development framework for supporting critical thinking.
Educational Technology Research and Development, 49(1), 15-34.
Penuel, W., Fishman, B., Yamaguchi, R., & Gallagher, L. (2007).
What makes professional development effective? Strategies that foster
curriculum implementation. American Educational Research Journal, 44,
921-958.
Polly, D. (2008). Modeling the influence of calculator use and
teacher effects on first grade students' mathematics achievement.
Journal of Technology in Mathematics and Science Teaching, 27(3),
245-263.
Polly, D., & Hannafin, M. J. (2010). Reexamining
technology's role in learner-centered professional development.
Educational Technology Research and Development, 58(5), 557-571. doi:
10.1007/s11423-009-9146-5
Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear
models: Applications and data analysis methods (2nd ed.). Thousand Oaks,
CA: Sage.
Remillard, J. T. (1999). Curriculum materials in mathematics
education reform: A framework for examining teachers' curriculum
development. Curriculum Inquiry, 29, 315-342.
Remillard, J. T. (2005). Examining key concepts in research on
teachers' use of mathematics curricula. Review of Educational
Research, 75(2), 211-246.
Richardson, V. (1990). Significant and worthwhile change in
teaching practice. Educational Researcher, 19, 10-18.
Russell, S. J., & Economopolous, K. (2007). Investigations in
number, data, and space (2nd ed.). Saddle Bridge, NJ: Pearson.
Smith, S. Z., & Smith, M. E. (2006). Assessing elementary
understanding of multiplication concepts. School Science and
Mathematics, 106(3), 140-149.
Stein, M. K., & Kim, G. (2009). The role of mathematics
curriculum materials in large-scale urban reform: An analysis of demands
and opportunities for teacher learning. In J. Remillard, B.
Herbel-Eisenmann, & G. Lloyd (Eds.), Mathematics teachers at work:
Connecting curriculum materials and classroom instruction (pp. 37-55).
New York, NY: Routledge.
Stein, M. K., Remillard, J., & Smith, M. S. (2007). How
curriculum influences student learning. In F. Lester (Ed.), Second
handbook of research on mathematics teaching and learning (pp. 319-370).
Greenwich, CT: Information Age Publishing.
Stigler, J. W., & Hiebert, J. (1997). Understanding and
improving classroom mathematics instruction: An overview of the TIMSS
video study. Phi Delta Kappan, 79(1), 14-21.
Supovitz, J. A. (2003). Evidence of the influence of the National
Science Education Standards on the professional development system. What
is the influence of the National Science Education Standards?
Washington, DC: National Academy Press.
Sutherland, S., with Castleton, L., Denison, M., Glanz, K.,
Silvestre, G., & Smith, J. (2006, April). The development of social
capital by school leaders. Presentation given at the Annual Meeting of
the American Educational Research Association, San Francisco, CA.
Swan, M. (2006). Designing and using research instruments to
describe the beliefs and practices of mathematics teachers. Research in
Education, 75, 58-70.
Swan, M. (2007). The impact of task-based professional development
on teachers' practices and beliefs: A design research study.
Journal of Mathematics Teacher Education, 10(4/6), 217-237.
Tarr, J. E., Reys, R. E., Reys, B. J., Chavez, O., & Shih, J.
(2008). The impact of middle-grades mathematics curricula and the
classroom learning environment on student achievement. Journal for
Research in Mathematics Education, 39, 247-280.
U.S. Department of Education. (2008). Foundations for success: The
final report of the National Mathematics Advisory Panel. Washington, DC:
Author.
van Es, E. A., & Sherin, M. G. (2008). Learning to notice in
the context of a video club. Teaching and Teacher Education, 24,
244-276.
Wilson, S. M., & Berne, J. (1999). Teacher learning and the
acquisition of professional knowledge: An examination of research on
contemporary professional development. In A. Iran-Nejad & C. D.
Person (Eds.), Review of research in education (pp. 173-209).
Washington, DC: American Educational Research Association.
Yoon, K. S., Duncan, T, Lee, S. W.-Y., Scarloss, B., & Shapley,
K. (2007). Reviewing the evidence on how teacher professional
development affects student achievement (Issues & Answers Report,
REL 2007-No. 033). Washington, DC: U.S. Department of Education,
Institute of Education Sciences, National Center for Education
Evaluation and Regional Assistance, Regional Educational Laboratory
Southwest. Retrieved from http://ies.ed.gov/ncee/edlabs
Drew Polly and Chuang Wang
University of North Carolina at Charlotte, Charlotte, North
Carolina
Jennifer McGee
Appalachian State University, Boone, North Carolina
Richard G. Lambert, Christie S. Martin, and David Pugalee
University of North Carolina at Charlotte, Charlotte, North
Carolina
Submitted June 5, 2012; accepted August 13, 2012.
Address correspondence to Drew Polly, Department of Reading and
Elementary Education, University of North Carolina at Charlotte, 9201
University City Boulevard, Charlotte, NC 28223. E-mail: abpolly@uncc.edu
TABLE 1
Descriptive Statistics of Student Assessment Gain Scores
Minimum Maximum M SD
First round (n = 629) -83.33 100 18.74 30.59
Second round (n = 542) -44.44 100 22.40 31.51
Third round (n = 450) -44.44 100 25.25 30.30
TABLE 2
Parameter Estimates of Two-Level Hierarchical Liner Models About
the Impact of Teacher Knowledge, Belief, and Practice on Student
Performance in Mathematics Assessments
First round Second round
Coefficient SE Coefficient SE
Knowledge gains -1.53 0.57 * -0.38 0.45
Belief in teaching
DC to T -10.31 7.91 3.20 7.43
T to DC -1.33 7.78 5.93 8.09
Learning
DC to T -6.00 6.34 2.52 9.17
T to DC 1.89 6.71 -14.77 6.77 *
Mathematics
DC to T -8.44 7.03 17.48 10.97
T to DC 5.37 6.01 -13.12 3.62 **
Teacher practice
T to S -10.90 4.72 * -4.55 6.14
T to T 9.15 13.99 -9.07 5.71
T to S vs. T to T -20.08 13.37 4.36 5.77
Effect size .55 .42
Third round
Coefficient SE
Knowledge gains 0.55 0.66
Belief in teaching
DC to T 24.31 4.95 ***
T to DC 19.10 10.53
Learning
DC to T -17.80 13.54
T to DC -6.81 19.80
Mathematics
DC to T -14.88 9.57
T to DC -34.06 19.48
Teacher practice
T to S 4.07 9.78
T to T 4.30 8.50
T to S vs. T to T -0.44 9.66
Effect size .58
Note. DC to T = teacher beliefs changed from discovery/connectionist
orientation to transmission orientation; T to DC = teacher beliefs
changed from transmission orientation to discovery/connectionist
orientation, the comparison group was teachers whose did not report a
change of their beliefs; T to S = that teacher practice changed from
teacher centered to student centered; T to T = that teacher practice
stayed as teacher centered, the comparison group was teachers whose
practice stayed as student centered; T to S vs. T to T = a comparison
between teachers whose practice changed from teacher-centered to
student-centered versus teachers whose practice stayed as
teacher-centered. Effect size represents the proportion of variance
the teacher-level variable accounted for student mean gains in each
round of assessment in mathematics.
* p < 0.05. ** p < 0.01. *** p < 0.001.
FIGURE 1 Teacher Beliefs Questionnaire. Note. This questionnaire
was adapted from Swan (2006). Permit for use was obtained on May
29, 2009.
Teacher name: -- Grade(s) taught: --
Indicate the degree to which you agree with each statement below by
giving each statement a percentage so that the sum of the three
percentages in each section is 100.
A. Mathematics is:
Percents
1. A given body of knowledge and standard procedures;
a set of universal truths and rules that need to be
conveyed to students:
2. A creative subject in which the teacher should take a
facilitating role, allowing students to create their own
concepts and methods:
3. An interconnected body of ideas that the teacher
and the student create together through discussion:
B. Learning is:
Percents
1. An individual activity based on watching, listening
and imitating until fluency is attained:
2. An individual activity based on practical exploration
and reflection:
3. An interpersonal activity in which students are
challenged and arrive at understanding through
discussion:
C. Teaching is:
Percents
1. Structuring a linear curriculum for the students;
giving verbal explanations and checking that these have
been understood through practice questions; correcting
misunderstandings when students fail to grasp what
is taught:
2. Assessing when a student is ready to learn;
providing a stimulating environment to facilitate
exploration; avoiding misunderstandings by the careful
sequencing of experiences:
3. A non-linear dialogue between teacher and students
in which meanings and connections are explored verbally
where misunderstandings are made explicit and worked on:
FIGURE 2 Teacher Practices Questionnaire. Note. This questionnaire
was adapted from Swan (2004). Permit for use was obtained on May 29,
2009.
Name: --
Indicate the frequency with which you utilize each of the following
practices in your teaching by circling the number that corresponds
with your response.
Practice Almost Sometimes Half Most Almost
Never the of Always
time the
time
1. Students learn 0 1 2 3 4
through doing
exercises.
2. Students work on 0 1 2 3 4
their own,
consulting a
neighbor from time
to time.
3. Students use only 0 1 2 3 4
the methods I teach
them.
4. Students start with 0 1 2 3 4
easy questions and
work up to harder
questions.
5. Students choose 0 1 2 3 4
which questions they
tackle.
6. I encourage students 0 1 2 3 4
to work more slowly.
7. Students compare 0 1 2 3 4
different methods
for doing questions.
8. I teach each topic 0 1 2 3 4
from the beginning,
assuming they don't
have any prior
knowledge of the
topic.
9. I teach the whole 0 1 2 3 4
class at once.
10. I try to cover 0 1 2 3 4
everything in a
topic.
11. I draw links between 0 1 2 3 4
topics and move back
and forth between
topics.
12. I am surprised by 0 1 2 3 4
the ideas that come
up in a lesson.
13. I avoid students 0 1 2 3 4
making mistakes by
explaining things
carefully first.
14. I tend to follow the 0 1 2 3 4
textbook or
worksheets closely.
15. Students learn 0 1 2 3 4
through discussing
their ideas.
16. Students work 0 1 2 3 4
collaboratively in
pairs or small
groups.
17. Students invent 0 1 2 3 4
their own methods.
18. I tell students 0 1 2 3 4
which questions to
tackle.
19. I only go through 0 1 2 3 4
one method for doing
each question.
20. I find out which 0 1 2 3 4
parts students
already understand
and don't teach
those parts.
21. I teach each student 0 1 2 3 4
differently
according to
individual needs.
22. I tend to teach each 0 1 2 3 4
topic separately.
23. I know exactly which 0 1 2 3 4
topics each lesson
will contain.
24. I encourage students 0 1 2 3 4
to make and discuss
mistakes.
25. I jump between 0 1 2 3 4
topics as the need
arises.
