Assessing Pedagogical Content Knowledge of Future Elementary Teachers

We reformed a course for future elementary teachers to infuse pedagogical content knowledge (PCK) into the fabric of a physics course. The modified course is structured around an instructional model called the pedagogical learning bicycle (PLB) that intertwines the construction of content knowledge (CK) with pedagogical content knowledge (PCK) using metacognitive reflection as a bridge between learning content and pedagogy. To assess the impact of the course experiences on their PCK, the future teachers were asked to work in groups to create lesson plans that incorporated their understanding of elementary children’s ideas of science and strategies to address these ideas. As a measure of PCK, we scored the lesson plans on the ways in which the students incorporated these ideas in their lesson. We describe the results of our study and discuss its implications for instruction.


INTRODUCTION
Research [1,2] has shown that most elementary teachers feel they lack adequate knowledge to teach science.While they may have learned about inquirybased teaching in science methods courses, their science content courses are taught more traditionally [3][4][5].Therefore, most pre-service teachers believe that inquiry-based science teaching is not possible in a real science classroom.
One of the most widely known curricula is Physics by Inquiry (PbI) [6] that emphasizes conceptual development and scientific reasoning skills.Students work in collaborative groups with simple equipment and use the results of their inquiry to construct mental models.Another, more recent effort is the Physics for Elementary Teachers (PET) [7] curriculum developed by Goldberg and others.The physics content is organized into seven cycles that revolve around the unifying theme of interactions of energy or forces.
In addition to enabling future elementary teachers to learn physics content and the nature of science, these curricula also facilitate future teachers to reflect on their own learning through a process of metacognition [8].Additionally, these future elementary teachers also learn how their future students think and learn about scientific phenomena.They also reflect on how their own learning is connected to the learning of these children.Recent research has shown that students' engagement in activities in which they discuss videos of elementary school children learning a science concept can improve the students' attitudes about science and beliefs about the nature of science [9].However, there has been little research on how these kinds of experiences facilitate the future elementary teachers' knowledge of discipline-specific pedagogy.

THEORETICAL FRAMEWORK
Shulman [10] proposed the inclusion of what he labeled pedagogical content knowledge (PCK), which refers to knowledge of discipline-specific pedagogy.PCK includes knowledge about how students learn the discipline, the prior knowledge and ideas that they bring to bear on their learning, and age-appropriate strategies to facilitate their learning of the disciplinary content.These issues are not solely pedagogical; rather they are intrinsically tied to discipline-specific content.Thus pedagogy and disciplinary content are inseparable.The goal of this study is to investigate the extent to which future elementary teachers' pedagogical content knowledge (PCK) is affected by interventions similar to those used in PET.
While PCK is a rather expansive construct, in this project we focus explicitly on one particular aspect of PCK i.e. future teachers' attention to students' prior knowledge.We acknowledge that this operational definition is limiting and does not include other aspects, including knowledge of epistemology and affect.
(PLB) -was an adapted two-layered 3E learning cycle [11].This model intertwined the construction of content knowledge (CK) with the development of pedagogical content knowledge (PCK) using metacognitive reflection (MR) as a metaphorical bridge between content and pedagogy.Figure 1 shows the pedagogical model used in the course.Week 1 was a typical content-focused 3E learning cycle and it had been used in this course over the past two decades [12].Week 2 was also a 3E learning cycle, but rather than content it focused on pedagogy.In the explore phase students viewed and discussed videos of elementary school children engaging in science.In the explain phase, they were introduced to summaries of research literature drawn primarily from Driver [13] and others on elementary children's ideas of science.In the elaborate phase they were expected to design learning experiences and assessments to address children's ideas of science.
The main difference between this course and the un-reformed course -which serves as a baseline for this study -is that the un-reformed course did not explicitly attend to the future teachers' PCK.Specifically, there was little attention to children's ideas of science and the prior knowledge that they bring to the class.Students in the baseline course were also not expected to reflect on their own learning.This course provided opportunities for future teachers to develop skills reflecting on their own learning of physical phenomena, understanding how children talk and learn about physical phenomena, and learning how research literature describes ways in which children think about physical phenomena.It also expected future elementary teachers to apply their integrated understanding of CK and PCK to develop ageappropriate lesson plans to teach physical science concepts in an elementary science classroom.The expected outcome was that future teachers completing the course will be able to design lesson plans that demonstrate a deep understanding of both content and pedagogy -in other words a more sophisticated PCK.
In the actual implementation of the course the elaborate stage of Week 2 was postponed until all units of the course had been completed (hence the dashed arrow in Fig. 1 leading into the elaborate phase of Week 2).Rather than have students design learning experiences at the end of Week 2 of each unit, students completed a project in the last two weeks of the semester, which served as the assessment of their PCK.Details of the project are described in the next section.

PARTICIPANTS
All of the students in the class were declared elementary or early childhood education majors.A vast majority of them were sophomores, and had not taken any science education methods courses.However, they had all completed a one-semester freshman course in which they observed elementary classrooms.Thus, all students had recent exposure to what went on in an elementary classroom.

Assessment Task
To assess students' development of PCK, they were asked to work in groups ranging from three to five on a capstone project.The project entailed deciding a topic for a lesson to be taught in a hypothetical elementary classroom.They were expected to design an age-appropriate lesson plan that would incorporate what is known about children's ideas of that topic based on what they had learned in the course through discussion of children's videos and summaries of research literature on children's ideas of science.The lesson plan was also supposed to be consistent with the National and State science standards, which they had been briefly introduced to before the start of the project.Each group prepared a five-page proposal describing their detailed lesson plan.The proposal was due two weeks before the end of the semester.In the last two weeks of the semester, each group was asked to present an overview of their lesson plan in a sixminute long presentation to the entire class.

Analysis Rubric
To assess the level of PCK we analyzed lesson plan proposals and presentations, taken together for evidence of PCK.Specifically we looked the extent to which whether these artifacts produced by the students for their capstone project showed evidence of attention to children's prior ideas of science specifically related to the chosen topic.We do not intent to imply that the complete PCK construct is captured only by the extent to which students incorporate their understanding of children's prior ideas.Indeed the construct of PCK is much more complex and possibly not completely measurable only by the artifacts created by the students.However, we believe that attending to children's topic-specific prior ideas is an important, if not central aspect of PCK.The proposals and presentations taken together were scored according to a rubric shown in Table 1.

Data Collection
To find out whether the course reform had improved students' PCK, we compared data collected from the reformed course (fall 2012) with a previous course (fall 2011) before the pedagogical learning bicycle (PLB) model had been incorporated.Both courses were taught by the same instructor.Students in both courses were administered the same assessment i.e. students in both courses were asked to design a lesson plan that incorporated children's topic-specific ideas.They were provided identical written guidelines for creating the proposal and presentation.They were also provided identical resources such as summaries of readings from the literature and access to the National and State science standards.The weight given to the project in the course in both semesters was identical and students were given the same number of weeks to complete the assessment.In other words, the only observed differences in student performance on the assessment could be attributed to the explicit use of the instructional model i.e.PLB.
In fall 2011, we collected data from a total of 110 students, comprising N = 27 groups in the baseline course.In fall 2012, we collected data from 143 students, comprising N = 37 groups in the reformed course.Because students worked in groups, the units of analyses were the artifacts produced by a group.

RESULTS
The results are shown in Fig. 2. A majority of the project artifacts created by students for their capstone projects in the baseline course (fall 2011) demonstrated evidence at level 0 for PCK.In other words most of the groups did not even mention children's ideas in the proposal or presentation of their lesson plans.Conversely, a majority (over 60%) of the artifacts created by students for their capstone projects in the reformed course (fall 2012) showed evidence of PCK at either level 2 or level 3.In other words, over 60% of the groups designed activities and/or assessments to address children's ideas of the relevant topic.

FIGURE 2. Comparison of PCK levels between the baseline course and the reformed course.
We conducted a statistical analysis of the data using the non-parametric Fisher's Exact test.This test is similar to an ANOVA in that it can be used to compare two conditions.However, unlike ANOVA it compares categorical data.We chose this non-parametric test, rather than a parametric test, because of the relatively small sample size of groups (N = 27 for fall 2011 and N = 37 for fall 2012).The results of the Fisher's Exact test indicate there is a statistically significant difference in the distribution between the baseline course (fall 2011) and the reformed course (fall 2012), χ 2 (3,64)=18.10,p<.001.This means that there is a statistically significant difference between the proportions of students in various levels.The fall 2011 group was significantly more likely to fall into level 0, while the fall 2012 group was significantly more likely to fall into level 3.There was a moderate effect size (Cramer's V=0.517).

CONCLUSIONS AND FUTURE RESEARCH
Pedagogical content knowledge (PCK) i.e. knowledge of the discipline-specific pedagogy is important for the preparation or future teachers.One important aspect of PCK is the extent to which future elementary teachers learn to attend to children's topicspecific prior knowledge while designing learning experiences.This includes designing activities as well as assessments that address children's prior ideas.
We reformed a large-enrollment (N>100) physics course for future elementary teachers to infuse PCK into the learning experiences of the students in the course.Specifically, we modified an existing course that was already based on the 3E learning cycle by creating a two-week-long sequence of two 3E cyclesthe first focused on content knowledge and the second focused on pedagogy.Together, the pedagogical learning bicycle (PLB) served as the instructional model for the course.
To assess the impact of the reformed model in this course on students' PCK we asked students to complete a capstone project in which they worked in collaborative groups to create lesson plans and present these plans to the class.The proposal for the lesson plan and the presentations were scored as per a rubric, depending upon the extent to which these artifacts incorporated children's topic-specific prior knowledge in both the design of the learning experiences as well as the design of the assessments.
Our results show that the PCK of future elementary teachers was significantly enhanced compared to the previous course which did not use the PLB model.Students in the reformed course were significantly more likely to incorporate children's prior ideas into both the design of learning experiences as well as assessments; whereas students in the baseline course were significantly more likely to not even mention children's prior ideas in their lesson plan or presentations.We can speculate that the activities in the second week of each bicycle urged students to think about how their own ideas compared with the kids' ideas facilitated the process of metacognition and might explain the results that we obtained.However, further research using individual interviews and observations would be needed to understand the mechanism by which PLB facilitates this process.
Overall, this study shows that instructional models such as the pedagogical learning bicycle (PLB) that explicitly combine the learning of content and pedagogy may have promise in promoting the pedagogical knowledge (PCK) of future teachers.

FIGURE 1 .
FIGURE 1.The pedagogical learning bicycle (PLB) model of instruction used in the course for future elementary teachers.

TABLE 1 .
Rubric for scoring the level of PCK evidence demonstrated in the project artifacts