Assessing for shifts in learner ’ s energy reasoning strategies

The Energy Project is a research effort aimed at increasing learner engagement with energy concepts in K-12 classrooms. We work closely with elementary and secondary science teachers, offering professional development opportunities in which teachers use energy tracking strategies to make sense of energy conservation in a wide range of complex physical phenomena. We value the construction and refinement of flexible, rigorous and intuitive energy models which will empower learners to make sense of phenomena and resources that they care about in the real world. Therefore, both in instruction and assessment we prioritize energy reasoning strategies over static lists of energy facts or correct explanations. In this paper we share preliminary evidence of significant shifts in teacher's constructive use of diagrams and rigorous attention to energy tracking when confronted with a novel energy scenario. We believe that this study has methodological implications for future research on shifts in energy understanding.


INTRODUCTION
Addressing the energy challenges of today and tomorrow will require energy experts in fields from municipal government to public health.These experts will draw from their diverse, sophisticated and nuanced understandings of energy in society that go far beyond static lists of energy facts.For these reasons we suggest that energy education efforts should prioritize energy reasoning strategies in addition to energy content knowledge.In this paper we will discuss two complimentary dimensions of energy reasoning: (I) using diagrams as tools for problematizing and refining energy ideas and (II) rigorously attending to energy conservation and tracking when analyzing specific energy scenarios.We will describe preliminary efforts to assess for shifts in learner"s use of these two energy reasoning strategies.We will focus primarily on observed shifts in the use of diagrams as reasoning tools but we will also present examples of rigorous attention to energy conservation and tracking.

THE ENERGY PROJECT
The Energy Project at Seattle Pacific University is a five-year NSF-funded project with an overarching goal of increasing learner engagement with energy in K-12 classrooms.We work directly with elementary and secondary teachers to help them build their personal understanding and formative assessment practices in the context of energy.Our instructional approach includes; Scaffolding productive learner engagement with specific scenarios that foreground challenging aspects of the energy concept; and collaboratively constructing dynamic energy representations (Energy Theater and Energy Cubes) that recruit learner ideas about real situations, mandate energy tracking, encourage sense making and promote scientific questioning and reasoning.We adopt a constructivist theoretical perspective in which we consider the organization of prior experiences and nascent scientific ideas into coherent models to form the foundation of scientific reasoning and learning.We have previously reported on the efficacy of Energy Project instruction by focusing on the quality of discourse that we observe within our workshops [1,2,3].In this paper, we extend this prior work by describing the use of pre-post written probes to study shifts in energy reasoning.

EXPERIMENTAL DESIGN
In order to study teacher growth in energy reasoning we have administered assessments near the beginning and end of two-week long workshops with teachers.These assessments ask teachers to analyze the energy processes associated with a variety of simple physical scenarios.An example question is shown in Figure 1.The pulley scenario was selected for several reasons.
Many learners expect that whenever potential energy is decreasing and kinetic energy is increasing the total energy of a system will be constant.The correct answer that the energy of the larger block1 is decreasing can be reached through multiple lines of reasoning that are sophisticated yet depend only on foundational energy ideas.Finally, as we will see below, rigorous attention to energy conservation and tracking can naturally lead learners to re-examine their initial ideas about this scenario.
We have used the pulley task and several similar tasks to study shifts in energy reasoning.In this paper we will confine our analysis to the pulley scenario though we saw similar shifts in energy reasoning strategies on other tasks [4].A pre-test including the pulley question was administered at the beginning of a two-week workshop for secondary science teachers in the summer of 2012.The post-test was administered near the end of the workshop.During the intervening two weeks participants were introduced to Energy Theater, Energy Cubes and Energy Tracking Diagrams [1,2,3].A total of 22 teachers in our workshop completed both the pre-and post-assessments.

RESULTS
The pulley task proved to be difficult with only 4 of 22 participants (18%) providing both the correct answer and correct reasoning on the pre-test.On the post-test the number of participants answering correctly with correct reasoning increased to 8 of 22 (36%).We think that this relatively small increase is encouraging considering both the difficulty of the question and the fact that no similar scenario was covered during the workshop.
We observed a more widespread shift between the pre and post-tests responses in the degree to which diagrams were used to reason about energy.On the pre-test, only 3 of 22 participants chose to include a diagram in their answer.These three diagrams are shown in Figure 2. Diagrams (A) and (B) appear to have been drawn to illustrate an idea about the forces involved in this scenario.We would suggest that these diagrams are illustrating reasoning that has already happened rather than being used constructively as tools in the reasoning process.Diagram (C) is more complex but it still appears that the diagram is primarily used to record this teacher"s ideas as she works through her analysis of this scenario.
On the post-test we asked participants to draw an energy tracking diagram whereas on the pre-test we merely invited them to "feel free to include diagrams".All but one participant included diagrams and the majority of them were using the diagrams as reasoning tools.This can be seen by closely analyzing the diagrams drawn by two participants (pseudonyms will be used to identify individual participants).Marcy was one of the 4 participants who were able to answer the question correctly on the post-test after having not answered correctly on the pre-test.She drew three separate energy tracking diagrams, crossing out the first two before arriving at her final diagram (Figure 3).Marcy also first circled the incorrect answer and then Looking closely at each of Marcy"s diagrams, including the one"s she has crossed out we can get some insight into the refinement of her thinking.The two diagrams that she crossed out are essentially identical.Marcy apparently made a number of modifications in the process of constructing her first diagram.When she settled on a temporarily acceptable diagram she decide to "redraw (it) so (as to make it) clearer" In her second diagram we see that she wanted to show the final kinetic energies of the two blocks as equal, presumably because they are moving at the same speed.
Why did Marcy feel compelled to draw another diagram on the back of her paper?She illustrates two new aspects of the energy process in her final diagram.First, she writes that "the tall block has more mass, so even though it is speeding up at the same rate as the short block, it has more kinetic energy than the short block" and her diagram reflects this idea.We are impressed with the refinement in her model and with this evidence that Marcy uses her energy tracking diagrams as tools for quantitative analysis.Marcy also shows kinetic energy in both blocks transferring to the air as thermal energy.We conclude that Marcy was able to use her diagrams as tools to arrive at her final answer and support her reasoning.She wrote, "It must be decreasing.It loses some energy as thermal to the environment.Plus some of its gravitational energy that is transformed to kinetic is transferred to (the) short block.Imagine no short block, the large block would fall faster because it kept all of the kinetics for itself." All four of the participants who changed to the correct response showed evidence of using diagrams constructively to support their reasoning.Fortunately, we observed similar use of diagrams to support energy reasoning among participants who did not arrive at the correct answer.Sara drew the energy tracking diagram shown in Figure 4 which is both elaborate and incomplete.In her energy tracking diagram Sara shows gravitational energy of the tall block transforming into thermal energy in the air, string and pulley as well as kinetic energy in tall and short blocks.The diagram appears to be unfinished because she has only placed a single unit of energy in the small block.Sara also expresses her sense confusion and explains on the back, "I get that the larger block is converting GE (gravitational energy) into KE (kinetic energy) as it is falling.I know KE is rising because it"s going faster.I know GE is lowering because it"s getting closer to the center of the Earth.Some GE is converted to TE (thermal energy) in air as the block rubs against the air.Now the energy story of the short block is totally baffling to me.The short block is gaining both KE and GE.It is moving faster and getting farther away from the center of the Earth.The force story tells me this is because the big block, which is more massive, is pulling on the short block.However, I can"t figure out the energy story.I can"t figure out how to ↑(increase) the KE & GE in the short block.My only idea is to treat the 2 blocks and the string as 1 object.As a whole, the system is gaining in KE and lowering in GE." We have included Sara"s entire explanation because it provides evidence that Sara is using her diagram for both representing ideas and to support her reasoning with those ideas.She begins with several statements about energy of which she is fairly certain.Each of these statements is also represented in her diagram.In this sense she is using the diagram to represent and graphically organize her knowledge.She also uses the diagram to support her reasoning.Sara"s inability to finish the diagram allows her to precisely identify her uncertainty regarding the energy of the smaller block.She recognizes that smaller block is "gaining both KE and GE" and she also recognizes her inability to adjust her diagram accordingly.
Sara cannot simply adjust her diagram to show increasing energy in the smaller block because she is using the diagram to rigorously track the energy in this scenario.Every energy unit on the right (final) side of her diagram must be accounted for with an energy unit from the left (initial) side.She realizes that she needs to show more units of both KE and GE for the smaller block on the right side than she has illustrated on the left but her rigorous attention to energy tracking prevents her from do so without also showing where the energy units are coming from.
Sara"s use of diagrams to support energy tracking is particularly impressive when contrasted with her pretest response to this same question."The energy of each block gets converted from potential energy to kinetic energy as they move.The amount of energy remains almost the same, w/ some being "lost" as thermal energy."(No diagram was included) We think the increased sophistication of her reasoning is noteworthy despite her failure to correctly answer the original multiple choice question.There is strong evidence that much of her post-test reasoning is directly supported by her diagram.In her diagram she includes both blocks, the pulley, and the air.This diagram appears to help her reason collectively about the energy story of all of these objects when striving to answer the posed question about the energy of the larger block alone.The diagram also appears to support metacognition and help her precisely locate a gap in her understanding.We anticipate that Sara"s ability to use energy tracking diagrams to support energy reasoning is evidence of preparation for future learning as described by Schwartz and Martin [5].Given more time to consider this scenario within a learning community, we expect that Sara would readily acquire the additional ideas necessary to correctly answer the original question.
Sara"s use of diagrams to support energy reasoning was characteristic of most of the post-test responses, including the responses of participants who did not correctly answer the multiple choice question.On the post-test, more participants were inclined to include the small block in their energy analysis.In addition, more participants were able to identify and articulate a specific gap in their understanding.

CONCLUSIONS
In this paper we have shown examples of an approach to assessing energy reasoning strategies through analysis of learner diagrams and written explanations.
Specifically we have presented qualitative methods for investigating learner"s use of diagrams as energy reasoning tools and their rigorous attention to energy tracking.We have also described a preliminary case study in which we observed significant shifts in teacher use of diagrams as energy reasoning tools.We believe that the energy reasoning strategies described in this paper can be flexibly applied to a wide array of energy related questions and challenges.

FIGURE 2 .
FIGURE 2. Diagrams drawn by participants on the pre-test.(A) Evidently drawn to show that the tension force on the larger block was upward and the acceleration was downward.(B) Evidently drawn to show the relative strength of the gravitational pull between the blocks and the Earth.(C) Evidently drawn to show that at the beginning "energy is gravity from big block.At end, energy is K from big block + K & G from little block."

FIGURE 3 .
FIGURE 3. Marcy"s post-test Energy Tracking Diagram showing the type of dramatic refinement that can occur when diagrams are used as reasoning tools.

FIGURE 4 ,
FIGURE 4, Sara"s post-test Energy Tracking Diagram illustrating the use of diagrams to support energy reasoning among a participant who did not reach the correct answer.