Bottlenecks In Solving Synthesis Problems

Synthesis problems, which are problems requiring the application of multiple concepts such as energy conservation and kinematics, are common in physics curricula, and improving students’ skills in solving such problems is typically a key instructional goal. Despite the prevalence and importance of synthesis problems, many students struggle with them more than with their single-concept counterparts. In order to identify possible bottlenecks on this task, we asked students to solve a problem synthesizing two different topics (including energy, momentum, circular motion, and kinematics) as well as a pair of single-concept problems involving the individual components, varying their sequential order (synthesis + single-concept versus single-concept + synthesis). We found that students’ primary difficulties arose not only from the deficiency in applying the individual concepts but also from the inability to recognize the relevance of both concepts, which in some cases may be caused by one dominant concept overshadowing the other.


INTRODUCTION
Over the last forty years, studies of problem solving in physics have typically focused on problems requiring the application of only one major concept area (e.g., Newton's second law or conservation of energy) [1]- [4].Past research has not explicitly focused on addressing the more complex concern of what we will refer to as synthesis problems, which involve multiple major concept areas separated in the teaching timeline.Conceptual difficulties, such as the inclusion of a centripetal acceleration constraint for an object undergoing uniform circular motion in a conservation of energy problem, or procedural difficulties, such as solving the system of equations necessary for deriving the relationships between velocities in an elastic collision, may present barriers that do not exist in corresponding single-concept problems.As such, synthesis problems seem to require skills not necessary for solving single-concept problems.
Understanding differences between single-concept and synthesis problems has pedagogical importance, as physics instructors typically include practice with and improvement in solving synthesis problems as part of their instructional goals, and application of physics concepts outside the classroom generally involves synthesis tasks.If synthesis problems present unique challenges for students, then it may be necessary to adapt physics instruction to address those challenges.
Some of the authors have previously shown that guided conceptual scaffolding improves student performance on synthesis problems [5].In this study, we set out to examine possible reasons for that improvement by evaluating possible difficulties students face with syn-thesis problems.We compare student solutions to singleconcept and synthesis problems to investigate two possible bottlenecks with synthesis problems: First, a student may not have sufficient mastery of the individual concepts involved.Second, a student may not recognize that the individual concepts are relevant.

EXPERIMENTAL DESIGN
We designed and administered sequences of singleconcept and synthesis problems to students enrolled in calculus-based electricity and magnetism course at the Ohio State University, a large public research university.Two synthesis problems were used, each combining two broad topic areas (conservation of energy and kinematics, conservation of energy and circular motion).Each synthesis problem was accompanied by corresponding single-concept problems involving the same individual physical concepts as the synthesis problem.Students solved a set of three problems, either two single-concept problems followed by a synthesis problem or a synthesis problem followed by two single-concept problems.The order was varied to determine whether the student solutions to synthesis problems may be affected by priming with relevant single concepts problems.Problems were presented in a quiet room as part of a flexible homework assignment for course credit, and an equation sheet used for exams in one professor's mechanics course was available.Students completed the problems at their own pace, typically finishing in 20 minutes.
As we are interested in comparing student skills when dealing with single-concept and synthesis problems, only those students who attempted all problems are analyzed.The threshold for an attempt at a problem was the inclusion of any information not given explicitly in the problem statement.Solutions including only the given parameters were not considered an attempt; anyone including additional calculations or equations, on the other hand, was considered to have attempted the problem.
A total of 192 students were randomly assigned to one of four conditions, varying problem type and order.161 students (84%) attempted all three of the problems provided.The precise distribution of those students is shown in Table 1.Of those students, 97 received the energy+kinematics problems (56 were given the singleconcept problems first, 41 were given the synthesis problem first), while 64 received the energy+circular motion problems (33 single-concept first, 31 synthesis first).
The energy+circular motion problems are shown in Figure 1.Due to limited space, the exact en-ergy+kinematics problems are not presented here.Those problems involved a spring launching a charged projectile into a region with constant electric field; students were asked to find a displacement through the electric field.
Rubrics for assessing student solutions were determined by two of the authors (DRW and RB), both with recent experience grading for introductory courses.They independently graded the solutions as if they were exam questions and noted the general approaches used by the students (judged by equations and process used), with 5 points assigned to each single-concept problem and 10 points assigned to each synthesis problem.Because single-concept and synthesis problems were designed to involve the same mathematical steps, the scoring rubric for a synthesis problem consisted of the corresponding single-concept rubrics for each half of the problem.The result is that the total single-concept and synthesis scores are determined using the same scale and should be directly comparable.
While overall agreement was generally high (at least 60% agreement across 12 graded questions with an average agreement of 74.2%), disagreements in scores, which came primarily from low-scoring solutions and small differences in interpretation of the rubric or a student's work, were discussed and resolved, leading to the agreed-upon scores presented here.Identification of correct general approaches was in complete agreement between the two graders.

RESULTS AND DISCUSSION
The graded results of these two problem sets suggest two conclusions regarding the bottlenecks outlined at the end of the Introduction.First, we observe that mastery of the related single-concept topics may be necessary but is not sufficient for success in solving synthesis problems.Second, we see evidence that the lower performance on synthesis problems may be the result of greater difficulty in recognizing relevant concepts.
One way to assess the relative difficulty of a synthesis problem and its corresponding single-concept problems is to compare the scores and score distributions.Table 1 shows the average scores on each problem type for each group, and Figures 2 and 3 compare each student's total single-concept and synthesis scores for the en-ergy+kinematics and energy+circular motion problems.While the score distributions for each set of topics and question order have different overall patterns, the most general feature of the scores is that students tend to score higher on the single concept problems than the corresponding synthesis problem.This observation is most clearly seen in Figures 2c and 3c; for each data set, a Student scores for the energy+kinematics single-concept and synthesis problems.Figure 2a shows scores for students receiving the single-concept problems first; Figure 2b shows students receiving the reverse problem order.The synthesis and each of the single-concept problems were worth 10 and 5 points respectively.The size of the symbol indicates the number of students with that score combination.The diagonal line represents equal single-concept and synthesis scores.Figure 2c shows the distribution of differences in scores (single-concept score minus synthesis score) for the two types of problems.
greater proportion of students have a positive score difference (defined by total single-concept score minus synthesis score) than a negative one.
A three-way (order and topic as between subject factors, and single vs synthesis problem as a repeated measure) ANOVA confirms that the single-concept score was significantly higher than the synthesis (F(1) = 42.2,p < 0.001, energy+kinematics: d = 0.38, energy+circular motion: d = 0.57).
Additionally, we see that the largest negative score difference is −5, while the greatest score difference is 10.This observation further suggests that no student was successful in solving a synthesis problem without having at least partial success with the corresponding singleconcept problems, while several students had a great deal of success with single-concept problems but failed to solve the synthesis problem.Overall, these comparisons of single-concept and synthesis scores thus suggest that some degree of mastery of single-concept problems is necessary (and therefore a possible obstacle for students) but not sufficient (and therefore not the only obstacle) for solving a corresponding synthesis problem.
Other results from the three-way ANOVA support our hypothesis for a second bottleneck: recognition of the relevant concepts.In the analysis, order was the only other significant main effect, with the single-concept problems first condition scoring higher on both problems (F(1) = 4.9, p = 0.03, d = 0.3).A significant orderproblem type interaction revealed that priming students with the single concept problems first increased the score on the synthesis problems (F(1) = 5.2, p = 0.02, d = 0.42).The fact that priming students with the relevant single concepts improves performance on corresponding synthesis problems supports the claim that recognition of relevant concepts can be a significant bottleneck.
To provide further evidence of the recognition bottleneck, we analyzed student solutions according to the physics concepts applied in solving the problems.To do so, we made generous assumptions about student applications of physics concepts; for example, a solution equating a force and kinetic energy was classified as an "energy" approach on the reasonable assumption that the student intended to use energy but incorrectly incorporated a force.This classification allows us to make inferences into which physics concepts students believed were relevant to a particular problem.The proportion of usage of each relevant concept is shown in Figure 4.
Figure 4 indicates that many students who correctly identified a relevant concept in a single-concept problem failed to use the same concept in the synthesis problem.Across the different topics where single-concept problems were administered first, 15% of the 89 students applied the correct concept for one of the single-concept problems but failed to apply the same concept to the synthesis problem.For the case where the synthesis problem was given first, 29% of the 72 students made the same mistake.This result is suggestive that many students do not recognize both relevant concepts in a synthesis problem, even though they may recognize them in single concept problems.

CONCLUSION
We investigated and provided evidence for two bottlenecks with synthesis problems.First, we used a within-  Figure 4a shows the responses of students receiving the energy+kinematics single-concept problems first.Figure 4b shows energy+kinematics, synthesis problem first.Figure 4c shows energy+circular motion, single-concept problems first.Figure 4d shows energy+circular motion, synthesis problem first.
student design to demonstrate that it is necessary to master the relevant individual concepts comprising the synthesis problem.While this is a straightforward and perhaps unsurprising result, we also found clear evidence that mastery of the individual concepts is not sufficient.We further found evidence for a second bottleneck, namely that even though students may have mastered the application of relevant concepts individually, students must still go through some process of recognition of which concepts are relevant.Priming students with the relevant concepts helped to improve their performance, possibly because priming may facilitate that recognition process.Note that, at least in this case, many students were able to successfully solve the synthesis problem once they correctly identified relevant concepts.
Overall, the results of this study clearly indicate that some students struggle to translate single-concept understanding into problems involving more complex, multiconcept scenarios.As such, we conclude that solving synthesis problems involves unique skills in addition to one's ability to solve simpler problems.The implication is clear: special emphasis on problem solving in class-rooms is warranted if we wish to include synthesis problems as an instructional goal or a means of assessment.

FIGURE 1 .
FIGURE 1. Single-concept and synthesis problems for the energy + circular motion problem.

FIGURE 3 .
FIGURE 3. Student scores for the energy+circular motion problems, using the same notation as Figure 2.

FIGURE 4 .
FIGURE 4. Proportion of different approaches for each topic and question order.Figure4ashows the responses of students receiving the energy+kinematics single-concept problems first.Figure4bshows energy+kinematics, synthesis problem first.Figure4cshows energy+circular motion, single-concept problems first.Figure4dshows energy+circular motion, synthesis problem first.

TABLE 1 .
Average scores on each problem in each sequence.Maximum scores are 10 points.Errors shown are standard errors.