Assessing Transfer of Learning in Problem Solving from the Perspective of Preparation for Future Learning

The preparation of future learning (PFL) perspective posits that transfer can be measured by how effectively students can learn to solve new problems. This contrasts with the sequestered problem solving (SPS) perspective which focuses on whether students can solve new problems unaided. We developed a tutorial to facilitate students‟ understanding and application of mathematical differentiation in physics problems. One group of students utilized our tutorial, while a control group received a traditional lesson on the same topic. After instruction, each group completed a SPS transfer task. Following that they received computer-based hints to aid them on same transfer task. The extent to which students successfully used these hints assessed their PFL transfer. We found that students who completed the tutorial did not outperform the control group on the SPS task, but they did outperform them on the PFL transfer task.


INTRODUCTION
There has been extensive research on students" difficulties in understanding and interpreting kinematics concepts [1][2][3].For example, students often confuse position, velocity, and acceleration.While interpreting graphs, they often conflate the slope of a line and the height of a point on the line.The concepts of velocity and acceleration are associated with the mathematical concepts of differentiation, rate of change, and limit.For instance, instantaneous velocity is the derivative dv/dt and average velocity is the ratio ∆v/∆t.So we designed an instructional intervention to facilitate students" application of these mathematical concepts in a kinematics context.We conducted a study to gauge the effectiveness of this intervention.The goal of the study was to investigate whether students would be able to transfer what they had learned from our intervention and apply their new knowledge to solve a new problem.

THEORETICAL FRAMEWORK
As per the classical view, transfer of learning refers to applying something one has learned in one context to a different context [4].Gick and Holyoak [5] studied analogical transfer by first giving participants a problem and solution and then providing them a second problem that could be solved by making an analogy to the first.Bransford and Schwartz [6] refer to this as sequestered problem solving (SPS) in that the learner is not provided any other additional help to solve the new problem.
One alternative to SPS transfer is the notion of preparation for future learning (PFL) transfer [6].PFL broadens the conception of transfer to include an emphasis on whether the learner can learn to leverage resources provided in a new context.So, instead of asking whether students can solve a problem without external help, we ask whether they are able to use help provided to learn to solve the problem.Rather than evaluating learners" performance on a one-shot task, the PFL approach focuses on "extended learning."The PFL perspective also suggests that assessments must move from static, one-shot tests to environments that provide opportunities for new learning [6].
Our study compares the consequences of SPS and PFL transfer perspectives on assessing problem solving.We developed research-based instructional materials which aim at facilitating students" application of differentiation to kinematic concepts.We then investigated the extent to which students using these materials showed evidence of SPS or PFL transfer and how they compare with students using more traditional materials.Based on SPS perspective of transfer, we investigated how students solved a problem on their own.Based on PFL perspective of transfer, we probed students" ability to utilize online hints to solve a problem.To differentiate the two different types of transfer tasks, we refer to the former one as SPS transfer task and the latter one as PFL transfer task.We explored the following research questions in light of our framework: 1. To what extent do students apply strategies from the intervention to solve a SPS transfer task?
2. To what extent do students learn to use online hints after the intervention to solve a PFL transfer task?
3. How does the performance of students who had the intervention compare with students who had traditional homework-style problems on both the SPS and PFL transfer tasks?

RESEARCH DESIGN
A comparison vs. treatment group, quasiexperimental design was used in this study (Fig. 1).Thirty-four students in a first semester calculus-based physics course volunteered to participate in our study that was conducted about three weeks after students completed an exam on kinematics.Students were offered extra credit equal to one homework set for their participation.The participants were randomly assigned to a treatment or comparison group.For the first 10 minutes, students individually attempted a baseline task (Fig. 2).The purpose of the baseline task was to investigate, (i) students" ability to solve this problem and (ii) strategies students would use for solving this problem prior to our intervention.For the next 50 minutes students in both conditions worked in groups of three.They were encouraged to collaborate with their partners while solving the problems.Students were not given any help from the facilitators to solve the problems.

Intervention Materials
Students in the treatment condition used the intervention materials.These materials were designed based on our research and aimed at improving students" understanding and application of mathematical concepts of velocity and acceleration.We facilitated students" relating kinematics concepts to the underlying mathematical concepts of average and instantaneous rate of change and limit.The materials consisted of six problems.Problems 1 and 2 focused on average and instantaneous velocity/acceleration in algebraic representations, while problems 4 and 5 focused on these concepts in graphical representations.Problems 3 and 6 were reflection questions.
In problem 1, students were given displacement as a function of time and were asked to find the average velocity during a time interval.Next, by taking shorter and shorter time intervals, they were asked to use the average velocity to approximate instantaneous velocity.In problem 2, students were given velocity as a function of time and asked to follow similar procedures as in problem 1 to find average and instantaneous acceleration.Problem 3 was a reflection question which focused on helping students to compare the first two problems.Problems 4 and 5 were graphical representations analogous to problems 1 and 2. Problem 6 was a reflection question designed to help students compare the previous problems.
In summary, the intervention materials included the following aspects: scaffolding to guide students to generate concepts themselves, two different mathematical representations, algebraic and graphical, and reflection questions to facilitate students to see connections between the problems.

Homework-Style Problems
Students in the comparison condition solved homework style problems (i.e., traditional end-ofchapter textbook problems) that covered the same concepts as the intervention.After completing each problem they were provided with solutions before proceeding to the next one.These problems were typically quantitative and required students to apply mathematical equations to find physical quantities.

Transfer Tasks
Students worked independently on the SPS transfer task which was similar to the baseline task except for some changes in surface features.They tried the problem by themselves and wrote their solutions on worksheets.Finally, students who were unable to correctly solve the SPS task were provided the PFL Baseline Task task which was identical to the SPS task, except they were provided hints through the MasteringPhysics TM online system.The goal of the hints was to activate the knowledge resources which were consistent with the tutorial materials and critical to solve the posttest problem.The hints were ordered based from a minimal amount of help to a maximum amount of help.The hints for the PFL task are shown in Table 1.Instead, find an approximate value. 2 You can find the velocity "v" at a time "t" using the information in the graph.3 You can approximate an instantaneous acceleration using an average acceleration.4 Instantaneous velocity is the slope of the tangent line in the graph of x(t).5 Acceleration between time t 1 and t 2 is .
After looking at each hint, students tried the problem again and submitted their new answer through MasteringPhysics TM .They recorded their thoughts after each hint even if they could not solve the problem.

RESULTS
The strategy that the intervention materials aimed to facilitate included finding the instantaneous velocities at two close times and then calculating the average acceleration during this time interval.This targeted strategy required application of knowledge resources such as approximation, average acceleration, instantaneous acceleration, and instantaneous velocity that were embedded in the intervention materials.We analyzed the correctness of students" solutions as well as the strategies they employed to solve the baseline and transfer tasks.

Baseline Task
None of the 34 students used the targeted strategy on the baseline task.Only one student from the comparison group got the correct numerical answer but did not use the targeted strategy.Other students" written comments provided no evidence of any strategy whatsoever.Their difficulties and errors can be classified into the following categories: lack of understanding of slope, misapplication of physics equations, or graphing errors.Eight out of 34 students attempted to calculate the acceleration by taking slope of the tangent line at time t = 4s.Even though some students claimed that velocity was slope of the tangent line, they still regarded the slope as the acceleration in their follow-up calculations.Nine out of 34 students tried to find the acceleration at t = 4s by applying kinematics equations that were applicable for situations with constant acceleration and were hence inapplicable in this problem situation.Several students used what we call the "sketch graph" strategy.They claimed that the x(t) graph looked like a cubic function and tried to sketch v(t) and a(t) graphs.However, they were unable to explain how this strategy could be generalized to an arbitrary graph.

SPS Transfer Task
On SPS transfer task, six students from treatment group tried to apply the approximation method, however, only one correctly used this strategy.The other five students were unable to correctly solve the problem due to a calculation error, misreading the values from the graph, picking too large time intervals for making approximation, or using the approximation idea twice.The latter refers to students who tried to approximate instantaneous velocity with average velocity even though they could approximate instantaneous velocity from the x(t) graph.The rest of the students used other ineffective strategies to solve this task.In the comparison group, none of the students used the targeted strategy.Eight students from the comparison group used the "sketch graph" strategy.The rest of the students either tried to use kinematics equations involving constant acceleration or attempted different strategies inconsistently.

PFL Transfer Task
Before students received any hints, only one of the six students in treatment group who attempted the targeted strategy correctly solved this problem.The other five students recognized the targeted strategy but used it inaccurately, most often by choosing larger time intervals.When students were provided online hints, they seemed to reflect on what they had done previously and refined their strategy by taking shorter time intervals to make approximation more accurate.After they worked through the hints, five of seven students from treatment group who attempted the targeted strategy solved the problem correctly.In comparison group, two students employed the strategy with the help of online hints although only one used it correctly.

Summary of Results
We found no differences between the groups on the baseline task.None of the students used the targeted strategy of approximations to estimate the acceleration at a particular point in time.On the SPS task, students in the treatment group marginally outperformed the comparison group.On the PFL task, where students were provided hints, students in the treatment group outperformed the comparison group even more than on the SPS task.In other words, students in the treatment group were better able to use the hints provided to correctly solve the problem than students in the comparison group.Figure 3 shows a comparison between the two groups on the three tasks.

DISCUSSION
In general, more students in treatment group showed a tendency to use targeted strategy than comparison group.As students from treatment group followed online hints, they became even more successful at solving the transfer task than students from comparison group.With the help of online hints, students could reflect on their previous solutions and improve their strategies.That students in treatment group were able to use the online hints more effectively than students in comparison group is evidence of PFL transfer.In other words, the intervention materials better promoted PFL transfer compared to homework-style problems, even though there were small differences between the two conditions on the SPS transfer task.The results of this study also make a case for using PFL transfer tasks, similar to the one described here, for assessing the impact of instructional interventions.

CONCLUSIONS
Student difficulties in kinematics have been well documented in the literature [1][2][3].Further, it is also known that many of these difficulties are related to students" inability to transfer mathematics concepts such as slope and limits to kinematics contexts.To address these difficulties we created research-based intervention materials and tested these materials in comparison with typical homework-style problems.In comparing the two conditions with regard to their effectiveness in promoting transfer, we incorporated the notions of both SPS and PFL transfer as proposed by Bransford and Schwartz [6].To facilitate PFL transfer we provided online hints to students that were designed to help them activate knowledge resources that they had presumably acquired through their working with the intervention materials.
After students worked on a research-based tutorial session, we found some evidence of transfer from the SPS perspective.By providing appropriate online hints to activate their knowledge resources, treatment students were better prepared to learn using the online hints to solve the transfer task indicating from the PFL perspective that the tutorial facilitated transfer of learning.Thus, evidence seems to indicate that compared to homework-style problems, our intervention facilitated transfer of learning from the SPS perspective and even more so from the PFL perspective.

LIMITATIONS & FUTURE WORK
It is important to point out that a majority of students in both groups were still unable to correctly solve the PFL task, which was a disappointing result.One limitation of our work is the generation of online hints.The same sequence of hints was provided to all students although their needs might have been different.We plan to develop a list of hints from which students can decide what level of help they might need.Such student-controlled adaptive hints might be more effective than the ones we provided at assessing students from the PFL perspective.

FIGURE 1 .
FIGURE 1.The research design of the study.

FIGURE 2 .
FIGURE 2. Baseline task used in the study.
Consider a particle which moves along a straight line.Use the position vs. time graph of the particle shown to estimate the acceleration

FIGURE 3 .
FIGURE 3. Percentage of students who correctly applied the targeted strategy in both groups.

TABLE 1 .
Hints provided for the PFL task.# Hint 1 You need not find the exact value of acceleration.