Can Visual Cues and Correctness Feedback Influence Students ' Reasoning ?

Research has demonstrated that attentional cues overlaid on diagrams and animations can help students attend to the relevant areas and facilitate problem solving. In this study we investigate the influence of visual cues and correctness feedback on students’ reasoning as they solve conceptual physics problems containing a diagram. The participants (N=90) were enrolled in an algebra-based physics course and were individually interviewed. During each interview students solved four problem sets each containing an initial problem, six isomorphic training problems, and a transfer problem. The cued conditions saw visual cues on the training problems, and the feedback conditions were told if their responses (answer and explanation) were correct or incorrect. We found that visual cues and correctness feedback significantly improves students’ abilities to solve the training and transfer problems.


INTRODUCTION
There exists a large number of visual environments in physics used for learning and/or assessment which contain information that is both relevant and irrelevant to the task at hand.To facilitate learning, one can aid the learner in focusing their attention on relevant information and to avoid focusing on irrelevant information.Visual attention may be redirected through the usage of cues.There exists a large body of research in a variety of contexts which utilize visual cues to increase learning in animations and static problems [1][2][3][4][5].In our previous research, we have found that incorrect solvers spend more time than correct solvers attending to the irrelevant features of a problem diagram [6].We have designed visual cues based on correct solvers' eye movement patterns for four sets of conceptual physics problems.While we observed the desired directionality for all problems (those who saw cues outperformed those who did not see cues), the results were only significant for one of four sets [7].In the current study we examine the same problems, but have redesigned the cues to be in line with relevant cueing theory.

THEORY
The function and mechanism of visual cueing can be interpreted through the lens of two theoretical frameworks.The first is the Cognitive Theory of Multimedia Learning [8] which is relevant to the usage of multimodal information in learning, and more specifically, cueing.The second is Representational Change Theory [9] and is related to the cognitive mechanisms involved in solving problems that require insight rather than mere algorithmic calculations.
The Cognitive Theory of Multimedia Learning identifies three processes involved in learning from information presented in multiple modalitiesselection, organization, and integration.Selection is the process of attending to specific pieces of sensory information.Organization is using the selected information in each modality to create a coherent internal representation.Integration is combining internal representations from different modalities with activated prior knowledge.These processes are influenced by the prior knowledge that a learner has.A framework for cueing proposed by de Koning [10] posits that cues can be used to facilitate all three of these processes by guiding learners' attention to essential information, emphasizing organization, and making the relations between elements more salient to promote their integration.According to Representational Change Theory, the representation of a problem in a solver's mind mediates the knowledge that they retrieve from longterm memory.When the way a problem is represented does not permit retrieval of necessary operators or possible actions an impasse or block occurs.There are three mechanisms by which an impasse is broken: (i) adding information to the problem to enrich and extend the existing representation (elaboration); (ii) replacing the existing representation with a more productive representation (re-encoding); or (iii) removing unnecessary constraints.
In the current study we explore visual cues that we predict will help the learner to re-represent the problem through elaboration and re-encoding, but not constraint removal.Selection cues suppress unnecessary features and enhance relevant features, causing the learner to re-encode the problem.Organization and integration cues provide new information by emphasizing the order in which to view elements of the diagram or by comparison of aspects of two or more elements.We hypothesize that a student may reach an impasse after learning that their solution is incorrect.This may occur in one of two ways.First, by telling students that their solution is correct or incorrect (correctness feedback), and secondly by successive cueing which emphasizes information that the student did not previously recognize as important.After considering this additional information the student may re-represent problem leading them to answer the problem correctly.We ask: Can visual cueing and correctness feedback help students correctly solve and reason about conceptual problems they previously were unable to?Furthermore, can cueing and feedback promote transfer?The transfer problems used in this study contain the same deep structure but contain different surface features [11].The cue highlighted the spaces between the balls in the order indicated by the gray numbers.How does the final speed of cart A compare to the final speed of cart B if the mass of the carts is the same and they both start at rest? (Frictional effects can be ignored.)

METHOD
The cue highlighted the initial and final positions of the carts in the order indicated by gray numbers.The motion of two objects is represented in the graph.When are the two objects moving with the same speed?
The cue was a set of red tangent lines which appeared on screen for the entire duration of the cue.Rank the changes in potential energy during the skier's descent down each slope from greatest to least.(Rank the change in potential energy not the total value.) The cue highlighted the heights while graying out the shape of the slopes for the entire duration of the cue.cues were described to students as hints which were meant to help them solve the problem.Students were instructed to press a button whenever they were ready to view the cue.They were required to view the cue at least once, but there was no limit on the number of times they could replay it.Students in the feedback conditions were told if their responses were correct or incorrect, but no further explanation was provided.To be considered correct, both the answer and explanation were required to be correct.

ANALYSIS & RESULTS
We expect students who are able to solve the initial problem correctly to answer a higher proportion of training problems correctly, regardless of which condition they were assigned.When comparing the percentage of students in each condition who answered the initial problem correctly, we find no significant difference between the conditions, χ 2 (3, 356) = 5.83, p = .120.Thus, we may consider the four groups of students assigned to each condition to be equivalent for further analyses.
The average performance of all four conditions is shown in Figure 2.There is a large increase in the percentage of students who are able to correctly solve the problems in the Cue + Feedback group after seeing cues on the first training problem and being told if their response to the initial problem was correct or incorrect.The increase in training problem correctness is more gradual and not as great for students who either saw cues or received correctness feedback (but not both).We do not observe any increase in the group who neither received feedback nor saw cues on the training problems.To compare the average percentage of training problems solved correctly by students in the four groups, a 4x2 ANOVA was conducted.The factors used in this analysis were the condition to which a student was assigned and the correctness of their solution to the initial problem, and the dependent variable was the average percentage of training problems solved correctly across all problems.The results are provided in Table 1.We find a significant main effect of condition and initial problem correctness.However, these effects are qualified by a significant interaction between the two factors, indicating that the effects of cueing and feedback on training problem performance depend on whether or not the student was able to correctly solve the initial problem.We find no significant effect of condition for students who are able to correctly solve the initial problem, meaning all groups perform equally well.Among students who incorrectly solved the initial problem, there is a significant effect of condition.Post-hoc comparisons made using the Tukey test indicate that students who saw cues and received feedback correctly answered the significantly highest percentage of the training problems followed, respectively, by those who only saw cues, those who only received feedback, and those who received neither.
Investigating training problem performance for

Results of Tukey Post-Hoc Test
each individual problem set, we find a similar pattern (see Figure 3).The results of the 4x2 ANOVA for each set are summarized in Table 1.For the Ball and Skier sets there is a significant interaction between the condition and initial problem correctness.We find a significant difference between the conditions in the percentage of training problems correctly solved, but only for students who incorrectly solved the initial problem.For both sets, students who saw cues performed better than those who did not see cues.However, these differences disappear when considering only the students who answered the initial problem correctly.For the Cart and Graph problems, there was a significant main effect of condition and initial problem correctness, but their interaction was non-significant, meaning that the effect of cueing and feedback on training problem correctness was not affected by whether or not the student answered the initial problem correctly.Students who saw cues and received correctness feedback outperformed all others The results of the transfer problems are provided in Figure 4.A Chi-Square test was performed to determine if there was a significant difference in the proportion of students in each condition who were able to correctly solve the transfer problem.The results are reported in Table 2.For all sets, a significantly higher percentage of students who saw cues on the training problems and received correctness feedback answered the transfer problem correctly than those in other conditions.Furthermore, for the Ball, Graph, and Skier sets, a significantly higher percentage of students who neither saw cues nor received correctness feedback answered the transfer problem incorrectly.

SUMMARY & LIMITATIONS
In this study, we investigated the influence of visual cueing and correctness feedback on students' solutions to insight problems.Across all problem sets, we find that the combination of cueing and feedback promotes the greatest performance on the training problems as well as the transfer problem.While these results are promising, the problems used in this study cover a narrow range of physics topics.Further investigation into the effectiveness of visual cueing and correctness feedback in other contexts should be undertaken.
The participants (N=90) were enrolled in a first semester algebra-based physics course covering mechanics.Each participant took part in an individual session lasting 50-60 minutes.During the interview, students worked through four sets of problems each containing diagrams with features consistent with novice-like answers documented in the literature and separate areas relevant to correctly solving the problem.Each set consisted of an initial problem, six isomorphic training problems, and a transfer problem which assessed the same concept, but had different surface features than the other problems in the set.Students were instructed to spend as much time as they needed on each question and to give a verbal answer and explanation when ready.The order of the problem sets and the training problems within each set was randomized.Participants were randomly assigned to one of four conditions: Cue + Feedback (C+F, N=22), Cue + No Feedback (C+NF, N=22), No Cue + Feedback (NC+F, N=24), No Cue + No Feedback (NC+NF, N=22).Those in the cued conditions saw colored shapes superimposed on the diagrams of the training problems (but not the transfer problems) for eight seconds.Examples of the cues are provided in Figure 1.The Two balls roll along the paths shown.A snapshot of the position of the balls is taken every second.At what time does Ball B have the same speed as Ball A?

FIGURE 1 .
Examples of training problems with the cue superimposed from the Ball, Cart, Graph, and Skier problem sets, respectively.The cue is represented by the gray shapes (and grayed slopes for the Skier).The cues lasted for eight seconds, and the numbers indicate the order in which the shapes appeared on the Ball and Cart problems.

FIGURE 2 .
FIGURE 2. The average percentage of students who correctly solved each problem across all four sets.

FIGURE 3 .
FIGURE 3. The average percentage of training problems correctly solved by students for each set.

FIGURE 4 .
FIGURE 4. The percentage of students who correctly solved the transfer problem in each problem set.

TABLE 1 .
Results of the 4x2 ANOVA with percentage of training problems solved correctly as the DV.Simple effects are used to probe a significant interaction between condition and initial problem correctness.The significance level for all tests is p < .05.Tukey post-hoc test performed on cells with * ( > denotes sig.diff.).

TABLE 2 .
Results of the chi-square test for the transfer problems.The significance level is p<.05.