Influence of visual cueing and outcome feedback on students ’ visual attention during problem solving

Research has shown that visual cues can facilitate problem solving by helping direct students’ attention to relevant areas of a diagram. We investigate the effect of visual cues and outcome feedback on students’ visual attention while solving conceptual physics problems containing a diagram. Students (N=89) enrolled in introductory mechanics courses were individually interviewed using a think-aloud protocol. Students solved four sets of problems, each set containing an initial problem, four training problems, a near transfer problem, and a far transfer problem. Students in the cued conditions saw visual cues on the training problems, and students in the feedback conditions were told whether their responses were correct or incorrect. Two weeks later, the same students solved near and far transfer problems from the main study. We found that a combination of visual cues and outcome feedback significantly improves performance on transfer and delayed transfer problems. Cueing and feedback influence shifts in visual attention to the relevant areas of the diagram on the transfer and delayed transfer problems differently.


I. INTRODUCTION
Research has shown that well-designed images can facilitate learning and problem solving, and poorly designed ones can inhibit problem solving by increasing cognitive load [1].We can facilitate learning by helping the learner in focusing their attention on relevant information and preventing them from focusing on irrelevant information.Visual attention may be redirected through the use of visual cues.Using visual cues to focus learners' attention on the relevant areas of instructional imagery, as well as to influence eye movements to facilitate problem solving, has been investigated and shown to be effective [2][3][4].
Problems in physics may require students to interpret visual information such as graphs and diagrams.However, most research on physics problem solving does not draw from research involving visual cognition.Thus, the motivation of this study is to investigate how cueing and feedback affect students' performance when they solve conceptual physics problems that contain diagrams.Specifically, we investigate the influence of visual cueing and outcome feedback on changes in visual attention as students solve conceptual physics problems.

II. THEORETICAL FRAMEWORK
Representational Change Theory (RCT) [5] proposes a set of cognitive mechanisms involved in solving insight problems.A problem is represented a certain way in a solver's mind, and this representation allows for the activation of relevant concepts in long-term memory.Impasse occurs if the problem representation does not permit the retrieval of necessary concepts.RCT proposes the following mechanisms by which impasse may be broken: (i) addition of information to enrich and extend the existing representation (elaboration); (ii) replacement of existing representation with a more productive representation (re-encoding); or (iii) removal of unnecessary constraints.
The use of different modalities in the provided information requires coordination of these modalities in the learner's mind.The Cognitive Theory of Multimedia Learning [6] identifies three stages involved in learning from multimodal information.Selection is attending to specific pieces of sensory information from each modality.Organization is using selected information in each modality to create a coherent internal representation.Integration is combining internal representations from different modalities with activated prior knowledge.All three processes are influenced by the learner's prior knowledge.
A framework of attentional cueing, in which cues can be used to facilitate all three processes, was proposed by de Koning et al. [7].Cueing guides learners' attention to essential information, emphasizes organization, and makes relations between elements more salient to promote their integration.Selection cues suppress unnecessary features and enhance relevant features of a diagram, causing the learner to re-encode the problem.Organization and integration cues provide new information by (i) emphasizing the order in which to view elements of the diagram or (ii) comparing similar aspects of two or more elements.We assume that learners may reach an impasse after being told their solution is incorrect (outcome feedback) or by successive cueing that emphasizes

PERC Proceedings,
information that the learner did not previously recognize as important.After considering this additional information the learner may re-represent the problem leading them to answer it correctly.

III. METHODS
The study consisted of two experiments, a main experiment and a delayed transfer experiment.The main experiment was 50-60 minutes long.The delayed transfer experiment was conducted 2-3 weeks after the main experiment and was 20-30 minutes long.Participants (N=89) in the study were students enrolled in first-and second-semester algebra-based physics courses.The students were invited to participate through an email informing them of an opportunity to earn extra credit for participation.The concepts needed to solve the problems in the study had been covered in class prior to participant recruitment.The problems were presented to participants on a computer screen.The interviews were audio and video recorded.Eye movements were recorded with an EyeLink 1000 desktop-mounted eye-tracking system, which had an accuracy of less than 0.50° of visual angle.All problems used in the study contained a diagram with two distinct features: an expert-like area containing the information relevant to solving the problem and a salient novice-like area containing information consistent with naïve conceptions documented in literature [8].
In the main experiment participants solved four sets of conceptual physics problems that covered the topics of energy conservation and speed.Each problem set consisted of an initial problem, four isomorphic training problems, a near transfer problem, and a far transfer problem.In the delayed transfer experiment, participants solved the near transfer and far transfer problems from the main experiment.The order of the problem sets, as well as the training problems within each set, was randomized to control for order effects.Figure 1 shows examples of the initial, near transfer and far transfer problems.
Participants were required to think aloud while solving the problems, and were given time to practice thinking aloud before the first problem set was presented.Participants observed as the interviewer solved one practice problem while thinking aloud, and then solved two practice problems while thinking out loud.
Participants were randomly assigned to one of four conditions: Cue + Feedback (C+F, N=22), Cue + No Feedback (C+NF, N=26), No Cue + Feedback (NC+F, N=20), or No Cue + No Feedback (NC+NF, N=21).Participants in the cued conditions (C+F and C+NF) saw visual cues i.e. colored shapes superimposed on the diagrams of the training problems for eight seconds at a time.Figure 2 shows an example of two training problems and a corresponding cue for the Skier problem set.The purpose of the cue was to facilitate the students to direct their attention to changes in vertical height for each slope (expert-like area), rather than the steepness of the slopes (novice-like area).Participants in the feedback conditions (C+F and NC+F) were told if their responses (both answer and explanation) to the previous problem were correct or incorrect, but were given no additional information.

A. Correctness
Figure 3 shows the average percentage of correct responses on initial, transfer, and delayed transfer problems solved correctly in all four conditions.To compare the performance of participants in each condition, a 4×5 repeated measures ANOVA was conducted with correctness as the dependent variable, condition as the between-subjects factor, and problem (the initial, transfer, and delayed transfer problems) as the within-subjects factor.The Greenhouse-Geisser correction was used since Mauchly's test of sphericity was violated.Results of the ANOVA indicated that there was a significant main effect of problem, F(3.64, 1280.35)= 29.13,p < .001,η p 2 = .077,but not of condition, F(3, 351) = 2.58, p = .054,η p 2 = .022.The main effect of problem is qualified by the interaction between problem and condition, F(10.94, 1280.35)= 2.49, p = .004,η p 2 = .021.This indicates that participants in each of the four conditions performed differently depending on the problem.Probing the interaction, we found no significant differences in the four conditions with respect to the average percentage of initial problems solved correctly by the participants, F(3, 1280.35)= 0.60, p > .05.For the near transfer and delayed near transfer problems, we found significant differences in the four conditions, such that participants in the C+F condition performed better than participants in the NC+NF and NC+F conditions (F(3, 1280.35)= 6.67, p < .001for the near transfer problem and F(3, 1280.35)= 7.62, p < .001for the delayed near transfer problem).We also found significant differences in the four conditions on the far transfer and delayed far transfer problems, such that participants in the NC+F condition are significantly outperformed by participants in all other conditions, F( 3

B. Visual Attention
To analyze the eye movements, areas of interest (AOI) were drawn around the thematically relevant and novicelike areas associated with the initial, transfer, and delayed transfer problems.The physical size of each AOI was taken into account by dividing the percentage of dwell time spent in each AOI by the percentage of area that each AOI subtends with respect to the whole diagram.The new measure is called percent time divided by percent area (PT/PA), also known as domain relative ratio [9].
Figure 4 shows the PT/PA that participants in each condition spent on the thematically relevant area of the initial, transfer, and delayed transfer problems.A 4×5 repeated measures ANOVA was conducted with PT/PA on the thematically relevant area as the dependent variable, condition as the between-subjects factor, and problem as the within-subjects factor.There was a significant difference in the PT/PA in the relevant area on the initial, transfer, and delayed transfer problems, F(2.17, 762.45) = 32.11,p < .001,η p 2 = .084.However, we found no significant main effect of condition, F(3, 351) = 1.91, p = .128,η p 2 = .016,and no significant interaction, F(6.52, 1280.35)= 0.93, p = .480,η p 2 = .008.Analysis of simple contrasts for the main effect of problem showed that PT/PA on each transfer and delayed transfer problem is significantly higher than PT/PA on the initial problem.We found no significant differences on the analysis of repeated contrasts.
To investigate how visual attention shifted on the near transfer problems or the far transfer problems, we also conducted two 4×3 repeated measures ANOVA, with edited by Churukian, Jones, and Ding; Peer-reviewed, doi:10.1119/perc.2015.pr.002Published by the American Association of Physics Teachers under a Creative Commons Attribution 3.0 license.Further distribution must maintain attribution to the article's authors, title, proceedings citation, and DOI.

FIG. 1 .
FIG. 1.Initial (top), near transfer (middle) and far transfer (bottom) problems used in the Skier problem set.

FIG. 3 .
FIG. 3. Average % of correct responses on initial, transfer, and delayed transfer problems.Error bars denote standard error.

FIG. 2 .
FIG. 2. Examples of training problems in the Skierproblem set, one without the cue (top), and one with the cue superimposed (bottom).The cue lasted for 8s at a time and is represented by the yellow shapes and grayed slopes.