Examination of S tudents ’ S elf-monitoring in P roblem S olving

For solving problems, especially ill-structured ones, a solver must employ metacognitive strategies including self-monitoring to be successful. Solvers who frequently examine their thinking are able to assess their progress, consider alternatives and evaluate their own work. In an introductory mechanics course, students were given class assignments where they were asked to record problem solutions using a think-aloud protocol. These solutions were recorded using Livescribe smartpens, which record and synchronize pen strokes and audio. To better understand the selfmonitoring process, as well as improve future versions of the class activities, we examined the recorded think alouds after the courses and thus far have identified three categories of self-monitoring.


INTRODUCTION
Problem solving is at or near the top of the list of necessary skills for college graduates.Science, technology, engineering and mathematics (STEM) accreditation organizations include the identification, formulation and solving of problems as desired program outcomes, with the American Chemical Society going so far as to refer to problem-solving as the "ultimate goal."[1,2 ] Employers rate problem solving/ critical thinking as one of the top five "very important" skills for job success and yet only 28% classify college graduates' problem solving as excellent.[3] Even with their strong emphasis on problem solving, STEM programs are often viewed as graduating students who have weaker than desired problem solving skills.[4] This mismatch between the goals of STEM educators and employers and the skillset of graduates may be traced to differences in the types of problems presented to students and graduates.Students tend to be presented with questions that are well-structured, which include all of the sufficient and necessary information.[5] These textbook questions can often be considered exercises as they are sufficiently familiar and straightforward; therefore, the majority of students can solve them with little effort.Conversely, graduates in the workforce are often tasked with solving illstructured problems that are missing necessary information, or contain ambiguous information that could result in multiple acceptable solutions.[6] In problems, the solver must make decisions when crafting a solution to a problem as the pathway is not readily apparent.
In this paper we present a brief description of a process that students can utilize when solving problems-self-monitoring one's progress so decisions can be made throughout the solution.We then describe the methods by which we observed students solving problems and finally we present observed patterns of self-monitoring statements.

SELF-MONITORING
Most textbooks and instructors do not model the non-linear thinking and decision-making required for solving problems, even though students have major difficulties lie with this aspect of problem-solving.[7] Instead, students are typically presented with contextspecific algorithms that are solely comprised of a set of steps that are to be followed sequentially.[8] By presenting algorithms to students, textbooks and solution manuals only reinforce the already strong disposition in students that problem solving is a series of standardized procedures, which usually manifests as "plug and chug" in STEM courses.[9] Students expect to solve all problems as if they were exercises-simply utilizing one idea or formula that matches the given variables.[10] Consequentially, when students graduate they are often ill-prepared, or unmotivated, to solve the ill-structured problems found outside of most classes.
By supplementing algorithms and heuristics with active decision-making, solvers are better equipped to solve problems.The term self-regulated learning (or simply self-regulation) is often used to describe this decision-making.Self-regulation can be considered part of the broader construct metacognition or "thinking about thinking".[11][12][13] Without using self-regulation skills, students often use the first idea that occurs to them, whether or not it is appropriate, and fail to explore alternatives.[10,14] For solving problems, self-monitoring is probably the most important component of self-regulation.[15] Self-monitoring is the "deliberate attention to some aspect of one's behavior."[16] This subprocess is what initiates the self-regulation process.Without selfmonitoring, there would be no basis for adjustments.It is often characterized as an executive process that activates and deactivates other subprocesses.[17] When engaged in self-monitoring, a solver's attention is on specific aspects of the solution, which prevents her from loosing focus.Self-monitoring generates feedback of what is working and what could be revised.Self-monitoring can also guide a solver to a more efficient pathway.Consequentially, those who engage in significant self-monitoring can solve more complex problem in more efficient ways than those who do not.[18] Alan Schoenfeld has had considerable success in improving students' problem-solving proficiency by instructing them on how to self-monitor their thinking.To help them avoid unproductive approaches or generate new ideas, he asks "What (exactly) are you doing?", "Why are you doing it?",and "How does it help you?" [19] While these prompts are successful at initiating self-monitoring, they do not illustrate all of the ways a student can reflect on her solution.In this project we sought out to identify the different ways students self-regulate their thinking while solving physics problems.

METHODS
We examined the recorded think-aloud solutions generated by students enrolled in an algebra-based mechanics course for life-science majors.The students had an average score of 75% on the Classroom Test of Scientific Reasoning.[20] This average approximately corresponds to the threshold for formal operational thinking.The class average normalized gain on the Force Concept Inventory was 0.57.
As part of the class, students received instruction and feedback on verbalizing their thoughts while solving problems so that they could later record narrated solutions independently.In a typical training activity in class, students work in pairs with one student solving a problem and the other student encouraging the first to explain her thoughts.Any time the solver was silent for more than a few seconds or said something that wasn't sufficiently clear, the second student was tasked with asking for clarification.These interjections were simply to aid verbalization of the thought process, not to correct errors in problem-solving or physics concepts.Those errors would be discussed after the solver completed her solution.
Six times in the semester, students were required to submit a recorded problem solution where they verbalized their thought process.These homework assignments usually consisted of one or two problems from, or inspired by, the University of Minnesota's Context-Rich archive.[21] Generally these were problems, rather than exercises, for the students enrolled in the course.Students were given credit for turning in solutions, independent of their correctness.
To record these solutions outside of class, each student was given a Livescribe smartpen.[22]These smartpens use embedded cameras to track and record the pen's strokes and microphones to capture the concurrent audio.[23]When synchronized with a computer, the software combines these two files into a single "pencast."The smartpens are very convenient for capturing think-alouds out of class as they allow students to make recordings anywhere they want.
These recordings were analyzed using grounded theory methodology.[24] Rather than beginning with a theory or model, this methodology relies on coding the recordings and identifying categories that emerge those codes.Eventually, the relationships and implications of these categories will also be studied.
After the course was completed, six research assistants looked for evidence of self-monitoring events among approximately one hundred recordings from thirty different students.Each recording was viewed independently by at least two researchers.Periodically, the researchers gathered to compare codes.
When there was disagreement on what constituted a self-monitoring event, other researchers were asked to observe the recording in question.Tentative categories were first identified midway through the viewing of the recordings.As the researchers viewed the other fifty recordings, these categories were modified in light of the new codes.For the purpose of this study, we have limited the codes and categories to students' conceptual, rather than epistemic self-monitoring.

SELF-MONITORING CATEGORIES
There are multiple ways that a problem solver could monitor her progress.We present three such possible categories that we have repeatedly observed among the student think-alouds.
These do not represent the entirety of self-monitoring, but they do represent the most common categories observed thus far.Two of the categories are where the solver is comparing two items for consistency, either between part of the solution and some prior knowledge or between two parts of the solution.The third category Checking for external consistency is a form of monitoring done when a solver compares an element of her problem solution (e.g. a value or a step in the plan) with something that she has done previously in another problem or with a real-life scenario.The solver is seeking assistance from experiences or information that exist outside of the problem or solution.Samples of this category that were found in student think-alouds can be found in table 1.
Comparing calculated or estimated values in a solution to real-world values is a common technique among our sample population and for many other students solving physics word problems.However, a solver can also compare other elements of her solution to external sources.For example, a student could ask "Does my interpretation of the problem align with ones that I have seen previously?" or "Does my planned solution align with plans that were used on other problems, which have similar contexts?"A similar category is where a solver compares elements of a solution to itself; see table 1 for representative quotes.When Checking for internal consistency a solver compares an element of her solution to something that has been done previously in the same solution.The solver is checking to make sure that the solution (calculations, assumptions, etc.) is self-consistent.
This internal consistency check could be used to juxtapose interpretations of components of the problem.(Do the presumed goals, conditions and contexts agree with each other?)The solver can also compare different components of the planned solution with the goal of identifying errors before beginning the calculations.(Do various assumptions agree with each other and the interpretation of the problem?)Finally, and more commonly among our students, the solver can compare two calculated values to see if the solver's plan, when executed, is producing coherent results.(Are different calculations yielding results that agree with each other, and the selected plan?) Like pedestrians at an intersection, solvers are often advised to pause before stepping forward so they may imagine what will happen next.By Assessing their readiness, solvers can determine if they are done interpreting the problem statement and ready to begin making assumptions and selecting models.(Does the problem interpretation accommodate doable plans?)Solvers can also look ahead at the calculations that might be required to enact the plan.(Is the plan feasible?Is there enough information to complete the calculations?)By assessing their readiness, solvers can evaluate whether the solution path is the correct or most efficient, see table 1 for examples.
Much like Schoenfeld's question "How does it help you?", assessing for readiness guides a solver through the solution, helping them decide when to move forward and when to step back so as to avoid a potential catastrophe.

CONCLUSIONS
By examining student think-alouds we have begun to identify categories of self-monitoring processes.When solvers engage in such reflective thinking they pause momentarily so they can step back from their solution and consider what's working or what can be improved upon as objectively as possible.In many ways this process replicates how a study partner might ask for clarification or suggest that alternatives that could be explored.The challenge is to train solvers to self-monitor so they can be independent, rather than relying on others to externally monitor their work.
Often in the think-aloud recordings, self-monitoring is a pause or a vague comment such as "that's weird."This is particularly true for our recordings, which we made by students outside of class, without any classmate, instructor or researcher encouraging verbalization.Nonetheless, we have found many articulate descriptions of self-monitoring and are continuing to cluster them into categories.
In this paper we have omitted examples of when a student monitors their own skillset or motivation, such as "Yeah, I'm kind of lost here, I haven't really been practicing these problems yet honestly…" or "I'm feeling lazy…".While this epistemic monitoring is incredibly productive, our current focus is how a solver monitors her solution, rather than herself.
As this project continues, we expect to examine think-alouds recorded by experts to ensure that we have a complete set of self-monitoring categories.These samples will also help us to articulate the differences in self-monitoring between experts and novices.
By better understanding the self-monitoring process we also expect to provide improved instruction for students.Rather than giving them generic advice to "pay attention to their thinking" we have identified specific, and effective, ways that they can do this.We are in the process of incorporating activities in class that help students to see the value of checking for external or internal consistency and assessing their readiness before tackling a new aspect of the solution.
In addition to using the recorded think-alouds for developing the self-monitoring categories, the recordings will serve as examples in future class activities.Students will analyze the recordings for both successful self-monitoring as well missed opportunities.Through this study of worked, and narrated, examples, students should be able to improve their problem-solving skillset and therefore be better prepared for solving not only exercises, but also illstructured problems.

TABLE 1 .
Sample quotes of student self-monitoring during problem solving