The Role of Student Reflection in Project-based Learning Physics Courses

This paper will present the results of a study of student reflections from several project-based learning (PBL) courses at Creighton, including an upper-division quantum mechanics course and a freshman-level integrated calculus and physics course. Student reflections give interesting insights into areas of the course, such as student motivation, changes in students’ epistemologies, teaming issues, and the effectiveness of the project-based pedagogy. We will present several lessons learned from having students write frequent reflective essays, demonstrate the power of using student reflection as a tool for PER research on attitudes and epistemologies, and discuss strategies for increasing the usefulness of embedded reflection within the course.


INTRODUCTION: REFLECTION AND PROJECT-BASED LEARNING
The field of education has long realized and valued the benefit of students actively reflecting upon their own learning and educational experiences.John Dewey, perhaps the most prominent 19th and early 20th century educational theorist, defined reflection as the "active, persistent, and careful consideration of any belief or supposed form on knowledge in the light of the grounds that support it and the further conclusions to which it tends" [1].Dewey famously stated that "We do not learn from experience . . .we learn from reflecting on experience." In fact, the Kolb learning cycle, based on the works of Dewey, Piaget, and Lewin, includes reflection as a fundamental step in the learning process [2].Clearly reflection is important, but as Spalding and Wilson point out, "Reflection is a mysterious concept to many students . . .few have written -or perhaps even thought -reflectively during their academic careers" [3].
Although work on student reflection appears most often in the education and teacher preparation literature, science educators have also begun to realize the vital place of reflection in the student learning process.For example, David Meltzer, a physics education researcher, points out that "highly successful physics students . . .are active learners.They continuously probe their own understanding of a concept . .." [4].Mason and Singh give an example from a quantum mechanics course in which students' performance on the same exam questions deteriorate between midterms and the final exam [5].This clearly shows a lack of student self-reflection and selfmonitoring skills.Scott, Seltzer, and Gladding studied the connection between explicit student reflection activ-ity and learning gains on course exams on the FCI [6].And finally, Etkina and May have performed vital work connecting students' epistemologies as evidenced by reflective writing and conceptual gains.For example, they found "that students with high conceptual gains tend to show reflection on learning that is more articulate and epistemologically sophisticated" [7].
All previous physics education related research that has been conducted on student reflections has occurred in traditional classrooms (albeit supplemented by active engagement strategies).Here we consider student reflections in a new context: the project-based classroom.To give a brief overview, problem-based learning or projectbased learning (PBL) was first introduced in the 1980s in the context of medical education.Since then, the PBL approach has spread to many disciplines such as biology, law, chemistry, physics, and business with great effectiveness.So what is problem-based learning?Briefly, it is a systematic way to introduce active, student-centered learning to both large and small classes.The essential features of problem-based learning include: 1. Learning begins with a project, which are complex and based on real-world scenarios.2.Not all information is given; students need to make assumptions and estimations.3. Students learn how to identify, search for, and use information outside the textbook.4. Students work in groups, and learning is active and connected.5. Faculty's role is that of a guide and mentor.
It is important to note that a course that contains projects is not automatically a project-based course (for an excellent discussion see [8]).Projects in a PBL course drive student learning and become the central focus; students learn new material through the project.A good PBL project is engaging, multi-staged, complex, openended, and perhaps most important, covers course content naturally [9].The draw of PBL in physics is that classroom instruction more naturally resembles the organic process of research and how physics is done in the real-world: students confront purposeful, open-ended, and ill-defined problems whose answers shed light on an interesting situation/question.Barbara Dutch and collaborators at the University of Delaware have done groundbreaking work on adapting this pedagogy to physics instruction [9,10].For example, one particularly useful resource developed at the University of Delaware is the PBL Clearinghouse [11].
For the last three years, G.D. has been teaching two distinct courses at Creighton University using projectbased learning pedagogy.The first course is Modeling the Mathematical World, an interdisciplinary course cotaught with a colleague in mathematics that combines the year long calculus-based introductory physics sequence with Calculus II and III.Students in this course are primarily freshmen and sophomores who intend to major in either physics, mathematics, or chemistry.This course is taught using a combination of lecture with active engagement techniques such as the use of clickers, lecture tutorials, and ranking tasks as well as projects centrally tied to the course material.The second project-based course is an upper-division quantum mechanics course which is taught solely using project-based pedagogy.For more information about the project-based quantum mechanics course see the companion article in this issue titled "Student Epistemologies in Project-based Learning Courses".In each course, projects culminated with a formal writeup in the style of a scientific journal article using the L A T E Xtypesetting language.
Reflection, we believe, is particularly critical in project-based learning courses.In the PER community we typically use conceptual assessments exams like the FCI or QMAT (and many others!) to gauge student learning or content mastery.However, the learning objectives for a project-based course go well beyond simple mastery of the course content.For example, in the modeling course three of the course objectives are the following: 1. Students will be demonstrate the ability to find and use outside resources in physics and mathematics to solve new and related problems.2. Students will interact and work effectively as a member of a project team.3. Students will demonstrate growth in their capacity for self-directed learning through reflection and self-assessment.
The development of self-directed learning capacity within students is an active research field within the problem-based learning community (see [12] for example).Qualitative methods are necessary to assess concepts like self-directed learning and epistemological growth and changes in student attitude, particularly with typical student enrollments of less than a dozen per course.Student reflections and interviews give researchers, first of all, ample material to analyze, and secondly key insights from students' own voices.Our work asks the following basic question: can student reflections in a project-based learning course give insights into students' attitudes, epistemologies, motivation, etc. that are difficult to assess in other ways, and if so how can this tool be developed to be more effective?

REFLECTIONS AND ANALYSIS
Students in each course completed roughly eight written reflections gathered electronically using our campus LMS.Students began the semester by reflecting on their goals, aspirations, and trepidations about the course, and ended by reflecting on whether or not they met their goals and how they grew as students and scientists during the course.Additionally, students also reflected on each individual project (three to four per semester) and specially reflected on teamwork and team functionality using the CATME online instrument [13].Project reflections prompted students to write about their experience during each project, using the following prompts (stated succinctly below to give the reader the essential points): 1. Most important thing learned (both personal and technical).2. Big ideas in the course in this project.3. Most interesting thing about the project.4. Development as a student/physicist?New skills? 5. What would you do differently?Student reflections were evaluated and sorted in a thematic analysis using the technique of emergent coding [14].In general, this technique has the advantage of not limiting the analysis due to adopting any one theoretical framework, and tends to reduce (though not eliminate) a priori categorizations due to instructor biases.However, we found that our coding scheme and the different types of student reflections matched up quite well with a reflection hierarchy from the educational literature.In this hierarchy, Valli, adapting the earlier work of Schön, posits five levels of reflection (although Valli's original work deals with teacher preparation and reflection on student teaching experiences, we have adapted and shifted the meaning of several of her categories while leaving the overall organization schema intact.)[15]: In the educational literature critical reflection is often taken to be a deliberate critique that "views the school and school knowledge as political constructions" with the aim of "changing teaching practices and school structures that foster injustice and inequality" [15].Here, we envisioned critical reflection a bit differently.One of the chief aims of project-based learning and other radical, student-centered pedagogies is to change traditional power structures within the classroom and to blur the lines between teacher and student, re-envisioning these roles.Hence in the project-based learning setting, critical reflection involves students examining the construction of the course, teaching methods used to deliver content, their own empowerment, and the extent to which they are actively engaged in and driving their own learning and education.
We identified and coded 204 separate entries in student reflections that fit Valli's reflection hierarchy, 130 in the modeling course and 74 in the quantum mechanics course.The modeling reflections are more numerous simply because modeling is a two-semester long course while quantum mechanics is only a single semester.Approximately 10% of all reflections did not fit into the reflection hierarchy and were not analyzed as part of this work, although they gave interesting insights into students' experiences in PBL courses.

LESSONS LEARNED/FUTURE PLANS
What did we learn from gathering and analyzing student reflections?First of all, we determined that students are in fact capable of serious and sophisticated reflection, even though they come completely untrained and unprepared to reflect in this way in a science course.This was not a foregone conclusion.As can be seen in Figs.
(1) and ( 2), student reflections in total are roughly evenly split amongst Valli's five levels of reflection.This is significant because in the educational literature the hallmarks of growth in student reflective ability are 1) "the increasing ability to distinguish between narration and reflection" and 2) "the increasing ability to write all five types of reflections" [3].Students who are able to reflect across all five categories display reflective maturity; additionally, these reflections are more useful to the PER researcher because they contain more epistemological and attitudinal data than simple, naive narrative reflections.However, as can be seen in Table (1), some students were obviously more comfortable with certain types of reflections and avoided others; not all students are able to reflect as deeply as others.For example, quantum mechanics students spent more time writing deliberative reflections than the modeling students; quantum students were juniors and seniors whereas the modeling students were primarily first year students, and we take this as a sign of maturity and epistemological sophistication.The reflections themselves serve as a portal to capture evidence of student epistemological growth.Our companion paper in this issue, "Student Epistemologies in Project-based Learning Courses", delves into this dimension of the reflections and further validates the use of student reflections as a tool for the PER researcher.As we stated earlier, one of the hallmarks of PBL is self-directed learning, and in the reflections we saw evidence that this was actually occurring and that students were internalizing both motivation and the learning process.The personal reflection category contains many demonstrations of student growth and shifting of attitudes.For example, a first-year student reflects, "I guess I am starting to think more like a mathematician/physicist, which I am happy with . . .and I enjoy telling my family or others about the projects I have done in this class, because it makes me feel so official and like a real college student."Another student states, "I have learned that it is much more important to learn the connections between things than to memorize each individual element you learn.I have always been the person who memorizes the equations for each situation, and I had to change this for this course.Finding the connections across topics and even subjects can be the most important part of learning any material", definitely a shift towards a more expertlike understanding of physics.
In the future we plan on expanding this study of reflection in PBL courses from two courses to additional courses across the curriculum, including courses in energy science as well as mathematics and physics.To encourage students to reflect on multiple dimensions of their experience (i.e. more than in one or two of the reflection categories), we are preparing both an introductory training module on the reflective process as well as providing explicit rubrics to the students that will be used to evaluate their reflections.The educational literature warns us that it is dangerous not to evaluate reflections in some concrete way.Up until this point, the instructor (G.D.) provided written comments on reflections, but gave a grade based only on completeness.To diversify the reflections we plan to provide students with examples of reflections of previous students, as well as providing concrete opportunities for particular types of reflection, rather than allowing students to choose their preferred mode.For example, we plan to implement once a week technical reflections on the course material; technical reflections are an excellent opportunity to bring misconceptions identified in the PER literature to the forefront, and have students reflect and grapple with difficult concepts.We also plan to increase the focus on deliberative modes of reflection by adding an emphasis on real-world applications and ties to other areas of science and other realms of applicability.

1 .
Technical Reflection: Students write specifically about course content matter and mathematical and conceptual ideas and problems.2. Reflection in and on action: Students reflect on their performance in the class, both in teams and individually (i.e.what students do on a day-to-day basis).3. Personal Reflection: Students discuss and chronicle personal growth, changes in attitude and epistemologies, and evolving relationships with other students.4. Deliberative Reflection: Students look beyond the course content and makes connections with other disciplines and other areas.In this type of reflection students try to see a bigger picture, examine issues with the course and with their learning from multiple perspectives, and take into account multiple voices.5. Critical Reflection: Students reflect on the construction, operation, and power structures of the course and classroom and provide constructive criticism to improve the learning environment.

TABLE 1 .
Student reflections by reflection hierarchy category for the Modeling