Development and Evaluation of a Large-Enrollment , Active-Learning Physics Curriculum

We report on the development and evaluation of Learning Physics (LEP), a new curriculum adapted from Physics and Everyday Thinking (PET). PET is an inquiry-based, hands-on, conceptual physics curriculum that was developed for small enrollment discussion/lab settings. The Learning Physics (LEP) curriculum maintains the same research-based learning principles as PET but is suitable for classes taught in a larger lecture hall setting. LEP incorporates both videos of experiments and hands-on activities that students can perform on small desks typical of lecture settings. LEP also incorporates assignments where students submit written work and evaluate each other using a web-based system. This paper describes the curriculum, and presents LEP students' content performance and views about science. Outcomes are compared to students in courses using PET.


INTRODUCTION
Learning Physics (LEP) is a new inquiry-based, conceptual physics curriculum that is suitable for a lecture hall environment.LEP is one of a family of related curricula: Learning Physical Science (LEPS) [1,2], Physical Science and Everyday Thinking (PSET) [3], and Physics and Everyday Thinking (PET) [4].PET and PSET are intended for small, discussionbased courses, while LEPS and LEP are designed for large enrollment classes taught in a lecture-format without a lab component.LEP is a one-semester curriculum with a student-oriented pedagogy designed to enable students to develop a deep understanding of the conceptual themes of conservation of energy and Newton's laws, as well as other topics.LEP is also intended to enable students to develop an understanding of important aspects of scientific thinking and the nature of science (NOS).
In developing LEP, we sought to address two issues from the earlier curriculum, LEPS, which was adapted for large-enrollment settings from PSET [1].While it was designed to adapt many pedagogical features of PSET, LEPS lacked or only partially included two important features: opportunities for hands-on exploration of phenomena, and construction of scientific explanations.
Time and logistic constraints make hands-on experiments impractical in lecture halls; therefore, LEPS students gather evidence to support their thinking by observing videos of experiments.Pilot and field tests indicated advantages to videos, including the ability to pause, slow down, or replay the action; better visibility for the entire class; elimination of setup time; and elimination of failed experiments.Nevertheless, hands-on experimentation directly engages students in a significant practice of science, so we designed LEP to include opportunities for this [5].
Constructing and evaluating explanations is also an important scientific practice developed and practiced in PET and PSET.However, in large enrollment courses, it is unrealistic for the instructor to evaluate and provide feedback on written work.For this reason, in LEPS students only evaluate example written explanations for completeness and accuracy.A focus on explanations is thus retained while reducing the grading burden and the scope of students' role in the activity.However, LEP includes structured writing and evaluation tasks using a web-based tool called Calibrated Peer Review (CPR) [6,7].

DEVELOPMENT PROCESS
LEP and LEPS were developed using similar sets of design objectives and learning principles [1].These build on an understanding of learning as a complex process that builds on prior knowledge, and is

CURRICULUM DESCRIPTION
Compared to PET, LEPS consists of shorter, more focused lessons with more guidance from the instructor and no hands-on activities.In LEP, about half the time is spent on shorter, heavily guided activities similar to those in LEPS.The other half of the time is spent on longer activities where small groups engage in hands-on activities and model building.Due to the lecture hall environment only some experiments are feasible.Further, the instructor guides these activities more than in PET activities, and frequently checks the class consensus on observations and conclusions.Table 1 lists the contents of LEP and the pedagogical format of each unit.

Instructor guided lessons
Units 1, 4, and 5 consist of ~30-minute lessons in which the instructor guides the entire class using PowerPoint slides.These units are similar to Units 1, 2, and 3 in LEPS, which are described in [1].Class time is spent on 'clicker' questions (which students answer with electronic response devices, often following a discussion with nearest neighbors), gathering evidence from videos of experiments and simulations, and making sense and summarizing questions.As the instructor navigates through the slides, the students work in small groups to fill in data tables and answer corresponding questions in lesson sheets that guide their work.

Small group hands-on lessons
Units 2, 3, 6, and 7 consist of ~75 minute lessons where students engage in more sustained group work devoted to hands-on activities (either experiments or simulations) and model building.The large lecture format of LEP had significant impact on the implementation of hands-on activities.Specific issues included the lack of laboratory facilities (student desks instead of lab benches), limited time (75 minute classes instead of 2 -2.5 hours typical in PET), and large numbers of students (the authors taught LEP with classes of 70-100 students, or as many as 25 groups of students).Through field tests, classroom observations, and revisions, several key features were identified for successful hands-on activities in this environment.Experiments needed to be pedagogically significant and relevant, but also robust, reproducible, and reliable.(Experiments with static electric effects, for instance, proved to be unreliable for use in this setting.)It was important that the kits were small and inexpensive, given the large quantities needed.Students picked up the kits at the start of each class and they were used on small armtablet desks.Figure 1 shows a kit of materials students used in Unit 6. Activities were broken into short sections with frequent consensus checks (often with clicker questions).It is very difficult for an instructor to interact with 20+ groups during a 10-15 minute investigation, so minimizing difficulties with interpreting instructions was important.Photos and video of setups and instructions proved extremely useful in this regard.Finally, students used simulations where relevant; groups were formed to ensure at least one laptop was available in each group.
As an example of the hands-on activities in LEP, Unit 2 is devoted to developing a model for magnetism, and is similar to Chapter 4 in PET.The unit consists of four 75-minute lessons.In the first lesson, students test what kind of materials interact with a magnet, observe how magnetized and unmagnetized nails interact, and use a compass to identify the ends of a magnetized nail as magnetic poles.In the second lesson, the students develop an initial model of magnetism to account for what happens when a nail is rubbed with a magnet and the interactions between magnetized nails.Videos are used to introduce new observations: a magnetized nail is cut in half and interactions between the two pieces are shown.The students revise their models for magnetism to account for this evidence.Two more lessons use simulations, experiments, and videos to guide students through refining and applying their model.

Writing and Evaluating Explanations
CPR is a web-based tool that supports students' construction and evaluation of explanations, including appropriate diagrams.The system includes a peer review process with a training component to prepare students for reviewing.The assignments proceed in three stages.Students first explore background material and submit a response (text and image) via the web.Next students read a set of "calibration" essays prepared by the curriculum developers.These cover a range of quality and include detailed feedback; this stage prepares students to perform reviews of their peers.Finally, students evaluate the work of several classmates and their own work; the weight given to their scoring depends on the system's measure of their quality as a reviewer.Students are scored on their text (via their classmates' evaluation), their calibrations (were they consistent with the curriculum developers?), their peer reviews (were their reviews consistent with their classmates' reviews?), and the quality of their self-assessment (was their evaluation of their own work consistent with their classmates' evaluations of it?).CPR thus provides scaffolding for students to learn how to evaluate and construct explanations, but without an unreasonable grading load on the instructor.
LEP students were assigned five CPR tasks, an initial practice task and one task each for Units 1, 2, 5, and 6.The Unit 1 task required students to construct energy transfer diagrams for a chain of interacting objects and determine the energy efficiency.For the Unit 2 task, students read a description and watched a video of an initially unmagnetized iron nail being magnetized, then hit with a hammer so that it becomes unmagnetized.At each step the nail is slid towards a compass, along its E-W axis.The writing prompt instructs students to use the model of magnetism developed in class to draw diagrams of the nail and write an explanation for the observed behavior.The Unit 5 task involved using Newton's Second Law to analyze a race between two carts.The Unit 6 task required students to use ray tracing to explain a corrective prism system.
Developing specific writing prompts and very focused/specific evaluation questions helped ensure students performed well in the calibration stage, and reduced problematic reviews and student complaints.

EVALUATION Study design
During fall 2012 and spring 2013, three of us taught LEP in a total of 5 classes at San Diego State University, California State University at San Marcos, and Tennessee Technological University.At two sites, LEP was a course for pre-service elementary teachers; at the other site, it was a general education course.The average LEP class size was ~75 students.We assessed the impact of LEP on students' content knowledge, performance on a written explanation task, and views about science and learning science.We also administered the content knowledge and written explanation assessments to students in 14 PET classes at five other colleges/universities.

Results
LEP students made statistically significant content learning gains, had more expert-like views about science and learning science at the end of the course, and outperformed comparison PET students on writing scientific explanations.

Conceptual learning gains
To assess students' content learning gains, we administered a 12-item, multiple-choice conceptual assessment at the beginning and end of the semester.The items were representative of the material covered in both LEP and PET, and were developed by the project team in consultation with an external evaluator.Students took the pre-test in class; the post-test was included as part of the final exam.The average gain (post-pre) for N=326 matched LEP students was 4.2 items or 35%.The average normalized gain [(postpre)/(100 -pre)] was 48.6%.Based on a paired t-test, the LEP students made statistically significant gains.A control group of PET classes also took the assessment under the same conditions.Figure 2 shows the average normalized gain for LEP and PET students.In both curricula, the average normalized gain is in the mid-to upper-40%, suggesting that students in both courses learned nearly half of what they could have.The difference between LEP and PET students is not statistically significant.

Written explanations
An explanation question, asking for diagrams and a written narrative, was part of the final exam in the LEP classes and PET comparison classes.The question concerned Newton's Second Law; LEP and PET each spend a unit on forces and motion.A 5-point rubric was used for scoring students' responses.Two raters scored all responses (LEP and PET).After rater training, the interrater reliability was 90%.Each rater scored half of the students in each curriculum to reduce rating artifacts.Figure 2 shows average scores for the students in each curriculum.The difference between the averages is 1.2, which is statistically significant (based on a Mann-Whitney test).We attribute this to the use of CPR tasks in LEP, which PET did not include (though PET students construct explanations in class and for homework).Further exploration of the impact of the CPR tasks is ongoing.

Views about science and learning science
The Colorado Learning Attitudes About Science Survey (CLASS) [8] was used to gather information on students' views about science and learning.CLASS was administered as a voluntary online pre-and postassessment in the five LEP classes, although students did receive homework credit for completing the surveys.CLASS survey results are analyzed by comparing students' responses to an "expert" response.A post-pre difference is reported in terms of percent shift towards or away from the expert response; a positive shift indicates more expert like responses at the end of the semester.Figure 4 indicates the average shift of 6.9% for 259 LEP students who completed pre and post CLASS surveys in fall 2012 or spring 2013 [9].Data from PET and PSET classrooms (9 courses, N=368), reported by Otero and Gray [10], are included in Figure 4 for comparison.The PET/PSET students had increased attitudinal shifts compared to the LEP classes, but the difference between means (1.9%) is not statistically significant: p=0.21.Further, the LEP classes compare favorably to large, traditional introductory physics courses, where negative shifts or shifts of +1-2% have been reported in large algebra-and calculus-based physics courses, and courses for non-science majors [11].

DISCUSSION AND CONCLUSIONS
The LEP curriculum has been implemented in lecture hall environments, with classes of 60-100 students, while including hands-on exploration of phenomena, and supporting students' construction of scientific explanations.Students in the pilot LEP courses learned significant physics content and developed more expert-like views about science and learning science.Their performance on a conceptual content assessment was similar to comparison students in PET courses, which had smaller enrollments and 4 -5 hours/week of class time (versus 2.5 hr/wk in LEP).LEP students outperformed the PET students on an end-of-semester written explanation.

FIGURE 1 .
FIGURE 1. Students in LEP do experiments in class with simple materials.This kit is for Unit 6, Light Interactions.