Efficacy of “ ALa-Carte ” Research-Based Curricular Elements

Many studies have shown the benefits of student-centered, interactive curricula when compared to traditional lecture approaches. However, wholesale adoption of research-based pedagogies may not accommodate differences in instructional resources across institutions. We aim to understand how interactions between several research-validated may affect learning. We have implemented a first-semester, calculus-based physics course that combines elements from several such curricula. Students' conceptual understandings were measured using the Force Concept Inventory (FCI) and showed significant gains for the modified course as compared to the traditional lecture course. We see that adopting elements of several different curricula may retain a positive impact on student learning.


INTRODUCTION
Interactive, research-based curricula have fostered improvements in student conceptual understandings compared to traditional, lecture-based pedagogy [1].However, the dissemination of interactive curricula that lead to improved learning (e.g.Ref. 2-7) has been slow.Factors that impede the wholesale adoption of research-validated curricula include: physical classroom space and resources, departmental and institutional structures and cultures, instructors' skills/training in enacting new pedagogies, and instructors' preferences to customize their courses [8].Some level of congruence between a curriculum and the local context is needed for an adoption to be successful or even attempted.More modular instructional components (e.g.Ref. [9][10][11] have been more widely adopted than whole curricula.
One approach is to adapt rather than adopt, by combining elements from several validated curricula to meet local needs and constraints.However, little is known about how the interaction of these elements might impact student learning.Benefits to learning may be retained, or some or all of the benefit may be lost.Case studies of combinations of curricular elements will lead to a better understanding of these interactions.
This paper presents a case study of a curricular reform at the University of Memphis.Student success rates in the introductory calculus-based courses have been alarmingly low (approximately 53% for the last 7 years).In order to address this low success rate, we designed a curriculum that utilizes elements from several research-based curricula, including SmartPhysics [12] and Modeling Instruction.Our research goal is not necessarily to determine the influence of individual curricular elements, but to determine how the combination of elements affects student learning.

COURSE DESCRIPTION
In Spring 2014, two sections of the first-semester calculus-based introductory physics course (PHYS 2110) were taught.One section ("Interactive", n=46) included elements from several research-validated curricula.The general philosophy driving these changes was to engage students actively in their learning of physics.Co-author Gire (a junior faculty member and with research interests in physics education) was the instructor of the course, and coauthor Goering was the teaching assistant.The second section ("Lecture", n=83) was taught in a traditional lecture format by a senior faculty member with more than 30 years of teaching experience and who has won the university's highest teaching award.
The students enrolled in this course are predominantly first or second year engineering majors and some mathematics and natural science majors.
Students in the Interactive section were asked to purchase online access to a SmartPhysics course, while students in the Lecture section were asked to purchase Knight's textbook Physics for Scientists and Engineers, 3 rd edition.and complete the associated checkpoints.The instructor reviewed students responses to the checkpoint questions and students' feedback about their understandings of the material covered in the prelectures prior to class.
Students in the Lecture section had no assignments before class meetings.The instructor did not expect students to read the relevant textbook sections before class.

Class Meetings
The Interactive section met for 85 minutes twice weekly in a large laboratory room with a 64 student capacity.Students sat at tables in groups of 3 or 4 students and a projector system displayed the material presented onto two large screens at opposite ends of the room.Class meetings included a Whole-Class Discussion (WCD) mode and a Small-Group Work (SGW) mode.

Whole-Class-Discussion Mode
The activities in the WCD mode included demonstrations, multiple-choice conceptual questions (ConcepTests) with peer instruction, and mini-lectures.
Demonstrations were done in front of the class with an Interactive Lecture Demonstration philosophy where students made predictions about the results of the demonstrations, engaged in peer discussions to justify these predictions, and compared their predictions with the actual results of the demonstration.
ConcepTests (multiple-choice conceptual or short calculational questions) were used with a Peer Instruction teaching strategy and either clickers or small whiteboards for reporting responses.For each question, the students were prompted to discuss (1) which ideas are important for answering the question, and (2) their reasoning for the option they chose.The small whiteboards were also used spontaneously during mini-lectures for making calculations that were then discussed with neighbors in a Peer Instruction manner.
Once or twice each class meeting, the instructor provided 5-30min mini-lectures to clarify or expand on points made in the pre-lecture.

Small-Group Mode
In SGW mode, students at each table worked collaboratively on solving selected physics problems.These problems included context-rich physics problems [13] and creating a physical description of a brief demonstration/experiment performed by the students each table.The student groups were asked to display their work on larger (2ft x 3ft) whiteboards.At the end of the activity, the instructors facilitated discussions among the groups.For the first month of the semester, the small groups were coalesced into two larger groups for "board meetings" [14].The students were expected to explain their reasoning, ask questions of other presenting groups, and answer questions posed by the board meeting leader.However, due to difficulties in establishing productive classroom norms for these board meetings, during the last part of the semester the course instructor facilitated more directed whole-class-discussions.
The Lecture section met for 55 minutes 3 days each week in a stadium style classroom to watch a lecture presentation and demonstrations.The instructor wrote on the large whiteboard at the front of the classroom while presenting his lecture and periodically asked Socratic questions and performed demonstrations.No teaching assistant was assigned to the course.

Homework
In the Interactive section, weekly homework assignments with quantitative physics questions were managed through SmartPhysics.Students received real-time feedback about their answers and some builtin hints and feedback suggesting specific errors that the student may have made.
Homework in the Lecture section was assigned from the textbook and solutions were provided, but homework was not collected.

Exams
The structure of assessments was different between the two sections.Six quizzes were administered biweekly in the Interactive section covering all of the material previously discussed in the course.Quizzes included 3-4 multiple choice questions and 3-4 long answer questions to be completed in 30 minutes.A comprehensive 2 hour final exam was given.
Four exams were administered in the Lecture section: 2 55-minute exams focused on an isolated subset of topics, a 55-minute comprehensive exam, and a 2-hour exam during finals week on rotational motion only.Exams included both multiple choice and long answer questions.

ASSESSMENT OF CURRICULUM
In order to assess changes in students' conceptual understandings, the students from both the Lecture section (n=23 matched pre/post-tests, response rate = 28%) and the Interactive section (n=23 matched pre/post-tests, response rate = 50%) were given the Force Concept Inventory (FCI) [15] in lab twice during the semester.The pre-test was given in the second week of instruction (the first lab meeting) and the posttest was given during the last full week of instruction for the spring 2014 semester.
The low response rate in the Lecture section is due to a larger fraction of students retaking the lecture course but not the lab (42% retakers for the Lecture section vs. 15% for the Interactive section).This is a selection bias in the data.The students included in the analysis are representative of the students who were taking the course for the first time rather than the full population of the course.Therefore, our analysis tells us about the effects of the instructional approaches on students with similar levels of post-secondary physics experience.However, we acknowledge that instructional approaches may impact the retakers differently and that the presence of more "retakers" in the lecture section may have an influence on the culture of learning in that section in interesting ways.
Although the response rates were low for both sections, comparisons of post-tests scores between research participants and the other students enrolled in the course indicate that the scores reported here are indeed representative of the course as a whole.
In order to assess other aspects of student learning in the course, pre/post CLASS responses and paired exam questions were also collected and individual problem-solving interviews were conducted.Analysis of these data is not presented here but will be reported in a future manuscript.

RESULTS
Due to the small number of students included in the analysis, pre-and post-test scores were tested for normality to determine if the data satisfies the assumptions of a parametric statistical test.The zscores of the four groups indicate no significant skewness or kurtosis (z < 1.96) and are therefore consistent with a normal distribution.A t-test on the FCI pre-test scores (see Table 2) shows no statistically significant difference between the pre-tests scores of the Lecture and Interactive sections, t(44) = 0.229, p = 0.820.Therefore, the student populations before instruction have similar levels of conceptual understanding before instruction, and differences in the post-test are likely due to differences in instruction.
A mixed-design repeated measures ANOVA of FCI scores was performed.A within subjects comparison shows that the FCI scores for both sections increased from pre-to post-instruction, F(1,44) = 61.8,p <0.001.However, a statistically significant interaction between the FCI Score and Section is detected, meaning that the gains in FCI scores are different for the two sections, with the Interactive section achieving a higher gain, F(1,44) = 6.13, p = 0.017, η p 2 = 0.122.Indeed, we see a statistically and pedagogically significant 18% difference in normalized gain between the Interactive and Traditional sections, t(44) = 2.50, p = 0.016.These normalized gains are consistent with those previously reported for interactive, research-based curricula [1,16].

LIMITATIONS
This study examines the combined effect of multiple research-validated curricular elements, rather than isolating the effect of each element individually or individual pairs of interventions.Controlled studies examining the interaction of pairs of interventions would provide more detail about specific those interactions.
This study makes comparisons of conceptual understandings as measured by items on the FCI.FCI scores are commonly used as a proxy for overall conceptual understanding in introductory mechanics courses, although the FCI does not cover all the topics in the course, only kinematics, forces and Newton's laws.
The different class sizes and different instructors for the Lecture and Interactive sections are confounding variables of the study.We argue that the experience and the University's recognition of the Lecture instructor's teaching skills lends strength to our claim that differences in student learning is due to the interactive nature of the pedagogy rather than an instructor effect.

CONCLUSIONS
Students in the Interactive section, which included interactive instructional elements from several research-validated curricula, demonstrate better understandings of physics concepts of force and motion than students in a traditional lecture section.The comparison of scores with the Previous sections suggest these improved understandings may be largely due to the use of ConcepTests with Peer Instruction, although other differences between the sections may have contributed.This study provides a proof-of-concept that departments need not adopt researchbased/validated curricula wholesale, but that improved student learning outcomes may be achieved by "mixing and matching" elements of these curricula to fit local needs and constraints.

FIGURE 1 :
FIGURE 1: Pre-and post-test FCI averages for both sections along with the resultant normalized gain.Error bars indicate the standard error.We also made comparisons between the Spring 2014 sections and five sections taught by author Gire in previous semesters (Fall 2010-Fall 2012, n=254).Those sections did not include the full range of interactive elements present in the Spring 2014 implementation, but did include ConcepTests with Peer Instruction using clickers and small whiteboards

TABLE 1 .
Skewness and kurtosis for FCI scores.

TABLE 2 .
FCI & Subset scores.Scores are reported as percentages and uncertainties are standard error of the mean.