The Impact of Course Structure on eText Use in Large-Lecture Introductory-Physics Courses

. Course structure - the types and frequency of learning activities - impacts how students interact with electronic textbooks. We analyze student-tracking logs generated by the LON-CAPA learning management system from nearly a decade of blended large-lecture introductory-physics courses at Michigan State University, as well as one on-campus course from MIT. Data mining provides estimates of the overall amount and temporal regularity of eText use, i.e., weekly reading versus review immediately before exams. For all courses studied, we compare student use of eTexts as it varies with course structure, e.g., from traditional (three or four exams, eText assigned as supplementary) to reformed (frequent exams, embedded assessment in the assigned eText). Traditional format courses are accompanied by little eText use, while high reading levels persist throughout reformed courses.


INTRODUCTION
Course structure, i.e., the types and frequency of learning activities, has a dramatic impact on students in introductory science courses [1].Improved performance and attitudes have been shown to accompany changes from traditional to frequent exam formats [2], as well as in the replacement of traditional lectures with active-learning activities [3,4].Similar gains have been shown to accompany the replacement of physical-text pre-lecture activities with online Media Modules [5,6].Missing from these studies are details describing how student behavior changed in conjunction with course structure.Such behavioral changes provide a link between course structure experiments and outcomes, potentially informing instructors, researchers, and even students to those habits leading to optimal outcomes.eTexts provide an interesting context for understanding the impact of course structure on student behavior.Data stored by Learning Management Systems (LMS) makes it possible to illuminate student habits and behaviors traditionally accessed via self-reported surveys.Additionally, research showing generally low use of textbooks in introductory physics courses [7,8], accompanied by the cultural perception that "students don't read the textbook", provides a baseline for experiments with methods of increasing eText use and engagement. 1 As the trend to incorporate technology into introductory courses continues, many instructors are facing the question of how best to integrate eTexts into their courses.Understanding how course design influences student use provides perspective for this issue.
This study explores the relationship between course structure and eText use.The LON-CAPA LMS provides records of student interactions with eText pages, allowing for measures of daily activity and fractional use.The data studied are nine large-lecture, first-semester introductory-mechanics courses from Michigan State University, and a single mechanics reform course from MIT using a flipped classroom.These courses are classified according to their relative differences in course structure and eText measurements are presented with respect to this classification.

COURSES AND METHODOLOGY
All courses within this study are centered around material from first-year introductory mechanics.Michigan State University (MSU) courses are representative of typical large-lecture introductory-physics courses and  [9,10,11].Each of the above courses utilized LON-CAPA (http://www.loncapa.org) in providing weekly homework and an associated eText; both resources were easily navigable at each student's initial log-in.Students view their respective eTexts as modularized html pages containing mainly static text and illustrations.MSU courses contained pages from the MSU Multi-Media Physics (http://www.pa.msu.edu/bauer/mmp/) group's introductory mechanics volume, which is similar to content from traditional introductory physics textbooks.Instructors of each course had freedom in which pages (chapters) to assign.Table 1 provides the total number of pages.The MIT course's eText was developed by the RELATE group at MIT and covers the typical topics in a calculus-based introductory physics [10].However, the reformed nature of the MIT course introduces certain concepts out of order compared to traditional texts, e.g., beginning with Newton's Laws instead of Kinematics.
In order to facilitate exploration of the link between course structure and eText use, all courses are classified as Supplementary, Traditional, or Reformed, based on the major differences in course structure: • Supplementary: eText secondary alongside a physical textbook, 2-3 exams, no embedded assessment.• Traditional: eText assigned as primary text, 2-3 exams, no embedded assessment.• Reformed: eText assigned as primary text, 6-7 exams, embedded assessment within eText.Data stored in LON-CAPA activity logs (tracking or server logs) allow the extraction of meaningful behavioral data [12].Methods in this study involve analysis of daily eText activity and the number of unique page accesses over the course.For time-series calculations, we sort the activity logs into student and time sorted files, simplifying parsing and the aggregation of relevant activity.Unique student accesses per page, as well as the total number of accesses, are tracked and stored while parsing these time-series.Preliminary analysis of fractional usage in eTexts has been applied to the data in this study and for open-online courses [13].

RESULTS
When do students read the eText?Plotting daily page views provides perspective on when students are reading.Figure 1 shows the daily page-views per student for two courses from the Supplementary, Traditional, and Reformed categories (course pairs are displayed vertically).Starting with Supplementary (note difference in y-axis scale), prominent spikes represent activity immediately before exams (exam frequency in Tab.1), implying that the majority of students access the eText only days before exams.Traditional has similar examination peaks, but with a greater overall number of views per student between exams.In Reformed, spikes associated with 6 midterms and a final are indeed present, but much less discernible due to prominent activity in non-test weeks.
Of equal interest in Fig. 1 is the decrease in daily activity following the first exam in Supplementary courses.This signal occurs in all three Supplementary courses and Traditional #1.This effect is less distinct in Traditional #2 and is not apparent at all in the Reformed courses.We hypothesize this to be a meaningful signal in understanding student perceptions of eText, e.g., after the first exam students see less value in regular reading.Such signals are apparent in preliminary analysis of lecture slide accesses in similar courses from MSU [2], as well as eTexts in open-online courses of varying format [14,15].
How much of the eText are students reading?Fig. 2 depicts the fractional usage of the eText in each course via complimentary cumulative distributions (CCDFs). 2  We indicate the mean fraction of eText accesses in each course via a single filled circle on each CCDF, e.g., the mean fraction accessed in the MIT course is ≈ 0.80.Supplementary courses (black) have the lowest eText use (mean access ≈ 0.10) with little variation among the courses.Reformed courses (blue) have the highest overall use (mean accesses ≈ 0.80), but with greater 2 We choose to plot CCDFs of unique eText page accesses as it allows all data to appear in a single plot.Each CCDF is read as the fraction of students f N accessing greater than f e fraction of available eText pages.N fraction of students viewing > f e q q q q q q q q q q Reformed MIT Traditional Supplementary Mean fe Accessed q FIGURE 2. Complimentary cumulative distributions of fractional eText usage, i.e., the fraction f N of students who have accessed greater than f e fraction of their respective eText pages.Data are color coded via the discussed categorization (Tab.To better understand how page access are distributed in the material, Figure 3 shows the mean views per chapter pages across each course category.Course structure effects are again apparent; Reformed courses are generally higher compared to Traditional.The number of views per page for Reformed courses is generally two or higher, while traditional ranges between one and two.Supplemental courses have relatively low views per page within each chapter.Also noticeable is the downward trend in eText use for Traditional and Reformed courses, possibly pointing toward a decrease in overall engagement throughout the semester (chapters were assigned in linear order with respect to the x-axis).Alternatively, instructors may have put less emphasis on later chapters.

CONCLUSIONS
Course structure impacts how students interact with eTexts.In traditionally formatted courses, large spikes in daily activity indicate that most students use the eText as a review tool before exams.Courses with Reformed formats have increased daily use and a periodicity pointing toward weekly use rather than review before exams.In addition, the fraction of eText accessed is higher in Reformed courses, indicating a positive correlation between eText accesses and course structure.As eTexts push toward replacing physical textbooks in introductory physics courses, we hope that this study will influence both the design choices of instructors and future research efforts aimed at understanding eText use.
The link between student behaviors relative to course structure is an important first step in understanding how eText (or textbook) use impacts learning outcomes.Once behavior is well understood, e.g., regular reading versus reading just before exams, subsequent connections can be made between individual behavior and learning outcomes.These connections will allow instructors to optimize learning in courses via structural considerations.In that same regard, students may also benefit from understanding how behavior leads to more learning.

FIGURE 1 .
FIGURE 1. Plots depicting daily page-view activity per student A(t)/N for two courses from the Supplementary, Traditional, and Reformed categories.

TABLE 1 .
Descriptive statistics for three major course structure differences.Supplementary, Traditional, and Reformed categories emerge (course labels) based on eText assignment, Exam Frequency, and presence of Embedded Assessment.
all took place in their Fall semester.MSU courses were aimed at two relevant student populations: algebra-based courses for university-wide enrollments and calculusbased courses consisting of mainly pre-medical students.The single MIT course is a reformed version of introductory physics designed for students not passing the required introductory mechanics course in the Fall semester.Students are introduced to the MAPS pedagogy and are lead through various online activities

Table 1
highlights these differences in more detail.The secondary implementation of an eText only occurs in the Supplementary courses.Standard large-lecture exam structure, i.e., 2-3 midterms and a final, occurs in both the Supplementary and Traditional courses.Reformed courses contain two structural changes, namely, frequent exams (6-7 midterms and a final) and embedded assessment in the eText.Unfortunately, the simultaneous changes to course structure do not allow for separate analysis of frequent exams and embedded assessment.
Mean views versus chapter in the MSU eText, normalized by number of chapter pages and counting number of students accessing at least one page within each chapter.Error bars are standard deviations.The Custom chapter supplemented or replaced content in the some of the Reformed courses, e.g., the 2D Motion and Temperature.similar to the Reformed courses.The MIT course (red) resides within the Reformed regime, providing a first indication of a cross-course and cross-institution link between course structure and student behavior.