How personal effort , student interactions , and instructor support relate to physics student satisfaction

We administered the Pedagogical Expectancy Violation Assessment (PEVA), a survey that provides information about students’ expectations, experiences, and attitudes pertaining to activities encountered in the classroom, to students enrolled in four sections of an algebra-based active-learning physics course at Eastern Kentucky University. We used results from the PEVA to investigate students’ overall satisfaction in the course. Students’ experiences of the 14 activities on the PEVA clustered into three factors: instructor-related activities, individual effort, and working with classmates. We found that, controlling for students’ expected final grade in the course, frequency of instructor-related activities and affect toward individual effort and working with classmates were good predictors of student satisfaction. PACS: 01.40.Di, 01.40.Fk, 01.40.gb


I. INTRODUCTION
Instructors in active-learning physics classes may have encountered discouraging comments from frustrated students who "had to learn everything by themselves" and who felt the instructor did not "teach," despite substantial learning gains.Our purpose in this ongoing study has been to understand the nuance behind such comments and to identify features of the classroom experience that are most related to student satisfaction in the course.

II. THEORY
The Pedagogical Expectancy Violation Assessment (PEVA) [1] is used to gather information about students' expectations and experiences about what happens in the active-learning classroom, based on the assumption that students would be faced with situations that could violate their expectations, and that when one's expectations are violated (i.e., either exceeded or not met), one's affect can change [2].In this line of research, we are specifically interested in students' expectations and experiences in active-learning physics classrooms.
The PEVA is given in pre-post format.The "pretest" probes expectations and initial attitudes, and the "posttest" probes experiences and attitudes near the end of the course.Sometimes, two "pretests" are given, two weeks apart, to gauge early shifts in students' attitudes.Using a seven-point scale ranging from "almost constantly" to "almost never," students report the frequency they expect to encounter (or did encounter) certain activities in the classroom, listed in Table 1.In previous work, we have seen that certain items factor somewhat consistently into clusters related to personal effort, work with classmates, instructor support (or relationship-building with the instructor), and how the instructor leads class time, based on students' reported frequency of experiencing those activities [3,4].
Consequently, we have made slight changes to the PEVA, such as removing a few items and reinstating two from previous versions: "Asking questions that I have" and "Presenting my problem solutions to the class."We anticipated that the former activity would factor with "individual effort" and that the latter activity would factor with "instructor-led class time," so that each factor would include three or four related activities.

PERC Proceedings,
In previous studies, we found that the better the students felt about how each cluster of related activities was implemented, the higher their overall satisfaction in the course [4].However, we were unable to disentangle whether the students enjoyed the activity itself or rather were pleased by how often they experienced it.For example, a high score on "laboratory activities" might indicate not enjoyment of labs at all, but rather relief that they did not spend much class time doing them.Thus, we have modified the PEVA to ask, "Which of the following statements best describes how you feel about this activity?"Students choose from three options (It is enjoyable; I don't enjoy it but I don't mind it; and I don't enjoy it and I don't like when we do it), which represent a positive, neutral, or negative affect toward that activity.Based on these new data, we expected that higher student affect towards frequently experienced activities would correlate with higher course satisfaction.
Potentially, expectations about how often activities would happen in class, the experienced frequency of those activities, and affect toward each activity could each contribute to students' satisfaction in the course.Thus, the purpose of this study was to determine which of those possible influences were most related to student satisfaction.

III. METHODS
The introductory algebra-based physics course at Eastern Kentucky University meets in 45-student SCALE-UP [5] style rooms for six hours a week for 16 weeks.While the approach to instruction is not strictly SCALE-UP in style, instructors integrate multiple modes of instruction including hands-on activities, clicker questions, and problem-solving sessions.Students enroll in the course primarily to fulfill program requirements, and they represent a wide range of programs from construction management to pre-medicine.In the Fall 2014 semester, four instructors (two female) each taught one section.They were free to choose activities and, to a lesser extent, the sequencing of the material taught.However, each instructor covered the same basic topics of kinematics, Newton's Laws, energy conservation, and rotational motion.
Students in all four sections were invited to participate in the PEVA to report their expectations and experiences in the course.The PEVA was administered online three times during the semester: week 0 (as the class began), week 2 (once students were introduced to the course), and week 14 (just before final exams).The earlier surveys asked about students' expectations, while the latter asked about their experiences.A total of 133 (79 female) students responded at week 0; 107 (64 female) students responded at week 2; 104 (62 female) responded at week 14.
As part of the week 14 survey, students were given an eight-question measure of course satisfaction, an extension of previous work [3].There were no problematic questions on the measure, which was was found reliable (α = 0.91).We summed the student responses on the eight items to create a total satisfaction score.This score ranged from 0 (extremely dissatisfied) to 48 (extremely satisfied), with a median score of 28, and was considered to be both continuous and approximately normal based on the Shapiro-Wilks test, W(102) = .98,p = 0.26.
Along with the PEVA, we also collected demographic information, including gender and the level of importance respondents placed on earning a high grade, learning physics, and having a good time in class.We also asked what grade students expected to receive in the course; we found in a previous study that students were accurate predictors of their course grade [3].

A. Factor analysis
We found that the items on the final survey clustered into only three factors based on students' perceived experiences in class (see Table 2), rather than the four factors that we had seen in previous work.We used a Principal Component Analysis with Varimax rotation and created factors with eigenvalues greater than 1.0, validating this choice with a visual inspection of the Scree plot.We selected items into a factor when that item had a score of 0.60 or greater on that factor and no score above 0.40 on any others, a conservative selection process [6].Items 7 and 14 each seemed to contribute to multiple factors; we thus excluded the items.Because those were the two items that we had added to the survey, the results we obtained were similar to those of previous studies with the major exception that the current study combined two factors ("instructor-led class time" and "instructor support") into one, which we call "instructor-related" here.
Curiously, when a factor analysis was performed on students' expectations at week 0, four factors emerged similar to those seen in previous analyses: "instructor support" included items 4, 7, 9, and 10, while "instructorled class time" included items 3, 8, and 13.
Apparently, students initially expected instructors to support the students in specific ways apart from how they were going to lead class time.For example, students who expect instructors to frequently provide feedback and encouragement might not also expect those instructors to frequently lecture or provide guidance for activities in class.However, those factors clustered together at the end of the semester, suggesting that students generally experienced all activities concerning the instructor approximately equally.Some of the instructor-related activities happen simultaneously for the entire class (e.g., lecture or problemsolving demonstrations), while others are highly individualized.Therefore, students interpreting all of those activities as being related hints that students' reported experiences were subjective.Students also might have interpreted all instructor activities to be manifestations of how supported students felt in the class.Thus, we speculate that two separate instructor-related factors may emerge when students believe that their instructor's responsibilities as a facilitator of learning are at least somewhat independent from whether they do more traditional activities (e.g., lecture or providing direct guidance for laboratory activities).

B. Course satisfaction
Because our goal was to uncover relationships between activities in the studio-style classroom and student satisfaction, we next explored how students' expectations at weeks 0 and 2, experiences at week 14, and affect toward the activities at weeks 2 and 14 all correlated with course satisfaction (no affect questions were asked at week 0).Because we were concerned with course satisfaction at week 14, we investigated the three factors generated from students' reported experiences in the course (Table 2).Doing so is consistent with previous analyses [3,4].
Affect towards "individual effort" at week 14 correlated positively with course satisfaction, r = .41,p < .001.Likewise, affect towards "working with classmates" at week 14 was correlated with satisfaction, r = .38,p < .001.The frequency of classmate interactions was weakly correlated with satisfaction, r = .22,p < .05.Course satisfaction was weakly correlated with how often students expected "instructor-related activities" at week 2, r = .26,p < .05,and strongly correlated with how often students reported experiencing instructor-related activities at week 14, r = .57,p < .001.Thus, the most satisfied students were apparently those with positive attitudes about individual effort and working with classmates, who expected and experienced more instructor-related activities, and who experienced more classmate interactions.
In previous work, students' expected grades significantly predicted course satisfaction [4]; we also found a significant correlation between students' satisfaction and their expected grade in the class at week 14, r = .31,p < .01,such that students who expected a higher grade tended to be more satisfied with the course.Expected final course grade was not correlated with their expectations, experiences, or affect toward any factor.
To better understand relationships among these identified variables (and to weed out potential spurious correlations), we next turned to multiple regression, with the goal of predicting students' satisfaction with the course.Because we wanted to control for the effect that expected grades might have on course satisfaction when we tested the relationships of the other variables, we performed the analysis in three steps.First, expected grade was regressed against course satisfaction, F(1, 100) = 10.71,p < .01.The multiple correlation coefficient was R = .31,R 2 = .097,meaning that approximately 9.7% of the variance in satisfaction was explained by the expected course grade alone.Next, we regressed all five potential predictors that were correlated with course satisfaction.Because the two predictors with the lowest correlation (frequency of classmate interactions at week 14 and expected frequency of instructor activities at week 2) did not significantly enhance the model, they were removed.
The final model showed that, in addition to expected grade, three variables were significant predictors of course satisfaction: attitudes toward individual effort, attitudes toward working with classmates, and frequency of instructor-related activities, all at week 14.The complete model was significant, F(4, 97) = 26.44,p < .001,R = .72,accounting for approximately 52% of the variance in satisfaction.Thus, while expected course grade did play a role in satisfaction, the other variables (attitudes toward individual effort and working with classmates and frequency of instructor-related activities) contributed a larger portion of the variance.We also confirmed that the section in which a student was enrolled (and thus, which instructor a student had) did not make a significant contribution to the model.

C. Student affect on individual factors
One benefit of the expectancy violations framework is an opportunity to learn students' attitudes and expectations when they enter a course and not simply how they feel once the semester is over.Thus, we were able to explore how students' attitudes regarding individual effort and classmate interactions may have changed during the semester.Some students seemed predisposed toward disliking one or the other of those factors, while other students apparently "learned" to dislike activities between weeks 2 and 14.These trends did not appear regarding the instructor factor, about which students consistently reported positive affect.
Because student affect varied within the other two factors, we used the criterion of answering "I don't enjoy it and I don't like when we do it" to at least one item in the factor to identify students with a negative affect regarding that factor.Table 3 shows the percent of respondents to each survey that fit that criterion.TABLE 3. Percent of respondents who had negative affect toward at least one item in the factor at each time.

Theme (item numbers)
Week 2 Week 14 Individual effort (1,12) 17.7% 26% Work with classmates (2,5,6) 8.4% 26% Five students who initially had negative affect toward the "individual effort" factor shifted to a more positive affect, two did not take the survey at the end of the semester, and 12 remained in the negative affect group; 44% of the members of the negative affect group in week 14 were in that group at week 2. Five students who disliked at least one activity in the "working with classmates" factor shifted into the positive affect group, while four remained the negative affect group.Therefore, most of the students in the negative affect group in week 14 were not in that group in week 2 for either factor.We therefore conclude that while some students may have been predisposed against certain activities, that predisposition did not solely determine their affect at the end of the semester.For example, further investigation revealed that an overwhelming majority (85%) of students who reported dislike for at least one item in the "working with classmates" factor disliked the laboratory activities that were done in class.Apparently, many students learned to dislike performing laboratory activities over the semester.
We then compared students with a affect in each factor to the rest of the sample.Specifically, we were testing the null hypothesis that there would be no difference between those two groups in the importance students placed on each of three priorities for the course during week 14: earning a good grade, learning physics, and enjoying inclass time.We rejected the null hypothesis for two cases.Students who held a negative affect regarding individual effort rated "learning physics" as less important compared to their peers, t(36.51)= -3.99,p < .001.Similarly, students who disliked working with classmates rated "enjoying my time in class" as less important compared to their peers, t(102) = -2.02,p < .05.
We also tested the null hypothesis that there was no gender difference between students with a negative affect in each factor and their classmates.We rejected the hypothesis only for one case: students who held a negative affect towards individual effort at week 14 were more likely to be female than their peers, z = -2.23,p < .05.No other gender differences emerged.

V. CONCLUSIONS
Our ongoing goal for this study has been to understand factors that are related to student satisfaction in active learning physics classes in order to provide insight and advice to instructors of such classes who face pushback.To that end, some details are coming into focus regarding students who are more likely to give such pushback.
Students who experienced more instructor-related activities were more pleased with the course, and students generally had a positive affect toward instructor-related activities.However, that factor included both traditional activities, such as introducing new material in lecture, and instructor support, such as providing feedback and encouragement.In this way, instructors face a challenge: they are likely to receive better course evaluations if they do more traditional behaviors, although doing so would undermine the active-learning philosophy of such a course.
Additionally, students who did not enjoy individual effort, and who did not enjoy working with classmates (most dramatically on in-class laboratory activities) at the end of the course were more likely to report low course satisfaction scores.Unfortunately, from our preliminary analysis, we were unable to uncover a way to predict which students would end up with a negative affect towards "individual effort" or "working with classmates" at the end of the semester.One interpretation of that finding is that while some students may have been predisposed against certain activities, other students develop a negative attitude toward activities in class.We speculate that a possible reason for that attitude shift is that they do not see tangible benefits from those activities.For example, because students who had negative affect toward individual effort tended to value learning physics less than their classmates, we might hypothesize that such students do not value the reward of learning that comes from individual struggle, especially if they do not see a benefit in their grade.
However, it is students' actual experiences and interactions that seem to shape their satisfaction in the class, not a particular predisposition, suggesting that instructors do have an element of control over student satisfaction.Therefore, any framing or affective maintenance that instructors do early in the semester should be supported and continued throughout the course.Such maintenance may reinforce the importance of struggle while connecting the value of physics understanding to students' everyday lives or future careers.

TABLE 1 .
Items on the PEVA administered in this study.

TABLE 2 .
Factors based on frequency of experience.