Using Expectancy Violation To Investigate Student Dissatisfaction In Studio Physics

Physics students, especially those in pedagogically reformed courses, are sometimes dissatisfied with the course structure. Expectancy violation (EV), which arises when students’ pedagogical expectations are not met, is a possible cause for this dissatisfaction. Previous research has identified instances of EV in reformed physics classes, but detailed investigations are needed to determine how EV relates to course satisfaction. In this pilot study, we paired a modified Pedagogical Expectancy Violation Assessment (PEVA) with a course satisfaction questionnaire to measure students’ perceived expectations, experiences, and satisfaction in three different physics courses: algebra-based SCALE-UP style at EKU (N=61), calculus-based lecture at UCF (N=179), and calculus-based SCALE-UP at UCF (N=88). Course satisfaction was positively correlated with performance and the number of positively-perceived EVs and negatively correlated with the number of negatively-perceived EVs. Students’ opinions about the frequency of a few particular activities predicted a large amount of the variability in course satisfaction. While inconclusive, these preliminary results guide reform efforts of the PEVA.


INTRODUCTION
Although research-based instructional strategies have successfully improved student learning, instructors in courses that implement them often struggle with student dissatisfaction and pushback.For example, SCALE-UP students have felt that they could have earned the same grade with less work in a traditional class [1].Such challenges merit our attention because negative student affect may lead instructors to give up on research-based instructional practices in the classroom [2].
One lens for understanding student dissatisfaction is expectancy violation (EV).An EV framework, or something similar, has been used successfully in the fields of communication [3] and business [4].Basically, EV postulates that people have expectations for each situation they encounter.Expectations are "violated" when actual experiences do not match those expectations.Violations may cause people to be pleased or upset, depending upon their "valence" of the experience: whether they perceive the mismatch to be positive or negative.The EV framework is attractive because it complements epistemological research and is actionable.By understanding students' expectations and their perception of their experiences, instructors can modify the framing and scaffolding of their courses.
In previous studies, we and others found that while students' expectations were not met for certain activi-ties, such as the frequency of lecture, in SCALE-UP style courses [5,6] and preservice teacher classes [7], instructors were sometimes able to shift students' initial expectations to better match their eventual experiences.Those studies also hypothesized an as yet untested connection between EV and course satisfaction.Here, we describe progress toward understanding such a connection.We present some of our preliminary findings, discuss their potential implications, propose substantial changes to the instrument, and discuss future research directions.

METHODS
At Eastern Kentucky University (EKU), introductory algebra-based physics meets in 45-student SCALE-UP style rooms for six hours a week for fourteen weeks.Multiple modes of instruction, including hands-on activities, lecture, and problem-solving sessions, are integrated together.Two sections of algebra-based physics were surveyed twice: at the end of the first and start of the twelfth week (N = 61 responded to both surveys).
The surveys were modified versions of the Pedagogical Expectancy Violation Assessment (PEVA) [5], including 18 items representative of activities that were typical of this course at EKU. Students reported their expectations and experiences in the following manner: during the first survey, they used a Likert scale that ranged from 0 (almost never) to 6 (very often) to a) retroactively report how often they expected to do each activity when they enrolled for the course and b) report their current expectations a week into the course.During the second survey, students reported both how often they engaged in each of the 18 activities and their "valence" for how often they engaged in each one.When reporting valence, students used a Likert scale that ranged from -3 (very displeased) to +3 (very pleased).The valence section was a new addition to the traditional PEVA.
On the first survey, students indicated their minimum satisfactory grade in the course.During the later survey, students reported the grade they expected.
All responses were de-identified but assigned a code so they could be paired; doing so allowed us to compare responses with students' test scores as a measure of their performance in the course and also use students' responses in validation interviews.A total of four students from EKU were interviewed, two from each section.
At the University of Central Florida (UCF), participating students were enrolled in second-semester calculus based physics, either in a traditional course that included separate lecture and laboratory components ("UCF-Lec"; N = 179) or one based on SCALE-UP ("UCF-SU"; N = 88).Students at UCF were only surveyed once, during the final two weeks of classes.They provided their initial expectations when enrolling in the course (retroactively), their experiences, and their valence for each item.Two additional activities, specific to the courses at UCF, were added to that survey.
Students at both universities were also given an eightquestion measure of course satisfaction that included items such as, "I looked forward to coming to class each day," and, "Overall, I am very satisfied with this course."Cronbach's alpha was above 0.9 at both EKU and UCF, indicating that it was internally very consistent.
A factor analysis of the 18-item PEVA based on students' experiences at EKU resulted in the identification of four themes among the activities: instructor-led work, relationship-building with the instructors, discussions with peers, and self-motivated work.In future work, we will revise the PEVA around these factors, taking into consideration student comments and preliminary results.

Satisfaction Is Related To Performance
Performance in the algebra-based physics course at EKU, measured by the students' scores on four chapter exams and the final exam, was correlated with their satisfaction in the course (r = .39,p < .01);students who performed better in the course tended to be more satisfied.Similarly, students who reported a higher expected final grade in the course had higher satisfaction (r = .38,p < .01).Students' predictions of their final grade were strongly correlated with their actual performance (r = .71,p < .001),which provides a good validity check for their responses.Furthermore, students whose predicted final grade for the course was lower than their minimum acceptable grade were significantly less satisfied with the course than those whose final grade was equal to or higher than their minimum (p < .05).Only about 10% (7 of 72) of the respondents from the first survey at EKU said that they would have been satisfied with a grade lower than a "B," although only about half of the students in each section actually earned an "A" or "B." This correlation between satisfaction and performance needs further investigation and can be interpreted multiple ways.Perhaps course satisfaction is a consequence of course grade; in other words, students who receive an A may be satisfied with the course regardless of their opinion of the different activities.Alternatively, highestperforming students may be the ones who buy in and figure out how to learn in the new setting, reaping the benefits originally intended in the reformed course.One of the themes that emerged when we factored the PEVA was related to self-motivated work.We hypothesize that students who report high levels of satisfaction with that factor will also have high performance and course satisfaction.Future work will seek to explore and disentangle this relationship between satisfaction and performance.

Satisfaction Is Related To "Valenced" Expectancy Violations
Violations were indicated when there was a mismatch between how frequently an activity was expected and how frequently it was experienced by a given student.The connotation of the word "violation" might lead us to expect students to be unhappy whenever a violation occurred, but the total number of EVs reported by each student was not correlated with their satisfaction in any of the three courses studied.Because EVs can be perceived as either positive or negative, we asked students to report how favorably they perceived the frequency with which they did each activity ("valence").When students provided a negative (positive) valence for an item for which he or she had experienced a violation, we called that a "negatively (positively) valenced EV." Not surprisingly, students who were more pleased with how often they did each activity in the course were more satisfied with the course overall.Moreover, course satisfaction was negatively correlated with the number of negatively valenced EVs and positively correlated with the number of positively valenced EVs (see Table 1).Instead of only looking at the number of violations per student, we needed to understand whether students felt positively or negatively about those violations.Students whose expectations were violated but who were pleased with the resulting experiences generally rated the course more highly than those who were displeased with their experiences.However, expectancy violation and valence are not, in general, independent.One way to explore their relationship is to investigate each item on the PEVA.
For example, students at EKU who "paid attention to lecture" less frequently than they expected were significantly less pleased about the frequency of lecture than those who received as much lecture as they expected (p < .05).However, there is reason to believe that instructors can influence whether that EV plays a role in students' approval of the course overall: for that item on the PEVA, students' expectations after a week in the course and their overall experiences were significant predictors for their valence, but students' initial expectations were not.Thus, it appears that instructors set the stage for their students: a real effect of instructors' framing of the course may be to shift students' initial expectations regarding the frequency and purpose of lecture, and that framing may improve students' overall satisfaction.
However, more research is needed to determine whether general patterns exist between expectancy violations and students' valence scores.Is it more important to meet students' initial expectations, to shift those expectations during course orientation so that they match students' eventual experiences, or to focus instead on changing students' epistemic beliefs of how class time should be spent?To better understand such dynamics in the future, we will obtain valence scores for all activities on both surveys, in addition to asking whether students would have preferred doing each activity more or less.

Satisfaction Is Predicted By Valence Scores On Certain Items
At EKU, total course satisfaction was predicted by students' valence scores on three of the 18 items on the PEVA: Solving physics problems individually (p < .001),doing "hands-on" physics activities (p < .001),and paying attention to lecture (p < .01).The linear regression model including those three items had an adjusted R-square value of 0.56, meaning that the valence for those three items together explains 56% of the variance in the course satisfaction scores.
At UCF, different models predicted course satisfaction for lecture and SCALE-UP sections.For the lecture sections, six items were significant predictors, with three of those being strongly significant (p < .001):Discuss responses to clicker questions, pay attention to lecture, and do "hands-on" physics activities.Together, those three items had an adjusted R-square value of 0.50.
For the SCALE-UP sections, four items were significant predictors, but only doing "hands-on" physics activities was strongly significant (p < .001).Discussing coursework with instructors outside of class, answering questions from instructors, and reviewing notes from class were also significant (p < .05),and the total adjusted R-square for that model was 0.59.
Because of the differences between the courses, it is not surprising that different items proved to be most relevant.At EKU, each 110-minute chapter exam consisted of five problems.A substantial amount of time was devoted each week for practicing problem solving both in groups and individually, while instructors provided direct feedback.However, students who were unhappy or neutral (providing a valence score that was negative or zero) with the amount of time spent in class on individual problem solving reported having done less of it (m = 3.7 vs 4.5, p < .01)than those reporting positive valence.Instructors did not force students to spend time working problems individually, but each student was provided with the same opportunity.Also, students at EKU reported as a whole that they spent less time solving problems individually than they had expected, even after orientation, despite a shift in their expectations during the first week.Thus it follows that students who chose to spend more time working problems individually in class were more satisfied with the course.
One of the activities added to the PEVA at UCF was "discussing clicker questions."In UCF-Lec, valence for this item best predicted course satisfaction.Students who were unhappy or neutral about the amount of time spent discussing clicker questions both expected to do this less (m = 2.9 vs 4.3, p < .001)and experienced it less (m = 2.4 vs. 4.4, p < .001)than those who were pleased with the amount of time spent on it.Students followed a similar pattern as with individual problem solving at EKU: although ample opportunities were provided to discuss clicker questions, students who were most dissatisfied apparently did not take advantage of them.However, it is unclear whether they disapproved of doing that activity and thought they had done it too much, or whether they wanted to do it more.This situation emphasizes the need to ask students about the amount of time they would have liked to spend on each activity.
Paying attention to lecture stood out both at EKU and UCF.In all classes, students were happier with the time spent on this activity when it was done more.The possible importance of shifting students' expectations regarding lecture by framing the course properly was discussed in the previous section.Finding valence on this item to be a predictor at EKU was a surprise because the course was designed to de-emphasize lecture; in UCF-SU, satisfaction with how often students paid attention to lecture was not a significant predictor of course satisfaction, which meant that there were other factors that were more relevant (such as spending time doing hands-on activities).
One possible explanation as to why it was so important at EKU is that the course itself relies on lecture more than we initially thought.Another possibility, supported with validation interviews, is that this item is not wellphrased.Perhaps students are focusing on the "attention" part, or they are considering multiple activities (e.g., introducing course content, discussing clicker questions, or holding class discussions) to be wrapped up in that single item.Based on these findings, we are planning to remove this item and replace it with more specific activities such as the instructor introducing course content and demonstrating problem solutions.As we revise the PEVA based on four factors that emerged from these preliminary results (as mentioned in the "survey design" section), both of those individual items will contribute to the "instructor-led work" factor.If valence for that factor is a strong predictor of course satisfaction at EKU, that will be an indication about how significant the students perceive the role of lecture to be in that course.

CONCLUSIONS AND FUTURE WORK
One of the major strengths with the PEVA is that it allows us to investigate students' perceptions about a variety of activities that are done in class.However, it is important that students are able to interpret these items as intended and that such items are representative of what occurs as a part of the course.For this implementation of the PEVA, we chose items for EKU and UCF that we expected to be relevant and interesting.However, certain items, such as "pay attention to lecture," provide too much variability in interpretation and should be changed.Furthermore, in a preliminary factor analysis, along with patterns of students' responses, we saw larger factors begin to emerge from the PEVA.We plan to revise the PEVA to emphasize such factors so that we can learn what types of activities in physics classes meet the most resistance and how students' expectations and experiences shape that resistance.
Because we found that course satisfaction is correlated with performance, we expect factors that correlate strongly with individual performance to have an impact on course satisfaction, at least at EKU.Specifically, because the number of valenced violations are correlated with course satisfaction, we may find that supporting students to understand certain instructional methods is useful and effective for some factors but not others.That support may take the form of explicitly demonstrating how the methods help students learn, scaffolding students' epistemological growth, or simply fostering an environment where academic risk-taking in that factor is encouraged (e.g., by rewarding students who ask questions of the instructor).If, for a particular course, interacting with classmates helps students perform better, but traditional instructor-led activities like lecture and labs do not, then results from the PEVA might tease out how to prepare students for that course as opposed to one where interactions with classmates are not as useful.
Activities whose valence score is a significant predictor of course satisfaction may be of particular interest to instructors concerned with student pushback.Instructors may benefit from improving students' perceptions of those activities, for example by providing evidence that the hands-on activities help on exams.Alternatively, instructors may adjust how frequently those activities occur, for example by increasing the number of lectures.
These preliminary results will help us reshape the survey instrument that we will continue to use to investigate students' satisfaction in physics classes, with the hopes that it will arm instructors with specific recommendations for areas to focus their support of students' expectations and epistemological growth.

TABLE 1 .
Correlations between course satisfaction and valence, positively perceived EVs, and negatively perceived EVs.All are significant at the p < 0.001 level.The mean value for each group is given in parentheses.