They're not buying what we're selling: comparing student-requested supports with instructional practice during group work

Studio physics is an interactive teaching method that benefits students’ learning over traditional physics teaching approaches. A hallmark of studio physics is that instructors minimize lecture time and maximize time spent with students working in groups. However, students need to be well-supported in the group discussions to achieve optimal learning outcomes. In this study, we distributed a student survey and conducted instructor interviews to investigate the support(s) students requested so they could actively engage in group work. The student survey data revealed that students wanted instructors to be approachable during group work and to have a mechanism that guarantees equal group contribution. However, there is a mismatch between the support provided by instructors and the detailed support requested by students. Unified and rigorous guidance for instructors is a necessity for students to actively engage in effective and inclusive group work.


I. INTRODUCTION
Research shows that courses with interactive engagement are more effective at promoting student learning when compared with traditional teaching approaches [1]. Education researchers and curriculum designers have developed various methods to introduce interactive engagement into physics classrooms. Studio physics is a prevalent approach with many studies exploring the corresponding implementations [2,3,4,5] and student learning outcomes [6,7]. Studio physics courses often feature sessions merging lecture, lab, and recitation, relatively small classes or sections, and collaborative group work. Compared to traditional teaching approaches, studio physics courses result in better conceptual understanding measured by research-based inventories [4,8]. However, the effects on problem-solving performance and scientific attitude are controversial [5,6]. Since studio physics usually involves students collaboratively working on instructional tasks to fulfill the learning objectives in the course, the learning outcomes of studio teaching is dependent on how instructors facilitate group work practices in their courses.
The physics department at the participating institution implemented a studio physics program for all calculus-based introductory physics courses at the main campus in the Fall 2019 semester. This studio physics program mainly follows the SCALE-UP model [9]. Faculty members are not required to teach the same content in different sections of a course while common lecture slides are provided. Due to infrastructure limitations (e.g., TA availability, number of faculty members, sets of equipment), physics courses at regional campuses often are not taught in the studio physics format. As part of a larger project to support physics instructors with implementing group work effectively and inclusively, the purpose of this study is to highlight student and instructor perspectives about group work in an introductory physics course to investigate the correspondence between their perspectives.

II. METHODS
This is a mixed methods study combining analysis of instructor interviews and student surveys. After receiving IRB approval, we emailed instructors who were currently or had previously taught introductory physics courses (i.e., conceptual, algebra-based, introductory physics for the life sciences (IPLS), and calculus-based courses all for non-physics majors) with group work components from all campuses. The group work activities varied across sections but consisted of either laboratory activities (Instructor 1) or a mixture of conceptual and quantitative tutorial problems (Instructors 2 and 3). All participating instructors were asked to invite their current students, who were 18 years of age or older, to participate in the pre-and post-surveys regarding their attitudes and beliefs about group work via a Qualtrics link in the courses' learning management system. Near the end of the semester, three instructors were interviewed regarding their current group work instructional practices. These instructors were selected because of their interest in the study, and they were using group work in their courses in some fashion. Additionally, we collected information about the instructors' background experience in relation to group work and studio style physics courses. Students were not compensated for participating in the study, while instructors received $20 for participation.
The semi-structured instructor interviews were approximately one hour in length and focused on three topics: 1) instructors' current instructional practices; 2) thoughts about a group work guide that is in development; and 3) instructors' prior teaching experience. We focused our analysis on the first topic for this paper. The student preand post-surveys were developed by the co-authors and both surveys took approximately 15 minutes for students to complete. The pre-and post-survey content consisted of multiple formatted questions (e.g., Likert-style, multiplechoice, free-response) regarding the students' experiences with group work and their attitudes. This information served as baseline data to be compared later in the study. The postsurvey questions were tailored to address the supports requested from the pre-survey. Other than the demographic questions, there were only two questions asked in both preand post-surveys and they are all included in Table II. In our student survey data set, we received 244 responses for the pre-survey (response rate: 59.2%) and 167 responses for the post-survey (response rate: 55.7%). We removed 34 responses from the pre-survey and 3 responses from the postsurvey because these participants did not complete the survey; 27 respondents from the pre-survey and 10 respondents from the post-survey because they did not consent to participating in the research study; and 7 respondents for the pre-survey and 13 respondents from the post-survey because these participants did not correctly  Table I includes the number (N) and percent for gender (F is female, M is male, N is non-binary, PN is prefer not to answer), race/ethnicity (N is American Indian/Alaskan Native/Native Hawaiian/Other Pacific Islander, A is Asian, B is Black/African American, H is Hispanic/Latino/a, W is White, M is multiple, and PN is prefer not to answer), disability identity (C is cognitive, H is Health, HE is hearing, V is visual, EM is emotion/mental health, P is physical/mobility, N is none, M is multiple, and PN is prefer not to answer).
answer the attention check question. A total of 176 participants were included from the pre-survey and 141 participants from the post-survey. The students who participated in this study reported their gender, race/ethnicity, and disability identity. Table I includes the pre-and post-survey demographics breakdown for the final set of student participants. Additionally, 25 participants completed both the pre-and post-surveys, which qualified for repeated measurements. We recognize that only a small portion of our participants completed both the pre-and postsurveys possibly due to a lack of incentive.
The instructor interviews were analyzed using Content Analysis [10] by the first author and agreed upon by the coauthors before publication. Then, two co-authors (M.M.P. and E.M.S.) used the method of Thematic Analysis to analyze the free responses from participants about the support(s) they felt they needed to be successful in group work. Each coder separately identified themes and aligned each response with a theme. From there, each coder discussed the interpretations they disagreed upon to come to an understanding of the responses. Overall, in the prediscussion, the two co-authors agreed on all the alignment of participant responses into themes for all except 7 alignments (out of 210). All four authors agreed on all major themes (i.e., themes representing more than 10 participants' responses) pre-discussion. After that, a quantitative analysis took place by the first author (X.W.) via chi-square analysis of the two questions appearing in both the pre and post surveys. The analysis assumed all student participants in preand post-surveys were randomly selected from a larger introductory physics student population. For the 25 student participants who completed both the pre-and post-surveys, chi-square analysis was conducted by the first author to probe their attitude changes.
The validity of our interpretations was supported by multiple researchers from different identities who contributed to the data collection, analysis, interpretation, and writing, as well as by having identities that varied along gender, race/ethnicity, disability status, country of origin, dominant language, role within the university, and previous teaching experience. A limitation of our study is that the design, data collection, and analysis of this study all occurred during the COVID-19 pandemic. In the semester the data was taking, the university required all courses to be taught online in the first two weeks. All classes in this study were taught in-person for the rest of the semester. This university's processes during this time assume that students and instructors were "progressing as usual" (i.e., mask is required in classroom while social distance is not required). Additionally, all the data in this study was collected from a predominantly white institution which limits the generalizability of the findings and our ability to explore intersectional student experiences. Finally, participating students were enrolled in courses from 12 sections, and we were only able to interview 3 of these instructors when the manuscript was written.

A. Student survey findings
The pre-survey primarily focused on students' general group work experiences and attitudes. 81.8% of students reported that they have taken a course (at the high school or college level) that required group work. A question students answered in the pre-survey was regarding if they prefer to study individually or in groups, and it was revealed that 58.0% of students said they sometimes prefer to study alone and sometimes in a group; 27.3% of students prefer to study alone; and 14.8% of students prefer to work in a group. When students were asked about their preference about how groups are formed, 24.4% of students preferred groups that are assigned by instructors; 32.4% of students preferred that instructors assign groups with input from students; and 42.1% of students desired to choose their groups on their own.
An open-ended question in the pre-survey asked students what support(s) they believe they may need to engage in group work effectively. Six major themes emerged which were: 33.5% of students wanted to be able to ask questions and interact with instructor(s) during group work; 27.3% of students hoped to see equal contributions from all group members; 9.1% of students were looking for clear expectations of group work; 6.3% of students hoped they could have assistance from instructor on how to begin their content questions; 5.7% of students expected to have some examples with explanations; and 16.5% of students in the pre-survey said they do not need any support and/or are unsure of the support they may need, or if they do not need any support at all.
The post-survey primarily focused on students' experiences in the physics courses they were taking during the semester. 99.3% of students agreed that the course contained group work activities on a regular basis, and 84.4% of students felt that the class time was evenly spent on lecture and group work. When students were asked whether they wanted more group work, 63.8% of students stated they wanted to keep the group work and lecture time distribution the same, 23.4% of students felt they wanted more group work time, and 12.8% of students said they want more lecture time. In the post-survey, we also probed multiple facets of student group work experiences. 78.0% of students felt that the instructor(s) and/or TA(s) did not provide adequate support; 80.9% of students felt that they could not ask questions to the instructor(s) (which was a requested support from students on the pre-survey); 77.3% of students reported they did not feel included in group work; and 78.7% of students frequently felt that they were isolated from their groups. 80.1% of students reported that the group work experiences were not a positive experience. We asked students in the pre-and post-surveys if students felt that group work contributed to learning and whether group work was a waste of time as shown in Table II. We combined the responses of "Strongly Disagree" and "Disagree" as the attitude of disagree and the responses of "Strongly Agree" and "Agree" as the attitude of agree. Then, we ran chi-square tests to statistically check if the distribution of attitudes significantly changed over the semester. The standardized residual of each attitude category was calculated to obtain a cell-by-cell comparison [11]. The results revealed that students' attitude distributions about the contribution of group work to learning question in the preand post-surveys are significantly different (χ 2 (2) = 2417.2, p < .001). This shows that at the end of the semester students' beliefs were significantly more negatively believing that group work cannot contribute to learning. The results also showed that students' attitude distributions of group work being a waste-of-time in the pre-and post-surveys are significantly different (χ 2 (2) = 1154.4, p < .001). This shows that students' beliefs changed throughout the semester about if students think that group work is a waste of time.
We identified 25 students with matched pre-and postsurvey responses, and we analyzed their attitudes, as shown in Table III. Since each question on the pre-and post-surveys have five Likert-styled options, there are a total of 25 different ways for a student to answer each question. Nine out of 25 pre-to post-survey response combinations are considered as no attitude change (i.e., when the participants' response did not change, and shifts from strongly agree/disagree to somewhat agree/disagree and vice versa), 8 out of 25 response combinations are considered as attitude change is toward agreement, and another 8 out of 25 questions are considered as attitude change is towards disagreement. We statistically compared the student attitude changes in these two questions to purely random chance. The results from the chi-square test showed the student attitude  Table III includes the chance of each type of attitude change if it is purely random and the measured attitude changes of the two questions in the pre-and post-surveys change in this statement, "I feel that working with my group members contributed to my learning" is not random (χ 2 (1) = 13.69, p < .001). As for the statement "I feel that group work is a waste of time," the results from a chi-square test showed that student attitude change is not random either (χ 2 (1) = 13.69, p < .001).
In summary, most students did not have a positive evaluation on the efficiency of group work in the physics class they were taking. About one third of the students in this study identified that being able to ask instructor(s) questions during group work is the support they need. Additionally, another third of the students in the pre-survey were concerned about equal group member contributions; while in the post-survey, it was revealed that students did not feel that the physics course they were enrolled in provided this support.

B. Instructor interview findings
Three instructors were interviewed in our study. Instructor 1 is an adjunct instructor at a regional campus as well as a high school teacher. This instructor has 9 years of experience with postsecondary physics instruction and has not taught any studio style physics courses. Instructor 2 is a teaching assistant professor at the main campus with 7 years of postsecondary physics teaching and 5 years of studio physics experience. This instructor engaged in a professional development workshop at the start of the data collection semester hosted by the co-authors about effective and inclusive group work. Finally, Instructor 3 is a teaching associate professor with 10 years of postsecondary physics and 6 years of studio physics experience and has engaged in professional development via the American Association of Physics Teachers (AAPT) and the Howard Hughes Medical Institute (HHMI).
Instructor 1 mainly used group work in the lab component of traditional physics courses. Instructor 1's perspective on effective and inclusive group work primarily focused on if the lab task component of the course was completed. When Instructor 1 was asked "What are you doing during class (regarding group work)?" Instructor 1 said "[…] I walk around, and I check to make sure that everybody has roughly the same data, you know, they're getting what I expect for the data." Later, when this instructor was asked to describe a good and a bad group work experience that has occurred with the instructor's students, this individual spoke about a recent pendulum lab in which students were getting the right data as an example of good group work, and a standing wave lab in which students became frustrated since they were not getting the right data, and thus the instructor consumed these emotions and also became frustrated.
When Instructor 1's was asked about if they considered students' identities inclusivity they said: "No, not really, the students tend to go to -gravitate to the groups that they feel most comfortable with…. I don't know if this is the right way of saying. I don't think race and gender have much to do with, you know, how to do the labs." Interestingly, shortly after this instructor gave an example of how identity affected group work, the instructor said, "Skills tend to be helpful if they can kind of steer groups that are, you know, have the same language abilities, or say, similar language skills together that, you know, that seems to help each other out." Later in the interview, Instructor 1 also brought up a case regarding an incident where a girl did not work well with two boys in their group discussion and this girl preferred to work with the other group of girls.
Instructor 2 had rich experiences in teaching studio physics courses. Their main concern for group work effectiveness was the non-uniform engagement across students and group work tasks. Instructor 2 described the group work experience in the current semester as: "Half of the students seem engaged…the other half, I think they are very low in energy…. regardless of what we do, I still find it very hard to, you know get them motivated." As for the group work tasks, Instructor 2 felt tutorial problems were: "boring, more mathematical than it should be".
When asked about if/how they incorporated inclusive practices, Instructor 2 said: "I would like to have my students feel like, it is an inclusive environment. Everyone regardless of your race, gender, religious, or nationality, you know, they all are treated equally." This instructor connected demographics with teaching practice: "That's why I am going around in tables, talking to every one of them as much as possible. I tried to make them feel like, you know, I care about every one of you equally." However, this instructor stated that "I have never, you know, assessed whether it is inclusive or not. I have never done that. Maybe I should do some kind of assessment that kind of, you know, gets opinion from the students, whether they found, like 'inclusive environment or not'." Instructor 2 cares about colleagues (including faculty and TAs) and students and can sense group work does not work for some students on some day and tried really hard to help students in the classroom.
Instructor 3 had the longest teaching record of postsecondary physics and studio physics teaching among all three instructors we have interviewed, which was ten years in length of time. This instructor expressed concerns for the loose structure of group discussion sessions and felt it reduced the efficacy of the group work saying: "I do think that the tutorials in an unstructured fashion…they suffer a little bit from, you know, disengagement… I can't quite pinpoint one activity that is spectacularly bad. But I do feel many of the tutorials [group work problems], the collaborative aspect of the tutorials is not super-efficient." Instructor 3 circled back to this topic several times later in the interview. For example, when discussing how to balance lecture and group discussion components, Instructor 3 said: "The other thing I noticed too is because there's no real structure to the tutorials and how group activities unfold, … I feel like the students when they're self-managing for that long stretch of time, it becomes ineffective after a while… As a recognition of that, I, my approach is to try to run shorter tutorials." This instructor considered tying tutorials to some low-stakes assessments but did not implement it due to being overwhelmed with developing studio physics courses from scratch and respecting academic freedom of colleagues.
As for Instructor 3's view of inclusivity, the idea of "community building" was mentioned several times and it was often connected to student career path: "I try to encourage them to work collaboratively. I'd like to, mention to them, even the community building aspect. I try to mention to them it is an important skill that is transferable to your career." Instructor 3's recognition of the current problems of group work aligns with that of Instructor 2. They both feel that grading group work might improve student group work engagement.

IV. CONCLUSIONS AND NEXT STEPS
This study analyzed the student responses in the pre-and post-surveys regarding their attitudes and experiences of group work and instructor interviews about the experiences facilitating student group work. At the beginning of the semester, many students wanted to be able to ask instructor(s) questions during the group work session and requested a mechanism to guarantee equal group participation among all group members. All three instructors in our study reported walking around the classroom and interacting with students. However, very few student participants in the post-survey agreed that they feel like the instructor(s) and TA(s) provided adequate support for students to engage in group work effectively and inclusively. This shows a possible mismatch between the supports instructors believe that they are providing with the beliefs of students. Instructors should periodically check in with students to ensure their needs are being met. Additionally, instructors should engage in professional development about literature-based best practices for supporting effective and inclusive group work.
The study data suggest that not all group work was perceived as useful by participating students, and instructors have different ideas about how they can support group work; which suggests researchers and instructors should pay more careful attention to how group work is structured and supported. Both students and instructors were unsatisfied with the current group work instructional practices, regardless of course content, format, and teaching location. Additional effort must be incorporated into these physics courses to improve the efficiency and inclusivity of group work in studio physics courses.

ACKNOWLEDGMENTS
This work is supported by a Scholarship of Teaching and Learning grant from University of Connecticut's College of Liberal Arts and Sciences