Acquisition of qualitative video data: methods and reﬂections in PER

Video Data is commonly collected in PER to allow for insights into how learning and teaching unfold over time. One might think the collection of video data is straightforward, but there are key decisions about gathering video that can profoundly impact the entire project. Here we take a microscope at the common practices of gathering video data in PER. Through two existing cases in the qualitative PER literature, we describe how and why PER scholars made those key decisions. We open the black box of research planning and decision-making when video data is being collected. By increasing transparency, we aim for the community to better understand the decisions made behind the scenes in the research process, which may strengthen other PER scholars’ future research endeavors.


I. INTRODUCTION
Video data is widely used in qualitative research in PER [e.g., [1][2][3].Video is important for allowing insights into how learning and teaching unfold over time, which has provided the field with new knowledge about student behavior and reasoning not accessible through audio or student artifacts.That is, through video analysis PER has gained important insights into social dynamics role in the learning and teaching of Physics, through analyses of gestures, body position, and facial expressions (e.g., gesture analysis, posture, social dynamics, positionality, etc.) [e.g., 4,5].One might think the collection of video data is straightforward.But, in a few weeks or months, one can amass a huge corpus of video, which can be overwhelming given the practical need of making decisions during the data analysis process.When large amounts of video are collected without careful consideration of the research goals, it can lead to ambiguities about next steps and the subsequent analysis.Adjacent fields, such as the Learning Science and Science Education have written about video analysis [6][7][8][9], but there are fewer new video methods pieces and there are differences across learning environments and emphases that can have ramifications in the gathering of video data.
Here we argue that there are key decisions to be made before and during data collection that results in the collection of the optimal amount for a given research goal.Those decisions can have a profound impact on nature of the video data collected and the entire project.Through two cases of PER articles engaged in video analysis, this paper aims to describe how and why PER scholars made those key decisions so that the larger PER community can better understand the decisions made behind the scenes of the research process, which in turn may strengthen their future research endeavors.By synthesizing key ideas and decision points around the gathering video data, we aim to illuminate these issues for a PER audience that might be new to video data collection and/or interesting in learning from these adjacent fields.
In what follows, we use two different cases from the empirical PER literature to highlight how video data collection decisions are impacted by the research goals, including the theoretical framing and assumptions.We specifically focus on decisions related to camera placement, the amount of video equipment used, and the level of detail needed in the videos.These two cases allow for a comparison of how video is used differently across learning environment structures (entire classroom environments versus small group work), including the roles of lab classroom technology and the instructor in the gathering of video data.Beyond these differences, across both cases, we show how the video data collection process was tailored to the specifics of the research goals.Thereby, we shed light on the behind-the-scenes research process of gathering video data in PER.Finally, we conclude with a series of questions and decisions points that are meant to support PER Scholars in planning their data collection in light of their research goals.
Our first case of how and why PER scholars made key decisions about video data collection comes from a qualitative analysis of a classroom of 9th-grade students participating in an embodied modeling activity known as Energy Theater [10].Energy theater was developed to engage "learners with key conceptual issues in the learning of energy, including disambiguating matter flow and energy flow and theorizing mechanisms for energy transformation."[11].In this instance, the students were modeling the steady-state energy of the Earth in an Earth science class.The teacher introduced the activity by creating space in the classroom for the activity and describing the roles: Each student represents one unit of energy.Ropes are used to delineate objects in the system (e.g., sun, earth).Movement from one point to the next indicated movement of energy between objects in the system.During the activity, students engaged in rich conversations about the key objects in the system, while negotiating details about the wavelengths of light and movement of energy in the system.Sometimes many individuals were talking at once, and sometimes everyone was silent while one person shared an idea aloud.A body of prior research has documented the advantages of this activity for supporting conceptual engagement about conservation, storage, transfer, and flow of energy along with the disambiguation of matter and energy [11][12][13][14][15].
During the class, the 9th-grade students were introduced to Energy Theater, its rules, and assumptions, and then told to model the steady-state energy of the earth.The authors collected data from three 45-minute class periods, each of which had two science classrooms that were team-taught by two teachers.One class period was not studied because the class sizes were so small that nearly no discussion happened, and discussion is key for video data that can be analyzed with this theoretical perspective.For each class period, the students first enacted Energy Theater in their classrooms.Then the two classes came together and watched their peers' Energy Theater enactment.Finally, there was a joint enactment of Energy Theater with all students from both classes.
The authors were interested in the conceptual learning that may have occurred during this activity.To capture this learning, the authors used a theoretical framework known as Coordination Class theory [16,17].Knowing that this theory had previously been used to capture learning in interview settings, the authors were interested in the affordances and limits of this theory in capturing classroom learning within Energy Theater.As some background, Coordination Class theory sits within the larger Knowledge in Pieces and Resource theory perspective [18,19].This theory is a model of a concept within a learner's larger knowledge system.Specifically, using this theory, one is typically aiming to capture how learners' understanding of a concept (e.g., force; accelerationor in this instance, steady state) changes over time through the application of the concept to new contexts, for instance, across different problems or learning environments.Within the theory, a potential learning difficulty is that of determining the same information from different contexts, given that likely different knowledge is being used in those contexts (e.g., homework problems vs. laboratory experiments-or in this instance, different models created in Energy Theater).

A. Video Data Collection Considerations
Given the theoretical framework and the specifics of Energy Theater, the video data collection plan was tailored to the project.The authors aimed to capture video data from as many students as possible and their teachers as they moved around the classroom, acting out the energy transfer and transformations across the objects in the scenario.It was important to collect information on the student's embodied actions with respect to the objects in the scenario, for instance, their position concerning the ropes representing the sun and earth.Needed was also information about their gestures or other body details that were used to convey information about their type or form of energy.For instance, students waved their arms back and forth, tag peers, and hold up their hands to signal different forms of energy.This type of information about embodied actions is foundational to the Energy Theater given its importance in the student's models of Energy flow, transfer, and transformations, and therefore key to our research goals and subsequent analysis.To capture this, the authors placed cameras on tripods at the room edges in each classroom.Another goal was to collect high-quality audio data of the students' discussions, as discussions are key for a Coordination class theory analysis.To collect this data, external audio recorders were placed around the necks of the teachers, given the likely importance of capturing good audio from them.We also hung a few audio recorders from the ceiling to further increase the likelihood of strong audio data from different locations in the classroom.Finally, we had unique concerns related to the logistics of the space (simultaneously collecting data in two team taught classrooms) and the scheduling of the classes (spare time between classes).Therefore, we placed equipment in both classrooms to avoid moving equipment during a hectic class.Given the classes' sequential nature and the school's short passing time, we had to check batteries and memory cards during the very brief classroom transitions and change the batteries and memory cards as needed to ensure no missing data.

B. Results and Limitations Given the Video Data Collection
The results found that the students changed their models in specific ways that better aligned their understanding of the scientific concept with their newly modified model.For example, in one class they recognized that in their model, energy was leaving the earth and going to the sun, which was nonsensical and not what they intended to represent.They subsequently changed their model so that when the people (who represented energy) left the earth, they exited the model, and then reentered the model in the sun.In this case, it was a single student who pointed out the issue during a group conversation when everyone was listening.We were able to show how a single student's contribution can dramatically affect the model and subsequent learning.Our ability to capture these findings was due to having the video of the entire class period, all the students' movements around the classroom, and clear audio of the whole class discussions.We likely would have missed these crucial details about how and why they changed their models if we had collected only written notes or only audio data.Furthermore, we might have missed key details if we had used fewer cameras.
However, there were limitations of the data we collected and that had ramifications.We could not analyze each student's individual learning over the entire class period.We also couldn't follow every idea that surfaced during the class discussion, and some ideas were lost to the chaotic parallel discussions.We saw some embodied actions that we could not connect to verbal explanations.These limitations meant that the analysis focused on learning at the classroom level, not the individual level.But, importantly, learning at the classroom level through the coordination class theory lens was the crux of the contribution given that previously coordination class theory had been used to capture individual learning, often in interview settings.

III. CASE 2: GATHERING VIDEO DATA OF SMALL GROUP WORK IN AN INTRODUCTORY LAB COURSE
Our second case of how and why PER scholars made key decisions about video data collection comes from a qualitative analysis of four undergraduate students enacting group sensemaking in an Introductory Physics for Life Science (IPLS) major's lab course [20].The IPLS lab course in question had been redesigned to emphasize a Three-Dimensional learning (3DL) approach [21] and included structures so that student groups had the experimental agency to develop and carry out experiments that explore the physical properties of biological phenomenon.For the article, we focused on a lab where the students were tasked to create an experiment to study Brownian motion to provide insight into how diffusion occurs.The students were provided with materials, such as synthetic microspheres suspended in fluid and microscopes.They also used an image-tracking software on a classroom computers that was stationed at their labs.This article sits within a body of research that is broadly focused on students' reasoning processes in various intro Physics labs that have been reformed [e.g., [22][23][24], but less work has focused on the moment-by-moment reasoning processes in these labs.For this article, the data collection happened through in-person class observations of students' small group work in the labs.The course mainly enrolled upper-division undergraduate students from life science disciplines (e.g., biology, kinesiology), and for most students, this was their first physics lab course.During the semester, students worked in small groups of 3 or 4 individuals on multi-week investigations where they would develop research questions, experimental design plans, and hypotheses, and then investigate, develop a scientific argument, and present their argument to peers verbally and in written form.Importantly, the article's first author served as TA for several of the observed sections, all of which used the lab classroom, and the fourth author served as the instructor of record for the course.
In this research, we were interested in students' sensemaking in these labs.Specifically, the moment-by-moment details of their sensemaking process.In the analysis, we focused on a series of inconsistencies in their sensemaking, specifically what the inconsistencies are about and what moves students enacted to resolve them.We view sensemaking as a "dynamic process of building or revising an explanation to 'figure something out'-to ascertain the mechanism underlying a phenomenon in order to resolve a gap or inconsistency in one's understanding" [25].At a high level, we focus on two distinct elements of the sensemaking process: the process of "figuring something out" through explanation construction, and the nature and recognition of the inconsistency being resolved in the sensemaking process.Within this theoretical framing, we had the following research question: What forms of inconsistencies are students in introductory physics lab courses sensemaking about, and what moves do students enact during this sensemaking to achieve resolution?

A. Video Data Collection Considerations
Given our theoretical framework and the IPLS course setting, our video data collection plan was tailored to the project.We aimed to capture the details of the sensemaking process while the students were collecting their data and analyzing their data, including their body positions, facial expressions, and gestures.We wanted to capture their discussions throughout the lab, knowing that the lab is a noisy, and sometimes chaotic, environment.Given these goals, we placed several external cameras on high shelves above the lab benches.The cameras had wide-angle lenses that were able to capture the entire workstation, including students' heads and body positions around the computers and microscopes.The cameras had external Bluetooth mics that were placed at the center of student workstations near computers, as this was where students congregated to capture their conversation.The backup audio recorders were next to the Bluetooth mics, positioned for optimal collection of small group discussions.
We also wanted to collect video data from multiple student groups and multiple labs to capture some variety in their sensemaking.Knowing this, we wound up collecting data from 13 groups (N= 38 students), each group enacted 4 labs, and each lab lasted for two or three weeks, typically 3-5 hours of video per week for each group.It varied due to the timing of other lab activities, such as warm-up activities and time allocated for writing lab reports.We also need a record of their computer screens to track the details of their data analysis in a spreadsheet.Specifically, we wanted to track how they arranged and manipulated numerical information on their spreadsheets, we wanted to see the different graphs created and how they modified them over the lab class.Thus, we collected this data using screen capture software that ran in the background of the lab computers.We also wanted to triangulate across video and written data with written work.Therefore, we collected their final lab reports, which were written in a scientific argumentation format.Finally, we didn't want to miss any data due to equipment or human error, therefore we were checking and charging batteries regularly and carefully moving the video files off the cameras and lab computers regularly.

B. Results and Limitations Given the Video Data Collection
The results found that students engaged in sensemaking to resolve conceptual and procedural inconsistencies.We identified instances when students recognized conceptual inconsistencies (identified in video by visual cues such as throwing one's hands up in frustration, facial expressions of discontentment/confusion, etc.) and engaged in sensemaking by juxtaposing their hypotheses and evidence and then constructing new scientific explanations (identified in audio by quick backand-forth discussion between students, coinciding with selfquestioning, interactions with TAs, and elements of mechanistic reasoning).Comparably, we identified instances when students a procedural inconsistency involved and engaged in sensemaking by proposing and testing a series of causes toward modifying experimental procedures or apparatus.We argued that both types of inconsistencies are generally productive, evident through video by students' demeanor becoming more positive and affirming, as well as through audio data which picked up their consensus building and descriptions of troubleshooting success and new experimental ideas.Our ability to capture these findings was due to having the video of their entire sensemaking process during the lab, along with the screen capture video of exactly how they analyzed their data in a spreadsheet.Our video was detailed enough that we could capture nuances in their facial expressions, body positions, and gestures that allowed additional evidence for their inconsistencies.If we had only collected written observational notes or only audio data, or more limited video data, these results would not have been discernable.
However, there were challenges and limitations of the data we collected and that had ramifications.We could not analyze certain instances in the data when students interacted with peers in another group or moved out of range of the audio recorders.Finally, we struggled to connect the two streams of video data, from the external cameras and screen capture.There were times we saw students on the external videos referring to something important on the computer screen but couldn't sufficiently discern what it was via the screen capture software.We missed some data when students took notes or enacted data analysis on their personal computers rather than the classroom computers with the screen capture software, despite the instructor's encouragement to use the classroom computers.We also had a few instances where we missed audio data because the audio quality wasn't sufficient to parse among many simultaneous voices.These limitations impacted our results, but in mitigating them, an important factor was the researcher also being the lab TA.In this capacity, he was familiar with the students' voices, which in turn supported his subsequent video analysis across these two video data streams and the external audio.As the TA, he had strong expertise with the Physics content and was familiar with the lab nuances, all of which facilitated a detailed analysis of the video.
IV. DISCUSSION: VIDEO DATA COLLECTION DECISIONS ARE IMPACTED BY THE RESEARCH GOALS Across these two cases, we showed that key decisions made before and during data collection can have profound implications on the data collected and the larger research project.Across the two cases, there were differences in the learning environments (whole class video data gathering versus small group video data gathering) and differences in the length of time (one class period versus a semester).The second case also included significant lab equipment and the use of screen capture software, while the first case included important embodied actions.Importantly, in both cases, details of the collected video were tailored to the specific research goals, assumptions, and theoretical background, and limitations of the data had ramifications on the results.In this article, we took a microscope to the common practices of gathering video data in PER to provide a window into how and why PER scholars made those key decisions.Our goal has been for the PER community to gain a stronger understanding of the important research decisions that are so often kept behind the scenes, and thereby begin to open the black box of research planning and decision making to encourage more transparency across the community.
Moving forward, for PER scholars, we suggest careful consideration of their goals, research question, and theory and how those factors can impact camera placement and amount of video to collect.
• Comparably, if one is interested in gestures or facial expressions of a small number of people, then the camera needs to set up in way to capture those details.As another example, if one is interested in a small number of people within a larger learning environment (e.g.small group work within a lab section or lecture), then the camera position needs to account for those individuals within the class.In many of these learning environments, there are likely to be overlapping conversations in a noisy room, and this reality needs to be accounted for in the recording equipment setup.For PER scholars embarking on new research projects that will rely on video data collection, we suggest careful consideration of these questions in order to implement careful data collection plans.Answers to these questions are going to be impacted by one's goals, research questions, and theory, but systematically articulating ones answers to these questions is likely to support the optimal amount and type of data for a given project.
In what instructional activity or learning environment does the phenomenon appear?For instance, perhaps the phenomena of interest occurs in small group discussions, whole class activities, lab courses, interviews, or lecture environments.It's possible the phenomena of interest is more likely to occur in certain courses with certain goals or instructional approaches, or perhaps it occurs in a wide-range of courses.Careful consideration of the learning environment is important for justifying ones choices and not inadvertently collecting data in a setting of convenience.•How long do you need to record to capture the phenomenon?Depending on the phenomenon and specifics of the instructional activity, one might decide to collect data for a single session, multiple sessions, or even a whole semester.Importantly, one might need many instances of a phenomena, or one might need only a couple instances, depending on the specifics of the research goals.For example, if one is focused on how conceptual knowledge about a specific topic changes over time, then likely many course sessions are an appropriate length of time.• How will you set up cameras/recording equipment to capture that phenomenon?Based on answers to the prior questions, it may be important to use multiple cameras or possibly a single camera will suffice.For instance, if one is interested in interactions among many individuals in a large lecture environment then multiple cameras would be needed to capture all the individuals, possibly arranged at different sides of the room.