Simulated Groupwork in an Asynchronous Course Learning about Radioactivity

Despite its various benefits, one disadvantage that learners in asynchronous physics courses face is the lack of moment-by-moment responsive teaching that generally facilitates learning. Previous research has documented the use of simulated Peer Instruction in an asynchronous environment by presenting students on an individual basis with alternative responses authored by the instructor or collected from previous students. Inspired by this work, we created and implemented an asynchronous learning module utilizing simulated groupwork. In this module, individual students listen to recorded discussions between artificial group members who frequently pause to ask the student for input. The student then types a response before the module proceeds to the next prompt. Through a survey administered pre and post-instruction, we compare student learning with learning gains from in-person instruction on the same topic. Furthermore, using alluvial diagrams, we identify trends in pretest/post-test improvement and responses to particular module prompts to gain insight into how to improve the curricular materials. Although this is only a pilot study ( N = 21 from a class of 29 students in College Physics 2) with specific learning targets (1. that radioactive material is ubiquitous and 2. that irradiation generally does not cause something to become more radioactive than it already is), our findings support our hypothesis that simulated groupwork can be an effective means to promote student learning in asynchronous physics courses in general.


I. INTRODUCTION
One of the benefits of interactive engagement teaching methods, like Peer Instruction (PI) [1,2], is that they provide instructors with information on what their students are thinking, so that the instructor can teach responsively.Most interactive engagement practices operate under the premise that students will be gathered together during class to share their ideas with each other and with the instructor.Although this is usually a good assumption, it is not the case in asynchronous courses, where students access materials, including recorded lectures, at whatever timing is conducive to their schedules.Asynchronous learning is empowering to diverse populations of students who would not otherwise be able to take university courses; however, it is more difficult to have learning be responsive to student ideas when it is asynchronous (e.g., [3]).In recognition of this, Englund et al. recently created simulated PI [4].In simulated PI, after an individual student provides a personal answer to a ConcepTest, answers of simulated classmates arguing for the alternative selections are presented to the student.The student then has the opportunity to change his or her mind before the correct answer is revealed.This is at least semiresponsive in the sense that the alternative reasoning is presented after the student has submitted an initial answer, and it presents the student with an opportunity to revisit the initial answer (see also [5]).
Our study is similar to that of Englund et al., except that our investigation concerns simulated groupwork that could potentially substitute group-based recitations in asynchronous physics classes.We created our simulated groupwork to enhance student understanding of radioactivity as part of College Physics 2 at the University of Alaska Fairbanks in spring 2023.Our research questions are 1) how successful was the simulated groupwork?and 2) how might the simulated groupwork be improved?

I.1 Student understanding of radioactivity: "get it away from me!"
Although education researchers have documented numerous student ideas about the topic of radioactivity (see [6] for a comprehensive literature review or [7] for an abridged review, including [8][9][10][11][12][13][14][15]), the "undifferentiated view" has received the most attention.Learners who are "differentiated" correctly understand that radioactive substances are distinct from the radiation they send out.While the radiation can cause damage to a victim by removing electrons, interactions of the radiation with the nuclei of victim atoms is sufficiently rare to generally be negligible; as such, irradiation does not generally make other objects radioactive.The "undifferentiated" view, on the other hand, lumps radioactive material and radiation-and consequently, irradiation and contamination-together into one dirt-like or even germ-like substance referred to by some education researchers as "it".Research has also shown that students tend to view "it" as, generally speaking, being dangerous and man-made.

II. METHODOLOGY
The simulated groupwork we created was based upon Johnson's Inquiry into Radioactivity (IiR) curriculum [16], which demonstrated success at developing the differentiated view in non-science majors at Black Hills State University [14].Although IiR was originally designed to fill a physical sciences elective for non-science majors, MS.Ed.students taking Hull's seminar at the University of Vienna condensed the first part of IiR into a version that would occupy only three class periods [17].In these three lessons, students are told that a Geiger Mueller (GM) counter is a device that they have seen on TV nearby Fukushima measuring radiation from radioactive sources.Learners explore the world around them with the GM counter and are generally surprised to learn (ideally) that radioactivity is ubiquitous (although some things are much more radioactive than others), and the term "background radiation" is introduced.Once students establish a baseline for how much the GM counter clicks in an empty classroom (the range of values for minute-long readings), they are then ready to put a strawberry into close proximity of a strong radioactive source to see that, even after 48 hours of being close to the uranium glass, the strawberry does not register an increased number of clicks on the GM counter.In contrast to typical student predictions, students recognize (ideally) that the radiation did not make the strawberry radioactive.For use in online classrooms or in classrooms that do not have GM counters, we created videos of these experiments.

II.1 Implementation and assessment of Inquiry into Radioactivity in College Physics 2
For the asynchronous College Physics 2 class, Hull created and audio recorded a dialogue between two virtual students, Alex and Bailey, as they progressed through the three condensed IiR worksheets.This dialogue was based upon personal experience of listening to typical in-person student responses to the prompts on the worksheets and knowledge of common student difficulties documented in research literature.Hull played this audio file while screen recording the various videos and scrolling through the IiR worksheets.For each prompt in the worksheets, Alex and Bailey ask the student for his or her own opinion.Using PlayPosit, we had the module stop at each of these questions and only continue once students submitted a response.This activity took the place of weekly online lectures and homework and students were awarded points for participation.Almost all students in the class completed the module (28 students out of 29).
Improvement in student understanding was assessed using a survey developed by Holzinger as part of her MS thesis [18].The survey includes three items to assess student understanding about the differentiated view (DIFF1, DIFF2, and DIFF3) and three items to assess student understanding about the ubiquitousness of radioactivity (UBIQ1, UBIQ2, and UBIQ3).DIFF1 and DIFF2 (from the work of Johnson) concern the irradiation of a strawberry [15].UBIQ1 (validated in Hull's earlier work) asks students if they would ever open a closet containing valuables if the closet became filled with radioactive gas, with a "correct" response being anything other than "I would never open the closet" [19].The remaining three items were new: DIFF3 asks if a radioactive stone can make passersby become radioactive; UBIQ2 is a checklist task to select all items that will make a GM counter click, with the correct answer being to select all ten items; UBIQ3 asks about the danger of radioactivity, with the correct answer being "can be dangerous and can be safe", as opposed to "is dangerous".Holzinger validated the new prompts, both via an expert panel and via survey validation interviews.She then invited adults who had graduated from high school in Vienna but who had not entered into a radioactivity-related profession to complete the survey, and 386 did so in 2021.She found that, regardless of high school attended and with newer graduates performing only marginally better, the vast majority of the public did not answer either the DIFF or the UBIQ sets of questions correctly.For example, about 65% of the public thought the strawberry would become radioactive.On UBIQ2, about 85% of respondents did not select "a school child" as something that would make a GM counter click.
Hull administered this survey to students in College Physics 2 in 2023 before and after the asynchronous IiR module for extra credit for completion.Of the 28 students who completed the module, 21 students also answered the survey pre/post.Names were used to match survey and module responses, and then names were replaced with pseudonyms.All methods were approved by University of Alasks Fairbanks IRB.We addressed our first research question: RQ1) how successful was the simulated groupwork?by looking at student improvement from pretest to post-test and comparing this growth with that documented by Jeidler with various instructional methods on radioactivity in Vienna [20,21].

II.2 Investigating room for improvement
To address our second research question, RQ2) how might the simulated groupwork be improved?, our focal point was the recognition that, as discussed in the introduction, asynchronous instruction generally is lacking in responsiveness to student ideas.It is possible to alleviate this somewhat by embedding branching points into PlayPosit modules.In the simplest version of such a branching point, students answer a multiple choice question.If they answer incorrectly, they are brought to additional instruction that is skipped by students who answered the prompt correctly.This is somewhat analogous to instructors checking in with students during groupwork and giving additional guidance if needed.Such a systematic method is less than ideal, as students are quick to realize if instructors are only checking in when they are incorrect; nevertheless, most instructors are most likely to intervene at such times, and so we approach our module improvement with this method as a first step.
With the desire to find the most efficient locations to introduce branching points, Hull developed and carried out a five-step process: 1) comb through student responses to the module prompts one by one and flag each response where an instructor should intervene, were the lesson to take place in a synchronous classroom; 2) identify the ten prompts (or, in case of a tie, the smallest number of prompts more than ten) in the module that received the most flags; 3) for each of these most-problematic prompts, identify the learning goal (for example, reaching the differentiated view) that the prompt most closely corresponds to; 4) identify all (e.g.) DIFF items on the survey and choose a criterion by which growth will be determined (here, due to small sample size, we coded a student as "differentiated" if all three DIFF items were answered correctly); 5) create alluvial diagrams (we used R Studio) to demonstrate student progression from the pretest (coded, e.g., as "differentiated" or "undifferentiated") to the module prompt (coded for whether the instructor should "intervene" or "pass" by the group discussion) to the post-test (coded the same way as the pretest).We will elaborate upon alluvial diagrams and this last step.
Alluvial diagrams simplify and highlight structural changes between clusters so that transition rates can readily be seen.Alluvial diagrams are a type of Sankey diagram, which are used in a variety of research fields, including physics education research [22][23][24]).In our study, there are eight possible alluvia for each [pretest -module promptpost-test] combination.Our names for alluvia discussed in the following paragraphs are labeled in Figures 1 and 2 in the Results section.
The substeps in step 5 for module prompts pertaining to the differentiated view are as follows: 5i) check if the "ceiling" alluvium [differentiated -pass -differentiated] dominates over all eight alluvia, indicating that the prompt is perhaps redundant and a candidate for removal to streamline the module; 5ii) from the four alluvia beginning with a "differentiated" pretest code, check the size of the "misled" alluvium [differentiated -pass -undifferentiated].If this alluvium is larger than the "ceiling" alluvium, then the prompt may be misleading students; 5iii) compare the four alluvia beginning with an "undifferentiated" pretest code.If the learning goal is particularly ambitious, the "baffled" alluvium [undifferentiated -intervene -undifferentiated] will dominate these four.If, on the other hand, great gains were seen in the survey pretest/post-test, then, ideally, the dominating alluvium will be the "victory" alluvium [undifferentiated -pass -differentiated].If this alluvium is the largest, or if it is the second largest with the "baffled" alluvium in the lead, then the prompt is most likely helpful with learning.If "victory" is the largest but "baffled" is also large, then the module prompt is one that can be potentially be improved with a branching point.If, instead, one of the two "other" alluvia is the largest of these four, or if "baffled" is the largest followed by an "other" alluvium, then the prompt is most likely not directly tied to the supposedly associated survey items and it should perhaps be replaced.

III. RESULTS
In the post-test survey, we asked students for their impressions of the learning module.The majority of students reported either neutral or positive feelings regarding the module overall (19/21) and about the simulated groupwork being better than pre-recorded lectures (15/21).Feedback included "…Breaking the material down piece by piece in the form of a conversation made it much easier to grasp".
Our first research question was RQ1) how successful was the simulated groupwork?In answering this question, we did not assume that the three DIFF items are equally difficult, and so we did not calculate an average score across the items (likewise for the UBIQ items).In future large-N studies, we can perform Rasch analysis to determine the relative difficulty of each item for a more nuanced analysis.For our pilot study, we applied the code, e.g., "differentiated", if all three DIFF items were answered correctly; otherwise, we applied the code "undifferentiated".
First, regarding the differentiated view, we saw improvement comparable to that of IiR when used as a fullsemester in-person class [14].Two of the 21 students left one or more of the DIFF items blank.From the remaining students, we saw a growth of 3/19 to 16/19 differentiated students (growth is statistically significant, according to a McNemar's test).This stands in contrast to the findings of Jeidler, who, looking at 200 high school students in Vienna learning with various instructional approaches that did not use the strawberry experiment, found that fewer than 10% of students were differentiated post-instruction [21].Regarding understanding that radioactivity is ubiquitous, we saw much more modest (not statistically significant) growth, from 4/20 to 7/20 students answering all three items correctly (one student left one of the three items blank).The hardest of the three items by far was UBIQ2.Whereas our growth was 4/20 to 11/20 correct respondents, the MS thesis of Jeidler found that students answering this item correctly also remained below 10% post-instruction.As this is only a pilot study and our sample size is small, we can only tentatively answer our first research question with the following: regarding the differentiated view, our lesson utilizing simulated groupwork was comparably effective to the original IiR which features genuine groupwork.Regarding the idea of radioactivity being ubiquitous and not necessarily artificial and dangerous, our lesson was comparably effective to other in-person instructional approaches covering radioactivity.However, in light of the emphasis the module places on using the GM counter to measure the radiation coming from everyday objects, we are optimistic that greater learning gains can be had from improved versions of the learning module.This motivates our second research question, RQ2) how might the simulated groupwork be improved?
To answer this second research question, the Hull examined the responses of the 21 students on the module, prompt by prompt.In total, there are 45 prompts in the module.Hull flagged one prompt, FUKUSHIMA, as being the most problematic, as 10 of the 21 students responded in a way that warranted instructor intervention (all other prompts had fewer of these flags).FUKUSHIMA asks students, after they have acknowledged that the strawberry is not more radioactive despite being in close proximity to uranium glass for 2 days, "What might be the cause for the increased radioactivity measurements near Fukushima?"A response that was coded as "pass by without intervening" was "… perhaps the radioactive material did not 'make' the ground radioactive, but just coated the ground in radioactive material…" On the other hand, a response coded as "intervention needed" was "the radioactive material was much stronger at Fukushima."We considered that this might be a good location for a branching point; namely, to replace the currently free-response FUKUSHIMA prompt with a multiple-choice version.Students who select that the increased GM counter readings were due to dust coating the ground near Fukushima would pass on with the lesson.Students who selected that the cause is higher radiation levels would listen to additional dialogue between Alex and Bailey, where the idea of breaking the uranium glass and embedding parts of it into the strawberry would be mentioned.Since this would require additional time for the learner, we first examined the alluvial diagram (Figure 1) for [DIFFERENTIATED.PRE -FUKUSHIMA -DIFFERENTIATED.POST] to see if performance on this prompt correlated with growth on the DIFF items.Here, each of the 19 respondents fell into one of five alluvia.The "victory" alluvium dominates, suggesting that this prompt might be important for student learning.Specifically, this alluvium shows students who, on the pretest, demonstrated the undifferentiated view by answering at least one of the three DIFF items incorrectly.The students "victoriously" answered FUKUSHIMA correctly and (at least plausibly) left the module with an improved understanding, as demonstrated on the post-test items.However, there is also a noticeable (three students out of 19) "baffled" alluvium, indicating that this, indeed, would be an appropriate prompt to build in a branching point.Had these three students received additional support on FUKUSHIMA, it is plausible to think that they too would have joined the "victory" alluvium.
Seven of the 12 most-problematic prompts were, like FUKUSHIMA, relevant to the differentiated view.Three of these seven prompts had alluvial diagrams similar to Figure 1 and were thus found to be candidates for branching points.The other four module prompts related to the differentiated view had no students in the "baffled" alluvium and so were judged to be sufficiently effective as is.Regarding seeing radioactivity as ubiquitous, the majority of the relevant module prompts were relatively issue-free for students.Only one of the 12 most-problematic prompts, INFRARED, was potentially related to this learning goal.INFRARED asks students to watch a video of a person rub his hands together and hold them over the GM counter and in front of an IR camera.Although this prompt is originally intended to help strengthen the differentiated view by showing that there are types of radiation that are unrelated to radioactivity, it should, ideally, also show that people, like other everyday objects measured, make the GM counter click.An example of a response coded as sufficient to allow the teacher to "pass" by without intervening was: "The heat detector measured the heat in the person's body, but there were not more clicks with increased heat."On the other hand, a response coded as requiring a teacher to "intervene" was "It seemed like the less heat around the detector, the less clicks that could be heard."This latter statement could reflect the idea that clicks decrease when the person comes near since people are not radioactive.To see whether this prompt affected student learning, we created an alluvial diagram (Figure 2).
Here, the "victory" alluvium does not dominate, nor does the "baffled" alluvium.Rather, of the four alluvia beginning with an incorrect response on the pretest, an "other" ("other3") alluvium is the widest.This suggests that the prompt is not closely connected to the learning goal.Since none of the other 12 most-problematic module prompts were related to the ubiquitousness of radioactivity, it was decided that other, more explicit prompts should be introduced to the learning module.For example, when students are watching videos of everyday objects being measured by the GM counter, a video of a person being measured (without the infrared camera) should also be added.The remaining 4 most-problematic prompts were related to the idea that it is random when an individual nucleus decays, the focus of our prior work [7,[25][26][27][28].

V. DISCUSSION AND CONCLUSION
To the best of our knowledge, we are the first to attempt to make instruction for asynchronous learners more responsive by simulating groupwork.Although this was only a pilot study, we found that most students successfully came to understand that radioactive objects do not, generally, make other objects radioactive in the way that germs or dirt spreads.Furthermore, we found improvement in students realizing that everything is already a little bit radioactive.At the same time, we identified room for improvement of the module by looking at patterns of student responses on survey items and corresponding module prompts.
We find the results of our pilot study encouraging, as we obtained conceptual gains comparable to in-person instruction on radioactivity [14,21].Our work was specific to student learning about radioactivity, and future work should investigate the creation and assessment of simulated groupwork for other physics topics.The procedure discussed in the methodology of visually inspecting alluvial diagrams is a generic one that can potentially be used as a starting point for improving curriculum in any topic.With a greater number of respondents, qualitative content analysis can be used to categorize student responses beyond the binary "instructor intervention needed" or "instructor can pass by without intervening".Similarly, finer-grained coding can be applied to the survey items.This analysis is important for optimizing asynchronous curricular content to be responsive to learners without being unnecessarily burdensome to them.

FIG. 1 .
FIG. 1. Alluvial diagram for [DIFFENTIATED.PRE -FUKUSHIMA -DIFFERENTIATED.POST].The top boxes are desirable responses: (from left to right) "correct" on all three DIFF items on the pretest; "pass" on the FUKUSHIMA prompt; "correct" on all three DIFF items on the post-test.