Impact of mathematical reasoning on students’ understanding of quantum optics

We report on an investigation in which we compare the conceptual performance of upper-level undergraduates and graduate students, who worked on two different validated Quantum Interactive Learning Tutorials (QuILTs). One of the QuILTs incorporates mathematical reasoning while focusing on helping students develop a good conceptual understanding of quantum optics using a Mach-Zehnder Interferometer with single photons and polarizers. Performance of students who engaged with this “hybrid” (integrated conceptual and quantitative) QuILT is compared with those who engaged with a conceptual QuILT focusing on the same topics without quantitative tools. We find that the posttest performance on conceptual questions of physics graduate students who engaged with the hybrid QuILT was generally better than those who engaged with the conceptual QuILT. For undergraduate students, the results were mixed.


I. INTRODUCTION, FRAMEWORK AND GOAL
In physics courses for majors and graduate students, in order to enhance student learning with quantitative tools, students must be given adequate opportunity to make the math-physics connection, interpret the mathematics involved and draw qualitative inferences from them [1]. Without this explicit focus on integration of conceptual and quantitative aspects of physics learning, quantitative problem solving can become a mere mathematical exercise instead of an opportunity to develop reasoning skills and "compile" new knowledge of physics [1]. In other words, for quantitative problem solving to be effective for developing a good knowledge structure of physics, students must learn effective problem solving strategies and focus on integrating conceptual and quantitative understanding. Moreover, while integration of quantitative tools can enhance physics learning, students are unlikely to benefit from them if they do not have facility with the requisite mathematics. In such situations, students can have cognitive overload [2] since working memory while solving problems has fixed capacity. The investigation reported here is based on the hypothesis that in the advanced courses such as quantum mechanics, integrating conceptual and quantitative understanding can help students build a more coherent knowledge structure of physics and develop their reasoning and meta-cognitive skills so long as it does not cause cognitive overload for students [2].
One of theseQuILTs focusing on the MZI with single photons and polarizers strives to help students learn about foundational issues in quantum mechanics using an integrated conceptual and quantitative approach (called the hybrid QuILT from now on for convenience). The conceptual QuILT engages students with conceptual reasoning only, under the assumption that the underlying quantum mechanics concepts involving single-photon interference and a quantum eraser are sufficiently complex, and that additionally incorporating quantitative tools involving product states of path and polarization may cause cognitive overload [2].
Developing a good knowledge structure of physics without using quantitative tools can be particularly difficult for students who are not experts in quantum optics. This is because using math can provide useful constraints to help students do appropriate sense making, so long as the use of math in this context does not cause cognitive overload. This led to the development of validation of the hybrid QuILT.
In both the hybrid and conceptual QuILT development and validation discussed elsewhere [4][5][6][7], research on student difficulties was used as a guide. The QuILTs use a scaffolded inquiry-based approach to learning. In the hybrid QuILT, students are asked to make conceptual inferences from quantitative tools. Since the learning goals of the hybrid QuILT pertaining to conceptual understanding are the same as for the conceptual QuILT, after traditional instruction in relevant concepts and after students engage with the hybrid or conceptual QuILT, they were administered the same validated pre/posttests to evaluate their conceptual understanding.
Here our goal is to investigate and compare the extent to which students who engaged with either the hybrid or the conceptual QuILT learned the underlying concepts.
We note that the interpretation of these experiments using quantitative tools in the hybrid QuILT involves making conceptual inferences using product states of path and polarization, but the underlying concepts can be taught qualitatively using only conceptual reasoning. We also note that both the conceptual and hybrid MZI QuILTs use the same visualization tools (simulations) involving the MZI with single photons and polarizers to help students learn about single photon interference and quantum spookiness, e.g., in the context of a quantum eraser. Both QuILTs focus on using several different contexts of the MZI experiment to help students learn topics such as the wave-particle duality of single photons, interference of a single photon with itself, and the probabilistic nature of quantum measurements. Students also learn how adding photodetectors and optical elements such as beam-splitters and polarizers in the paths of the MZI affect the measurement outcomes. The difference between the two QuILTs is that, in the hybrid, some of the guided learning sequences involving conceptual reasoning only are replaced by those sequences involving a combination of both conceptual and quantitative reasoning. This is done with the assumption that students in quantum mechanics courses will benefit from the opportunity to make qualitative inferences from quantitative tools and will not have a cognitive overload [2] despite increased mathematical sophistication of the hybrid QuILT.

II. METHODOLOGY
Materials: Each of the validated QuILTs about the MZI includes a pretest, to be administered immediately after traditional instruction, on the concepts involved in the MZI experiments with single photons and polarizers, but before students engage with the QuILT. An analagous posttest is administered after students finish working on the QuILT [3][4][5][6][7]. The questions on the pretest and posttest, are open-ended and were validated in prior research [3][4][5][6][7]. In particular, the pre/posttests for the hybrid QuILT include the same MZI experiments as the conceptual QuILT with only one isomorphic variation [3][4][5][6][7]. The open-ended format requires that students generate answers based upon comprehension of underlying concepts. Both QuILTs and pre/posttests are available on PhysPort. The same rubric was used to grade the pre/post responses of students who used the hybrid or the conceptual QuILT.
Students Involved and Implementation: Once the hybrid and conceptual MZI QuILTs were validated, they were administered to upper-level undergraduates (physics juniors and seniors) and first year physics graduate students at a large research university. Students first had traditional lecture-based instruction in relevant concepts before engaging with the corresponding QuILT, both in the upperlevel undergraduate quantum mechanics course and first year graduate students in a core quantum course.
The data from graduate students for the conceptual QuILT are for the first-year graduate students for two consecutive years, who were simultaneously enrolled in the graduate core quantum mechanics course and a semesterlong teaching of physics course, which is a mandatory pass/fail course. In this latter case, the pretest, conceptual QuILT, and posttest were part of the teaching of physics course to help graduate students learn about the tutorial approach to learning and teaching (the core graduate quantum instructor in those years was reluctant to administer the conceptual QuILT in the graduate quantum mechanics course due to time-constraints). Group A and Group B of undergraduates engaging with the hybrid QuILT are separate cohorts of students in consecutive years. These groups are considered separately based on remarkable differences in performance, despite being occurring at the same institution. This is possibly an instructor effect.
In all, this amounts to four consecutive years of implementation involving one of the two QuILTs for the undergraduate and three consecutive years to the graduate students (see Fig. 1). All data was collected in year preceding the COVID-19 pandemic.
All students received credit for completing the pretest and QuILT, and graduated students who engaged with the conceptual QuILT received credit for completing the posttest. All other students received quiz credit for their posttest based on correctness. This accounted for small portions of students' homework and/or quiz grades for their respective courses. All students using either of the QuILTs were then given the posttest in their respective classes. All students had sufficient time to take the pre/posttests. Two researchers graded all of the pre/posttests on the rubric, including partial credit, and the inter-rater reliability was better than 95%.
We consider that a cohort on average has a strong mastery of the concepts with scores above 85%; moderate, above 70%; fair, above 50%; poor, below 50%. Rather than hard cutoffs, these are rough guidelines used for comparison.
The traditional lecture-based instruction by the course instructors included an overview of the MZI setup. Students learned about the propagation of light through the beamsplitters, phase difference introduced by the two paths of the MZI, what happens when the detectors "click," and the effect of polarizers in various locations of the MZI in different experiments students engaged with in the QuILTs. We note that the researchers who developed and validated the QuILTs did not dictate how the topic of the MZI was to be covered in traditional lecture prior to the administration of the pretest. Instructors were advised on what should be covered before the pretest and provided a suggested grade incentive for student participation on the pretest, the QuILT used, and the posttest. Then, students were administered the pretest in class. Next, part of the hybrid or conceptual QuILT that students worked on in class was facilitated for all of the courses by one of the authors of this article, who filled in as a guest instructor. Students worked through part of the hybrid or conceptual QuILT in class and were given one week to work through the rest of the QuILT as homework. Thus, there was a level of uniformity in the in-class implementation of both of the QuILTs in all of the classes. Below, we compare student data from the hybrid MZI QuILT with data from the conceptual MZI QuILT.

III. RESULTS AND DISCUSSION
Results from the hybrid MZI QuILT: Table I shows that the graduate students and both groups of undergraduates (Groups A and B) performed poorly on the pretest after traditional instruction but the posttest performance after working on the hybrid QuILT was good for the graduate students and undergraduate group B. Moreover, comparison of undergraduate groups A and B shows that for both the pretest and the posttest, group B performed significantly better overall. Analysis of individual questions (not shown in Table I due to space constraints) shows that questions that address more basic concepts (e.g., no polarizers are involved so that the Hilbert space is two-dimensional) have similar posttest scores in both undergraduate groups A and B. This suggests that the hybrid QuILT was effective in helping these students learn the basic single-photon concepts. In contrast, Table I shows that group B, which tended to also perform better on the pretest, scored considerably higher than group A on the MZI arrangements involving higher mathematical complexity (four-dimensional Hilbert space involving product space of both path and polarization). Given that these classes were taught at the same university, in subsequent years, it is reasonable to consider the undergraduates to be comparable in prior mathematical knowledge. One possible reason for the differences between these groups may lie in the clue that comes from the difference in their pretest scores, i.e., how much the students in groups A and B had learned about these concepts after traditional instruction, which was by different instructors. Table I shows that undergraduates in group A had very low pretest scores before they engaged with the hybrid QuILT. This may potentially be due to how the course instructor for undergraduate group A taught this material, how much time the instructor devoted to this topic before students took the pretest and how this material was incentivized. Although the breakdown by questions is not shown in Table I, we note that the pretest scores are very low on some of the questions even for the graduate students, but they still managed to perform significantly better than undergraduate group A. Table I shows that graduate students performed comparable to undergraduate group B (which generally had higher pretest scores on more mathematically complex questions on the pretest than graduate students). One hypothesis for this difference between the average posttest performance of undergraduates in group A and graduate students, despite both groups having poor pretest performance (see Table 1), is that the graduate students on average may have better mathematical facility than the undergraduates. This advantage may have reduced their cognitive load while learning from the hybrid MZI QuILT despite having low level of initial physics knowledge of the MZI (as inferred from their pretest performance shown in Table I). Since students' working memory while learning from the hybrid QuILT is limited, if students have both limited conceptual and quantitative facility with the hybrid QuILT content, integration of conceptual and quantitative understanding while learning from the QuILT has the potential to cause cognitive overload and leave few cognitive resources for meta-cognition [2]. It is possible that graduate students' poor initial conceptual understanding of the MZI, as manifested by their pretest scores, did not increase the cognitive load as much as for the undergraduates in group A since graduate students on average had better mathematical facility, which has the potential to reduce cognitive load and leave some cognitive resources for conceptual reasoning and sense-making [2] while learning from the hybrid QuILT. In fact, Fig. 1 shows bar graphs of the percentages of students for the three student groups that engaged with the hybrid QuILT. This figure shows that both the undergraduate group A and some graduate students performed poorly on the pretest (Fig. 2a), and the pretest scores of many students in these groups are very low after traditional instruction (for undergraduate group A, this did not improve significantly on the posttest as shown in in Fig. 2b). We hypothesize the possible reasons for the observed trends by synthesizing these different comparisons. One hypothesis concerns the prior mathematical preparation of student groups as it pertains to their facility with making appropriate math-physics connections and learning new physics concepts in a math-rich hybrid QuILT context. On average, graduate students at this institution are typically assumed to be more experienced in applying advanced mathematical skills to physics contexts and more comfortable changing between and connecting different representations. This is in contrast to the upper-level undergraduate students. Since the undergraduate quantum course is mandatory for all undergraduates, whether they are graduate school bound or not, there is a greater diversity in students' quantitative proficiency in the undergraduate course than in the graduate course. The difference in the ability to make the mathphysics connection may manifest in graduate students not having as much cognitive overload as undergraduate group A when using mathematical representations to learn physics concepts (even if their initial physics knowledge of these concepts was not very good, as evidenced by low pretest scores in Table I and Fig. 2). This is especially important Fig. 2 Bar graph depicting the distribution of student pretest scores for each hybrid QuILT group with bar graphs 2a and 2b depicting the distribution of student average pretest and posttest scores, respectively, for each hybrid QuILT group.
because in the case of the hybrid MZI, the more mathematically complex concepts (after the polarizers are introduced) involved math-physics connection in the context of a four-dimensional Hilbert space. On the other hand, undergraduate group B may have benefited from the hybrid QuILT as much as the graduate students, as evidenced by their posttest performance, since they had the best conceptual knowledge of the underlying physics among the three groups as evidenced by their pretest scores (see Table  I and Fig. 2a), which may have reduced their cognitive load when engaging with mathematically complex concepts and making conceptual inferences from them [2].
Comparison of the hybrid and conceptual groups: Table II shows the pre/potest performance of students who engaged with the conceptual QuILT (both undergraduate classes were combined here since performances were not significantly different). A comparison of Tables I and II shows that both groups of graduate students performed similarly on average on the pretest. However, on the posttest, the hybrid QuILT group performed better than the conceptual QuILT group. One hypothesis for this difference is that the graduate students are likely to have higher mathematical facility and the ability to make the mathphysics connection. Thererfore, the integrated conceptual and quantitative MZI QuILT did not cause cognitive overload for them and is better at improving their understanding than the conceptual MZI QuILT. One hypothesis for why undergraduate group A performed well on questions without polarizers but not on others (breakdown is not shown in Table I for each question) may be that the increased mathematical reasoning for them in the hybrid QuILT may have caused cognitive overload for these students for questions with polarizers (involving product states of path and polarization). Regarding concepts in a twodimensional Hilbert space, undergraduates in Group A appear to have benefitted more from the hybrid QuILT as reflected by their posttest performance (not shown separately in Table I). However, with concepts involving fourdimensional Hilbert space (both photon path states and polarization states), their performance is poor. In contrast, the undergraduate group B which engaged with the hybrid QuILT exhibited better pretest performance than the undergraduates who engaged with the conceptual QuILT. This is likely an instructor effect, with the instructor of the hybrid group B preparing and incentivizing students better than the instructor of the undergraduate conceptual QuILT group. The undergraduates in group B outperformed all other undergraduates on the conceptual posttest.

IV. DISCUSSION AND SUMMARY
We compared upper-level undergraduate and graduate student performance on conceptual questions after engaging with hybrid or conceptual QuILTs to help students learn about fundamental QM concepts in the context of quantum optics. Both QuILTs use a guided inquiry-based approach to learning and use research as a guide. By comparing the performance on conceptual pretest and posttest, we find that integrating conceptual and quantitative aspects of the MZI with single photons in the hybrid QuILT may have provided opportunity for more effective learning for graduate students and undergraduates who had an adequate first coat of conceptual understanding of the MZI experiments when compared to students who engaged with the conceptual MZI QUILT. On the other hand, undergraduates who had very low pretest scores before engaging with the hybrid QuILT (see Table I and Fig. 2) exhibited similar or worse conceptual learning than those who engaged with the conceptual QuILT. One possible hypothesis for the significantly better posttest performance of graduate students compared to the undergraduates, both of whom engaged with the hybrid QuILT and performed poorly on the pretest, is that the graduate students, on average, had better ability to make the math-physics connection and benefited more from the mathematical representations of concepts in the hybrid QuILT without experiencing cognitive overload despite having low level of conceptual understanding manifested by their pretest scores. Adequate prior physics and mathematical facility above a certain threshold may be necessary for students to make an appropriate math-physics connection and significantly benefit from the hybrid QuILT that integrates conceptual and quantitative aspects.

ACKNOWLEDGMENTS
We thank the NSF for award PHY-1806691.