Investigating context dependence of introductory and advanced student responses to introductory thermodynamics conceptual problems

We use a validated conceptual multiple-choice survey instrument focusing on thermodynamic processes and the first and second laws of thermodynamics as covered in introductory physics to investigate the context dependence of introductory and advanced student responses to introductory thermodynamics problems after instruction. The survey has conceptual problems that incorporate many contexts with the same underlying principles and concepts involving internal energy, work, heat transfer, and entropy. Here we focus exclusively on entropy. This study used data from over 1000 college students in introductory-level algebra-and calculus-based physics courses as well as upper-level thermodynamics courses. In addition to prior research, think-aloud interviews with a small subset of students in which they were asked to answer the survey problems while thinking-aloud were useful for understanding the context dependence of student responses in some situations, and why students may have greater difficulties in some contexts than in others. Here we present analysis of data in multiple contexts reflecting students’ ideas about the change in entropy of a gas in spontaneous/irreversible processes and in cyclic processes. We find that a persistent belief in the constancy of entropy even for spontaneous/irreversible processes is a common difficulty among introductory students across problems with different contexts, while upper-level students had great difficulty across contexts in which identifying entropy as a state variable is important. For example, overall, upper-level students struggled somewhat more than introductory students with the fact that the entropy of the system does not increase, e.g., in cyclic processes after one complete cycle. Our findings using a validated survey confirm the findings of prior research in multiple contexts.


I. INTRODUCTION AND GOAL
Physics is a discipline that is devoted to explaining diverse physical phenomena using just a few basic physics principles.To develop expertise and learn physics effectively, it is essential to unpack the underlying meaning of the abstract principles and concepts to recognize their applicability in diverse situations [1][2][3][4].Research shows that identifying and applying relevant physics principles and concepts involved in different contexts is an important hallmark of expertise in physics.Many physics courses focus on helping students learn to discern the deep similarities between problems that share the same underlying physics principles but have different "surface" features, so that students can transfer their knowledge across different contexts to solve problems.Transfer of knowledge refers to the application of knowledge and skills learned in a given context to other contexts [5][6][7][8][9].
Two physics problems that appear to be very similar to a physics expert because both involve the same physics principle may not necessarily look similar to novice students who are still developing expertise [10,11].For example, a study on the categorization of introductory mechanics problems [10] based upon similarity of solutions indicates that experts usually group problems based upon the underlying physics principles while novices are more likely to be distracted by other features of problems, and may group problems based on the surface features such as the inclined plane, spring, or pulley even if the underlying physics principles to solve them are different.
The different ways experts and novices categorize problems may also reflect the different ways in which their knowledge is organized [10,11].Research suggests that experts in physics have a hierarchical knowledge structure (schemas), in which the most fundamental physics principles are placed at the top, followed by layers of subsidiary knowledge and details [12][13][14][15].This well-organized knowledge structure facilitates their problem-solving process [12][13][14].It also guides the experts to recognize the deep features of the problems and makes the transfer of their knowledge to different contexts easier.
Since helping students recognize the applicability of the physics principles and concepts they have learned and apply them correctly in different contexts is an important goal of physics education for science and engineering students, many research studies have investigated students' ability to transfer their knowledge to different contexts [16][17][18][19].Cognitive theory suggests that transfer can be difficult especially if the source (from which the knowledge is to be transferred) and target (a problem at hand to which the knowledge is to be transferred) do not share surface features.The source may be the context in which a particular physics principle or concept was learned, while the target may be quite a different context.
The failure to appropriately transfer can often be attributed to the fact that knowledge is encoded in long-term memory (LTM) with the context in which it was learned and the features of the target problem to be solved may not lead to accessing relevant resources in LTM even though the two problems share deep features [1,20].Solving problems in new contexts correctly requires unpacking and understanding the applicability of the physics concepts in diverse situations.
Research shows that the robustness of the knowledge structure and the context in which the knowledge is acquired can affect an individual's ability to apply knowledge flexibly across different contexts [10-14, 16, 17].For this reason, prior studies have used various scaffolding mechanisms to assist students in learning to transfer their knowledge correctly in different contexts [21,22].For example, students can be taught to use isomorphic (similar structure) problems and analogical reasoning to link problems that involve the same underlying principles and concepts [21][22][23][24][25]. Isomorphic problems have the same underlying principles or concepts but have different surface features.To improve transfer of knowledge across contexts, it is important to help students contemplate the applicability of the same physics principles and concepts in different contexts and learn to decontextualize knowledge and store it in their LTM at a more abstract level [1].
When transfer of knowledge in a given context is expertlike and the knowledge which is accessed helps students solve a problem correctly, it can be called positive transfer.However, Lobato posited that transfer should be considered from the perspective of the person solving problems [7].In her actor-oriented transfer framework, if students transferred knowledge in a manner that was not useful to solve a problem correctly in a given context, this is still a transfer of some knowledge from students' perspective [7].We can call this type of transfer a negative transfer [7].Consistency of positive or negative transfer can be measured by using isomorphic problems and investigating how consistently students perform across different contexts.A consistent positive transfer of their knowledge could signify that students have a good knowledge structure of the underlying concepts and principles, and appropriate knowledge is accessed from the LTM regardless of the contexts and surface features of the problems [1].If students have the relevant knowledge in their LTM to correctly solve the problems posed, a consistent negative transfer could mean that students are unable to recognize the applicability of relevant concepts and principles and may be getting distracted by some surface features of the problem or may have a strong alternative conception pertaining to some problem features [10,11].An inconsistent positive transfer to solve problems could imply that students have some knowledge of the underlying concepts and principles, but that it does not amount to a mastery of the material because the appropriate knowledge is accessed from LTM and applied correctly only in some contexts [1].
Here we discuss an investigation of how introductory and upper-level students access their knowledge about entropy and make use of it in various contexts requiring application of the same underlying concepts.Prior research suggests that both introductory and upper-level students have many difficulties with introductory thermodynamics concepts [26][27][28][29][30][31][32][33][34][35].All upper-level students were once introductory students, so the consistency with which they use their knowledge to solve conceptual introductory thermodynamics problems across various contexts-combined with information about solution methods typically used by introductory students-can provide insights into the learning process that are potentially helpful to instructors in better meeting the needs of students in their physics courses.
There are many factors that can affect student ability to transfer their knowledge such as the context in which the concept was learned, how similar the surface features of the new problems are to the ones they have solved in the past, and whether certain features of the problems act as distractors.Here are our research questions: RQ1: To what extent are introductory and upper-level student responses dependent on the problem context for problems with the same underlying concepts related to entropy?
RQ2: How different are the context dependencies of introductory and upper-level student performance on entropy problems across different contexts sharing a common theme?

II. METHODOLOGY
The Survey of Thermodynamic Processes and First and Second Laws-Long (STPFaSL-Long), a validated survey instrument with 78 problems, was used in this research; the instrument focuses on introductory thermodynamics concepts.The details of the development and validation of the STPFaSL-Long can be found in Ref. [36] and the survey can be found here [37].Most problems on the survey have four possible answer choices; most of the problems dealing with entropy in different contexts have options asking whether it increases, deceases, remains the same in the given situation, or whether there is not enough information.Only 22 out of 78 problems are true/false (T/F) problems.Here we only focus on clusters of problems in multiple contexts in which underlying concepts about entropy are similar.
This investigation on the context dependence of student responses uses survey data after instruction (post-test) in relevant concepts.In particular, the written data analyzed here were taken by administering the survey in proctored inperson classes as a post-test after students had learned the relevant concepts, but before their final exam in the course.Students were given some extra credit for completing the survey.These written student data are from 12 different inperson courses from five different large public institutions and all students completed the survey in class on Scantrons in a 50-minute class period.We discuss analysis of context dependence in the written data from three groups of students: 550 students in the introductory algebra-based (Int-alg) physics course, 492 students in the introductory calculusbased (Int-calc) physics course, and 89 students in their upper-level thermodynamics course.Students in the Int-calc courses were typically engineering majors with some physics, chemistry, and math majors, while students in the Int-alg courses were mainly biological science majors and/or those interested in health-related professions.Students included in the upper-level group were typically physics majors in thermodynamics courses or Ph.D. students in the first-year, first-semester of their graduate program, who had not taken any graduate-level thermodynamics.(Since the survey was administered as a pre-test to this latter group of students, they were presumed to have taken upper-level undergraduate thermodynamics.) The interview data are from 11 introductory and 6 upperlevel students from one institution who volunteered after an opportunity to participate in this study was announced.Each interview lasted between 1-2 hours in one sitting depending upon students' pace.The interviewed students were given $25 for their participation.The interviews used a semi-structured think-aloud protocol.Students were asked to think-aloud as they answered the questions and were not disturbed except to keep talking if they became quiet.Only at the end did we ask them for clarifications of points they had not made clear.

III. RESULTS
Table I shows responses to problems related to entropy change posed in different contexts.Three distinct student populations are represented: Int-alg, Int-calc, and Upper Level.

a. ΔS>0 for the universe in an irreversible process
Table I focuses on the context dependence of student responses on problems 17, 21, 53, 67 and 75 pertaining to the fact that for the universe, ΔS > 0 for a spontaneous/irreversible process.The fifth problem in this category, problem 67, involves an isochoric process with net heat transfer to an ideal gas; it is explicitly identified as an "irreversible" process in the problem statement, the only one of the five so identified.
Although four of these problems, 17, 21, 53 and 75, have contexts involving isolated systems with spontaneous processes, they have different surface features that can distract students.For example, Problems 21 (shown in Fig. 1) and 75 can be viewed as similar to each other.Problem 21 involves a free expansion with an ideal gas initially in one chamber expanding into a vacuum when the stopcock is opened, while problem 75 involves two different non-interacting ideal gases initially in separate chambers mixing with each other.The context of problem 75 can be viewed as analogous to the free expansion in problem 21.
FIG. 1. Free expansion of a gas in an insulated container.As shown in problem 21 of the STPFaSL-Long, the gas is initially in thermal equilibrium and confined to the left chamber.When the stopcock is opened, the gas is allowed to expand evenly until the system reaches equilibrium.The problem asks if the entropy of the system increases, decreases, or remains the same after the stopcock is opened and equilibrium is reached.FIG. 2. Two solids in an insulated case.As shown in problem 17 of the STPFaSL-Long, one solid is at an initial temperature of TC and the other has an initial temperature of TH where TC << TH.The solids are placed in contact and the problem asks if the entropy of the combined system of two solids increases, decreases, or remains the same once thermal equilibrium is reached.
Problems 17 (shown in Fig. 2) and 53 can also be viewed as a similar pair, perhaps even more so than 21 and 75.Problems 17 and 53 both have two sub-systems at different temperatures in contact with each other, with heat transfer between them.(There are two solids at different temperatures in thermal contact in problem 17, and two gases at different temperatures in thermal contact in problem 53.)Therefore, problems 17 and 53 share an identical element-heat transfer between the hot and cold subsystems-and thus are even more similar to each other than is the 21/75 pair.
Despite the similarities linking the 21/75 pair and the identical element joining the 17/53 pair, the only feature that the 17/53 pair truly shares with 21/75 pair is that they are all irreversible.That is, all four problems involve isolated systems undergoing spontaneous processes leading to ΔS>0 for the universe.One might then expect similar student outcomes for the 21/75 pair on the one hand, and for the 17/53 pair on the other, but not necessarily similar results when comparing the two pairs.In fact, that is exactly what is found: response rates for the 17/53 pair are almost identical, but very different from the 21/75 pair which showed many more correct responses.Moreover, while the 21/75 pair had lower error rates than 17/53, the results for 21 and 75 were not as similar to each other as those in the 17/53 pair that shared an identical element.As for why error rates in 17/53 were higher for introductory students than in the other pair, interview evidence suggests that students may view free-expansion as a clearer signal of entropy increase than heat flow.
Table I shows that a majority of the upper-level students provided the correct response regardless of the context of the problem, but there is a large context dependence for both introductory groups, with a large fraction of the introductory students not providing correct responses for each problem context.In particular, upper-level students remain fairly consistent with their responses for all five problems, with percent correct ranging between 78%-87%.This suggests that most upper-level students can consistently correctly transfer their knowledge across different problem contexts involving ΔS>0 for the universe in spontaneous/irreversible processes.Thus, we will now focus mainly on introductory student responses to these problems.
The introductory groups had a common but highly contextdependent alternative conception that ΔS=0 for the universe in the spontaneous/irreversible processes.Table I shows that introductory students struggled the most with problems 53 and 17, as less than 25% of them provided correct responses to either of those problems.(These two problems both involve spontaneous heat transfer from the hot substance to the cold substance; the substances were solids in problem 17 and gases in problem 53.)Furthermore, Table I shows that the contexts of problems 17 and 53 are so challenging that roughly two-thirds of both introductory groups provided incorrect responses stating that entropy of the universe in these processes does not change.Interviews corroborate these findings.For example, on problem 17, one interviewed student who thought ΔS=0 said, "Since there was no loss [of heat] to the environment, we will assume that the entropy has not changed."On the same problem, another interviewed student said, "Change in entropy must be zero because of equilibrium." Table I also shows that for both introductory groups, the correct response rates on problem 21 (free expansion) are only slightly higher than on problem 75 (gas mixing).However, there is a large difference in introductory groups' incorrect ΔS=0 response.While 47% and 43% of the Int-calc and Int-alg groups, respectively, responded ΔS=0 for the mixing process (problem 75), the corresponding percentages for the free expansion process (problem 21) are 29% and 27%.Response rates for problem 67, the isochoric process explicitly identified as irreversible, fell between those for problems 21 and 75, indicating that even this problem was quite challenging.
In summary, upper-level students performed significantly better overall than introductory groups and were more consistent in their responses.Introductory students' responses were context dependent, with mixing/free expansion problems being easier for them than were heat transfer problems.The correct response rates of the two introductory groups for all problem contexts related to entropy of the universe in irreversible processes are similar, all in the 20-50% range.

b. In a complete cycle, ΔS=0
Table I also shows the context dependence of student responses on problems 8 and 24 pertaining to the fact that ΔS=0 for a gas that undergoes a full, complete cycle.Problem 8, shows a PV diagram, shown in Fig. 3, with three processes that form one complete counterclockwise cycle while problem 24 shows a PV diagram with four processes that form one complete clockwise cycle, shown in Fig. 4.After a complete cycle, the initial and final states are the same, so there is no change in entropy of the gas.For problem 8, the types of processes in the counterclockwise cycle are not explicitly mentioned, but for problem 24, the specific processes constituting the clockwise cycle are explicitly given.The answer options for these problems were that in one complete cycle, the entropy of a gas would either increase, decrease, remain the same, or there wasn't enough information.Table I shows that students in all three groups had difficulty with the fact that entropy is a state variable and ΔS is therefore path independent.In particular, during a thermodynamic process, regardless of how a gas gets from its initial state to its final state, only the two end points determine the change in entropy.In a cyclic process after one complete cycle, since the gas ends up in the same state that it started in, ΔS=0.
Table I shows that the upper-level and introductory student groups had reasonably consistent performance in the two contexts.However, the possibly slightly worse performance of all three groups on problem 8 compared to problem 24 may be due to the contexts of problems 8 and 24 involving counterclockwise and clockwise cycles, respectively.The correct response rates for problems 8 and 24 for upper-level students were 49% and 57%, for Int-calc group 54% and 63%, and for Int-alg group 65% and 67%.Thus, introductory groups slightly outperformed upper-level students.The comparison of the performances of upper-level and introductory students shown in Table I suggests that there is no significant learning in the upper-level courses pertaining to entropy being a state variable.This is a cause for concern that upper-level thermodynamics instructors should take into account.
The most common alternative conception for all groups was that the entropy of a gas increases after a complete cycle.Table I shows that across both problems, the upper-level students have a stronger alternative conception (approximately one-third of the upper-level students) than introductory students (approximately one-fourth of the introductory groups) that the entropy of a gas increases after one complete cycle.This is echoed in interviews with upperlevel students.For example, for problem 24, one upper-level interviewed student said, "is heat in or heat out related to the entropy?" and then tried to determine the heat transferred into the system in each step of the cycle in order to find the change in entropy.It is true that in a clockwise cycle, there is a net heat transfer to the system, but this upper-level student was distracted by the individual processes and reached the incorrect conclusion, not recognizing that entropy is a state variable and is therefore unchanged since the initial and final states are identical after a cycle.

IV. SUMMARY
We find that introductory students' application of their knowledge about entropy depends heavily on problem context.Introductory students' responses on irreversible processes were context dependent; they performed better on processes involving gases expanding than on problems involving heat transfer.Interviews suggested that many students are unaware that heat transfer processes involve net entropy increase.For cyclic processes in which ΔS=0, correct responses were given slightly more often by introductory students than by upper-level students; both groups' correct response rates were in the 49-67% range.Lack of opportunity to develop a robust knowledge structure can prevent even the upper-level students from solving conceptual introductory problems successfully in different contexts.Therefore, one major goal of both introductory and upper-level physics instruction should be to help students construct robust schemas so that their well-organized knowledge structure can assist them in recognizing the applicability of different physics concepts learned in different contexts.

FIG. 3 .
FIG. 3. The PV diagram of a gas undergoing a complete counterclockwise cycle in problem 8.

Table I .
Percentages of introductory algebra-based (Int-Alg), calculusbased (Int-Calc) introductory physics, and upper-level students whose post-instruction responses were in the categories shown for problems probing issues related to entropy in various contexts.The problem number for each problem is shown.