Finding Evidence of Transfer with Invention Activities: Teaching the Concept of Weighted Average

Coming to grips with the nature of measurement and uncertainty is often a common but implicit learning goal for many undergraduate physics labs. As educators, our intent is to have students be able to transfer their knowledge to novel situations: we aim to transform novices into experts. In the first-year physics laboratory at UBC, our approach to teaching weighted averages— among other concepts—involves the use of invention activities. These invention activities actively engage the students, are intended to stimulate creative thinking, are particular in their brevity and high level of structure, and are designed to precede both explicit instruction and reinforcing practice. The merit of having students inspect the fundamental makeup of a problem before being taught to solve it has been shown as useful support for the formation of an initial orderly schema (i.e., preparation for future learning). The transfer of knowledge can be rather difficult to detect in a sequestered problem solving environment, but we claim to have found some evidence of its occurrence. In a situation for which a weighted average is required, we observe significantly more students paying attention to the uncertainty associated with the problem. Given the well-documented challenges associated with teaching the nature of measurement and uncertainty—and while many students still fall short of remembering or applying the correct formula of a weighted average—we interpret this transfer of a concept as a small victory.


INTRODUCTION
One of the key, although often implicit, components of many undergraduate physics laboratories is the development of an understanding of the nature of measurement and its accompanying uncertainty; or, at least, it should be [1].A general finding of physics education research is that students can perform well on sophisticated tasks while still holding serious misunderstandings about the underlying concepts.Novice thinkers might ably reproduce various technical manipulations of data in one context while, at the same time, lack the cognitive scaffolding that permits an expert to organize and apply this knowledge in another [2,3].As educators, our aim is to transform novices to experts; we want our students to be able to transfer their knowledge to novel situations.We have taken an invention-then-telling approach to teaching weighted averages and have found evidence of transfer.The more commonly observed lackof-transfer can be framed in terms of students' inability to construct a coherent schema in the learning domain to begin with [4], thereby impeding their abilities to apply their knowledge in new domains.
The Concise Data Processing Assessment (CDPA) [5] has helped us to confirm our intuitive sense that our students have really struggled with, among other things, the significance of the uncertainty associated with a particular measurement.We view this work as an extension into the field of students' comprehension of experimental evidence and the models of student thinking about dispersion in data, termed the point-and set-paradigms by the PER group at the University of Cape Town [6,7,8].The point-reasoning paradigm-novice-like thinking-is characterized by the belief that each measurement is independent of the others and that individual measurements do not need to be combined.A measurement is perceived as leading to a single point-like value, rather than to establishing an interval of trustworthy values.In its extreme form, this way of thinking manifests itself in the notion that only one single measurement is, in principle, required to establish the true value.The set-reasoning paradigm-expert-like thinking-is characterized by the belief that each measurement is only an approximation of the true value and that deviations from it are random.A number of measurements are perceived as requisite to build a distribution that clusters around a particular value; to get a best estimate of the true value, measurements must be combined using formal mathematical procedures that describe the data collectively.Student thinking exists somewhere along the spectrum between these two extremes, and we aim to move it closer to the expert's.

INVENTION ACTIVITIES
The value of having students explore the underlying structure of a problem before being taught to solve it has been demonstrated as effectual support for the construction of an initial coherent schema (i.e., preparation for future learning) [9].One way of getting students to explore underlying structure is to have them complete activities as a preparation for future learning.One such type are "invention activities," which actively engage the students and stimulate creative thinking, are relatively brief and highly structured, and precede both explicit instruction and reinforcing practice.These tasks present a set of deliberately selected cases and an objective to invent a compact description (typically mathematical) that generalizes across the cases.Finding the correct answer to the task at hand is unimportant [10], as the point is only to construct a coherent schema in a given domain so that the students are prepared for future learning.The invention activity facilitates students in detecting important structure in the given cases and in building an organizational scaffolding that prepares them to then understand conventional descriptions.Once the activity has been completed and the students are ready to learn, they are told the expert knowledge and follow-up with practice.Studies on the added benefits of the invention-then-telling approach reveal profound differences when students are presented with more expert-like tasks that include learning new related ideas and applying their knowledge to new situations [9].
An effective invention activity has several specific features [11].First, the activity must have a clear and challenging goal, often of developing a compact and consistent description or representation of the fundamental attributes across the given cases.The solution usually involves integrating several features of the cases into one single representation (e.g., a ratio).Second, the activity should present multiple contrasting cases simultaneously to the students.Contrasting cases assist in the development of early knowledge because they help learners to notice new features or structure and to develop new interpretations.Learning to perceive has been described in terms of observing what discriminates one thing from another [12] and contrasting cases are a powerful way to help people discern differentiating properties.Cases should systematically vary on key parameters so students can see how these variations relate at a deeper, structural level.Third, the activity should involve student collaboration, and be done by pairs or groups of students.This carries the advantages of a greater number of ideas considered, as well as with peer instruction.It is also important to be mindful of: context (the task should involve things relatively familiar and meaningful to the students); level of difficulty (the task should be structured so that students typically achieve partial success); and the absence of jargon (the task should be free from subject-specific vernacular, which commonly triggers students to attempt recall of formulae they have already learned rather than inducing a response more closely related to dealing with the development of a new process).
We have designed a set of invention activities focusing on some of the most difficult concepts covered by the learning goals of our first-year physics lab course for honours physics students.This course consists of four sections, populated by as many as forty-eight students per section.Each 3-hour lab section is headed by the very same instructor, and is further supported by two experienced teaching assistants.The invention activities are completed during lab time and are given, on average, every other week.Once the invention activity has been completed, the students apply the new knowledge to in-lab practice problems and then, again, to the data they subsequently collect.A previous article [13] presents an example of a two-part invention activity that successfully prepares students for lessons on creating histograms and calculating standard deviations; i.e., subsequent transfer assessment tools demonstrated that students were learning, retaining, and re-applying this knowledge in a transfer setting.In this paper we describe an invention activity that focuses on weighted averages and share encouraging, albeit modest, signs of transfer success.

OSTRICH EGGS: A WEIGHTED AVERAGES INVENTION ACTIVITY
The contrasting cases presented with the invention activity are shown in Tab.I (the students also received this data in graphical representation), and the explicit instructions are as follows: "During an episode of Iron Chef, a televised cooking competition, teams are provided with an unusual ingredient that they must use in the meal they are preparing.In a local competition, four contestants are provided with a selection of ostrich eggs as their unusual ingredient, from which they must pick only one to cook with.In order to select the biggest egg for their dishes, the assistant chefs must find the largest ostrich eggs available.Each of the four assistant chefs chooses an egg and measures its diameter four times, in the interest of improving the accuracy of their measurement.Now they need to report the diameter of their chosen egg to their head chef, but they're not sure what to do with their four separate measurements.Throughout the next sections, your job will be to invent a method that each assistant chef can use to yield a single value, which they can then report as the diameter of their ostrich egg."TABLE I.The data associated with the invention activity; all the numbers are in units of centimetres.The contrasting cases are: Chef A has four small and equal uncertainties; Chef B has three small and equal uncertainties and one large uncertainty; Chef C has one small uncertainty and three nearly equal large uncertainties; Chef D has four moderate and equal uncertainties.To be clear-and fair-we did not expect students (in 2012: 129 students, working in 57 groups of two or three) to invent for themselves the correct formula for a weighted average over the course of a 30-minute exercise.Student inventions were varied, and 20% completely neglected to incorporate uncertainties into their methods at all, calculating simply an unweighted arithmetic mean.A relative majority of students (44%) attempted to weight the data by the inverse of their uncertainties but did not normalize their formula, an example of which is shown in Eq. 1 and resulting in a final answer that is dimensionally incorrect.Only 14% of students attempted to both weight the data and normalize their formula, an example of which is shown in Eq. 2. The remaining 22% of invented methods do not sort neatly into any of the above three categories.
Upon completion of this activity, direct instruction was given on how an expert would manage this data but only after the students had put substantial thought into what would be needed when combining such data.The expression for a weighted average [14] when there are N separate measurements of a quantity xi, each with an associated uncertainty si, is given by Eq. 3.
As the weight attached to each measurement involves the square of the associated uncertainty, any measurement which is much less precise than the others contributes very much less to the final answer.

MEASURING TRANSFER
With respect to learning how and when to calculate a weighted average, indications of transfer-near in context but distant in time-have surfaced in a sequestered problem solving setting.Our assessment item, taken from the CDPA [5] and shown in Fig. 1.
This item presents two separate measurements and a situation for which a weighted mean is reasonably expected.(This is explicitly different than directly asking students to calculate a weighted average given two data with uncertainty.When asked in this way, we have found it to result in decreased performance on the item.From interviews, our hypothesis for this result is that once you have a trigger that has students hunting for an equation, they become significantly less able to think outside of that box and reason their way to a simpler solution or to simply reproducing what it is that the equation does.We often underestimate the power and prevalence of informal heuristics; it is our experience that students will often prefer these over formal methods.) Student performance on this item, measured in a post-test two months after the activity, is shown in Fig. 2.
Presented in decreasing order of sophistication of student thinking, the five options in Fig. 2. are: Option b is the correct answer and represents those who recognize that the measurement of (90 ± 12) mL/s is significantly poorer than (100 ± 1) mL/s.Students arrive at this answer either by calculating a properly weighted average or by discarding the significantly less reliable measurement altogether.
Option d represents those who recognize that important information is contained within the uncertainty but who misinterpret or misapply this information.We view this as "not-so-wrong" an option, and argue that it straddles the point-and set-paradigms of thinking.In selecting this option, students use the number 105.5 mL/s, which is  exactly halfway between the values of (90 + 12) mL/s and (110 -1) mL/s.This thinking is point-like in that it assigns particular importance to a singular value; namely, the midpoint of the two values.But this thinking is also setlike in that it treats the data as two separate distributions that cluster around particular values.
Option e was initially included to provided an apparent balance in the options presented and was previously not a known model of student thinking.Ensuing student interviews have shown that some who select this option have weighted the relative importance of the numbers in a reasonable way but subsequently struggle with the algebra that follows.
Option c represents those who discard the more reliable measurement.
Option a represents those who ignore the information contained in the uncertainty altogether and calculate simply an arithmetic mean.
These results imply progress when compared against previous years in which an equal amount of in-lab, timeon-task was spent on direct instruction of and practice with weighted averages.The number of students who calculate an unweighted arithmetic average (option a) has decreased considerably from four years ago, from 30% to 12%, in conjunction with the introduction and refinement of the invention activity.A chi square test of proportions, using a 2x2 contingency table of year by correctness, was conducted to compare the fraction of students selecting option a over the years.There was a statistically significant difference in the proportion of students selecting option a in 2009 & 2010 (75 of 254 students) and in 2012 (15 of 129 students): c 2 (1) = 15.25, p = 0.0001.These results suggest that the Ostrich Eggs invention activity does have an effect on getting students to consider the uncertainty associated with data.At the same time, the number of students who consider the uncertainties in their solutions (option d) appears to have increased, although not by enough to claim statistical significance.This shift away from ignoring uncertainties entirely towards incorporating them (i.e., "paying attention" to them) in their solutions supports the notion that high-quality invention activities can indeed better prepare students to learn from future instruction.While the number of students who chose the correct answer (option b) has not increased, we do see improvement in the quality of students' mental models, evidenced by the majority of them who take the uncertainties into consideration.

CONCLUSION
The activity described above has been designed to prime students' minds for a subsequent lecture on calculating a weighted average.In getting students to invent original solutions to novel problems, the activities serve to prepare students for future learning.The transfer of knowledge can be particularly difficult to observe in a sequestered problem solving scenario, but we have found evidence that it is occurring.Many students are still not remembering the entire formula required to calculate a weighted average, but they are now transferring in "attention paid" to the uncertainty associated with the problem.Considering how difficult it has been to make inroads with the nature of measurement and uncertainty, we interpret this transfer of a concept a minor victory.

FIGURE 1 .
FIGURE 1.The transfer question probes whether students know that the measurement with the much larger uncertainty should carry much less weight.The correct answer is (b).

FIGURE 2 .
FIGURE 2. The transfer results.In 2009 and 2010 (N = 254), students received traditional instruction for how and when to calculate a weighted average.In 2011 (N = 132), the first iteration of our invention activity focusing on weighted averages was deployed.In 2012 (N = 129), an improved second iteration, presented in this paper, was used.