Learning about Educational Change Strategies: a Study of the Successful Propagation of Peer Instruction

Millions of dollars and uncounted hours have gone into the development of research-based instructional strategies for the improvement of undergraduate physics. However, many of these instructional strategies have extremely limited impact beyond the developer(s). Peer Instruction is the best-propagated research-based instructional strategy in undergraduate physics. We use case study methodology to construct the story behind how Peer Instruction became widely known and widely used. This case study uses interviews with the original developers and document evidence such as publications and presentations to build a rich description of the early events and propagation strategies used (knowingly or not) that have led to the spread of Peer Instruction. Peer Instruction's primary dissemination vehicle was story, through talks and the user's guide (which closely aligns with the talks). Further, many of the talks (perhaps as many as 600 in total) were given as departmental colloquia, thus leading to buy-in from departments as a whole unit. Finally, the Peer Instruction User's Manual gave potential adopters the tools they needed to implement it quickly and easily, and the team continually supports potential adopters on an individual, involved basis. These findings have implications for physics education researchers who want to increase the impact of their work.


INTRODUCTION
Researchers in STEM education have developed many new teaching strategies and innovations over the past 30 years that can be shown to have a positive effect on student learning [1].These innovations rarely spread outside of the developing institution or even the developing faculty member [2].One researcher might develop a classroom tool that works in their own circumstances (their own classroom with their own students and their own department), but getting this new tool put into productive use by other instructors is a challenge for many complex reasons [3].Prior research has identified substantial barriers to instructional change [4] and has identified the failure of currently-used change strategies to overcome these barriers [5].Missing from the literature is rigorous research about change strategies that do work.
One way to learn about change strategies that do work is to study instructional innovations that have been successfully propagated.We are involved in a larger research project with the goal of developing a model for effective propagation.This model is built on various sourcesthe literature, typical practice, and highly successful practice.
This paper describes how we use case study methods to describe the story of highly successfully propagated research-based instructional strategies to test and elaborate the current state of the model.We have conducted one other case study [6] and draw on the work of others in literature [7].This paper focuses on our case study of the propagation of Peer Instruction [8].
Peer Instruction is a research-based instructional strategy which changes pedagogy to be more studentcentered.It is a modification to traditional lecture to engage students through conceptual questions (ConcepTests) which they then discuss with each other.Resulting gains in student learning are well-documented [9].

Visualizing Effective Propagation
We have found it helpful to express propagation as a visual, three-stage model.Our propagation model is based on prior work [10] as well as the study discussed in this paper.Although the evolution has been ongoing, three important versions of the model are given below.Note that typical propagation practice follows version 1.
The primary goal of this study is to use a rich case study of the successfully propagated strategy of Peer Instruction to test and elaborate on the current state of the propagation model: 1. Does the model developed to understand other innovations also represent the propagation of Peer Instruction?

METHODOLOGY
Grounded theory [11] guides the overall design behind building the model of successful propagation actions.This allows the researcher to generate a "theory" based on data, often doing a literature search after the initial theory has been developed.Data is analyzed as it is collected, rather than after all the data has been collected, allowing the researcher to tailor future data collection to fit new questions that arise during analysis.Since this study looks at multiple successfully propagated instructional strategies, case study methodology [12] was used to build each case.Case study is ideal for studying contemporary events over which the researcher has no experimental control, and the research questions seek explanatory answers.
One key component of case study research is the inclusion of multiple sources of data.Multiple data sources allow the researcher opportunities for triangulation, or checking one source of data against another in a chain of evidence for claims.Triangulation is a vital component in establishing construct validity for a case study.
The sources of data include, in order of relevance: interviews with the Peer Instruction team (N=4), PowerPoint documents for workshop and colloquium presentations over a fifteen year period, the Peer Instruction User's Manual [8], videos of talks available on YouTube, academic publications about Peer Instruction, and press releases.
Interviews were used as a primary source.They ranged from 30 to 90 minutes, were conducted over the phone and recorded, then transcribed verbatim.Interviews were semi-structured, using set questions but allowing for follow-up questions during the interview.
Analysis was an iterative process using multiple data display techniques.Interviews were coded line-by-line using active codes (i.e., "Leveraging network"), other documents were used to confirm/disconfirm connections made from the interviews, and these fed into pre-identified stages of successful propagation (Figure 1).We then compared the development and propagation with existing change literature [13,14,15] and case studies of other educational innovations [6,7].Finally, the interviewees were asked to read over early write-ups and make comments to ensure the findings represented their views and were factually accurate.When questions arose from other sources of data, interviewees were contacted via email to clarify.

Defining the propagation model
Prior work [10] was used to develop this three-step model for successful propagation with emphasis on active involvement with the target audience (Figure 1, version 3).As an idealization of a messy process it is important to keep in mind that in reality, these stages overlap and may also be revisited.
Interactive development means to develop the new innovation with collaborators and/or future adopters.This means having feedback from the beginning, possibly a trial period collecting data from other implementations to strengthen the innovation.With other people involved from the start, there are more stakeholders in the new innovation and also more opportunities for feedback.
Interactive dissemination means to engage the target audience directly to get the word out about the innovation.Typical practice uses academic, mass-media channels almost exclusively (such as articles and talks) which can only succeed in raising awareness of the innovation.According to change literature [14] possible adopters need more than mere exposure to the innovation before they adopt it.Offering an interactive multi-day workshop is an example of interactive dissemination.
Support means to provide some sort of assistance to adopters as they attempt to use the innovation.For example, being available to consult during the beginning of a secondary implementation.Support is less well-understood in the context of higher education than the other two stages listed above.

RESULTS
The results are given in three parts, organized by the model described above.

Interactive development
Peer Instruction was developed primarily by Mazur but with collaborators at Harvard and other institutions (interviews).Early on, Mazur had funding to support three students as staff on the project-they gathered data and learned directly from student interviews [16].
Further, there was frequent interaction with other members of the PER community.Early ConcepTests were developed by other researchers [17] and they form the backbone of the types of questions that make Peer Instruction effective.ConcepTests have been tweaked by the Mazur group over the years.This interaction with other researchers continues today; there is a current initiative to combine Peer Instruction with Just in Time Teaching (interviews, NSF grant search).
Mazur knew from the beginning that the change he wanted to occur in classrooms, students talking to each other, had to be a modification and not a radical departure in order for it to be widely adaptable [16].In the first NSF grant used to develop Peer Instruction, Mazur teamed up with another institution besides Harvard in order to make Peer Instruction an innovation that would work in other settings.

Interactive dissemination
Here is the place where the propagation activities used by Peer Instruction depart radically from typical practice.Departmental colloquia were the dissemination engines for Peer Instruction.Mazur gave a talk at AAPT in 1991 about using computers to teach physics (it hinted at using ConcepTests) and this was the beginning of many more talks to come.Whereas we know from surveys of principal investigators that they expect other faculty to adopt their innovation based on brief exposures to the work [10], the typical conference talk was just the beginning when it comes to the widelypropagated Peer Instruction.For the audience, the talk functioned as a brief advertisement for physics faculty to invite him to go into more detail [16].Mazur has given over 600 departmental colloquia (interviews, Mazur group website).
Making the time to travel constantly to discuss Peer Instruction with physics departments led to widespread awareness and buy-in from whole departments, and a snowball effect of more invitations to speak.Over time, the willingness to make propagation a priority has remained a feature of the team.
Other propagation strategies used were the writing of the user's guide, running workshops, and producing videos (the DVD).Notably, the talks do not follow the usual structure of an academic talk, but rather are structured as a story, a point seen in change literature [15].

Support
Supporting adopters is a priority for the Mazur group and they have tried many ways to do so.Here is an illustrative quote from a past post-doc: "…we did some work in the Dominican Republic where we were working on pedagogical innovation and […] some professors came to Harvard from the Dominican Republic and learned… Peer Instruction and Just-in-Time Teaching and […] some other assessment techniques… And then actually Eric I went to the Dominican Republic and did additional talks there and visited with teachers who we had met on their trip to Harvard to talk about what their implementation was, and then I went back a third time and we did sort of an innovation showcase where […] they did posters that talked about how they had implemented the practice.And then of course sometimes people will email me personally and ask me, you know, people from that group will email me personally and ask me questions and things like that." We highlight this quote because the actions differ dramatically from that of usual practice; we know that principal investigators often leave dissemination for the end of the grant and then, once the grant is over, stop work on it entirely [10].
In addition to the direct assistance to put Peer Instruction into place, there are efforts to develop online modules which will allow instructors to train themselves (interviews).1990: The Force Concept Inventory comes into existence, leading to Mazur trying it with his Harvard students [8,17].
1991: Fledgling form of Peer Instruction first implemented [8].Mazur also gives an invited talk at AAPT which discusses conceptual questions.[17] 1992: Sheila Tobias featured Peer Instruction as a chapter in her book [16].Later in the year, Mazur's development grant for Peer Instruction was funded (it had been rejected once before.)Mazur begins to be invited to a deluge of colloquia.(Interviews) 1996: Mazur wrote and completed his book, with Tobias's encouragement and the publisher's support.Peer Instruction is one of the innovations at the New Faculty Workshop.

Does the model developed to understand other innovations also represent the propagation of Peer Instruction?
Peer Instruction did not on the surface slot easily into the propagation model that fits other innovations, with clear interactive development, interactive dissemination, and support stages-at first it appeared to be developed in isolation, using mass-media talks to disseminate, with no clear support structures in place.However, using our propagation model to analyze the data allowed us to follow up and actively pursue areas where it did not appear to match, uncovering along the way points that were omitted in initial interviews or never mentioned in other data.The model is thus a powerful lens through which to examine the story of Peer Instruction, which on the surface could have appeared not useful to the faculty member who has an idea for an innovation and needs strategies to propagate it.

How does Peer Instruction fit into or modify the model?
This study of Peer Instruction allowed us to unpack some meaning within the three stages, giving more examples of how to accomplish these items.
Interactive development: Student interviews and bringing in collaborators early on allowed for feedback, a feature shared by other successfully propagated innovations [6,7].

Interactive dissemination:
The primary dissemination of Peer Instruction was done through narrative storytelling using concrete quotes and examples [8,15].This is a departure from the usual structure of academic talks.Further, departmental colloquia served as a primary dissemination method, leading to buy-in from the entire department.This aligns with literature suggesting that the department is a key unit of change [13].
Support: The Peer Instruction book, freely available to physics faculty, provides adopters a set of resources to use, rather than needing to come up with materials themselves [7,15] Also, the team makes propagation through personal assistance and continual innovation a priority, which it has in common with other cases [10].
The model of interactive development, interactive dissemination, and support will continue to be added to with the study of future cases.

2 .Figure 1 .
Figure 1.Our changing model for successful propagation.The boxes are actions conducted by a change agent.While they are depicted in linear order for simplicity they can and do overlap in practice.Version 1 represents the typical, naïve view of propagation.Versions 2 and 3 are more sophisticated, departing from the typical practice of isolationist development and passive, mass-media dissemination.The item "support" is new in the model and not as well understood.

Figure 3 :
Figure 3: Timeline of early events in the history of Peer Instruction.