Examining the Use of PhET Interactive Simulations in US College and High School Classrooms

A growing number of K12 and college educators are using the collection of interactive simulations developed by the PhET Interactive Simulations project at University of Colorado Boulder. The design of each simulation seeks to support educators in achieving a range of pedagogical goals through implicit scaffolding. To decrease barriers to adoption, the simulations are designed for flexible use, available free of charge, and usable across platforms. The resulting simulations are compatible with diverse learning goals, implementation contexts, pedagogical approaches, grade levels, and learners. While website statistics establish that over 45 million simulations are run per year, these statistics provide little information about who uses the simulations and how they are being used. Here we report on analysis of responses to a large-scale, self-report survey from 1,233 high school and 276 college educators who are using PhET in their physics instruction, to provide insight into who uses PhET simulations, with which populations of students, towards what pedagogical goals, and with what instructional approaches. We found that new and experienced teachers use simulations, with diverse student populations, and to support a wide range of pedagogical goals. The responding high school and college educators were found to be similar in the goals they intended use of PhET simulations to support (for example, to develop conceptual goals, and to conduct science inquiry), but differed in their instructional approaches.


INTRODUCTION
One goal of PER is to promote widespread, sustained, effective implementation of the many strategies, technologies and curricula that have been shown through research to be effective at improving student learning [1].However, such widespread change has proven to be elusive at the college level [2,3].A majority of physics faculty (90%) are aware of at least one research-based instruction strategy (RBIS), and 80% have tried to use such a strategy in their teaching [2,3].However, one-third of faculty discontinue the use of these innovations [2], likely due to barriers encountered during implementation [2,4], such as time and resources.Additionally, many faculty modify the approach of an RBIS to such a degree that the lack of fidelity impedes its effectiveness.
The PhET Interactive Simulations project (PhET) is an established RBIS.A central component of PhET is the goal of supporting effective and widespread implementation directly through the design of the simulations [5].
First, a set of pedagogical goals are specifically supported through implicit-scaffolding in the design of the simulation [6].These pedagogical goals focus on enabling students to:  Engage in scientific exploration -posing questions, designing experiments, using evidence, etc.  Develop conceptual understandingwith models, cause-effect relationships and representations. Make connections to everyday life. View science as accessible and enjoyable. Take ownership of the learning experience.
Second, barriers to using the simulations are minimized by attending to important accessibility goals.Accessibility is addressed by making the simulations:  Available for free, across platforms and offline. Designed to be flexible tools which can be integrated into a variety of settings or instructional approaches. Able to address a variety of learning goals, including content, process, and affective goals.
Existing data shows that the PhET simulations have been successful in reaching large numbers of users in college and K12 environments.A total of 130 simulations have been published, with over 45 million runs per year, including usage in all 50 states, at both the K12 and college level.However, this data does not provide insight into how or why educators make use of the simulations, or if their goals align with the pedagogical goals designed into the simulations.
To learn more about the PhET simulation users and their implementation, we conducted a survey of over 1,500 college and high school physics educators across the United States.Our research objectives were to identify (1) who uses PhET simulations, (2) with which populations of students, (3) towards what pedagogical goals, and ( 4) with what instructional approaches.In this paper we report the results from analysis of the survey responses.These findings provide insight into whether or not the simulations are being used to support the intended pedagogical goals, as well as evidence for whether the designs are reaching the intended accessibility goals.

STUDY DESIGN
PhET conducted a broad online survey of K12 and college educators using PhET simulations.The K12 and college surveys were very similar, with questions prompting information on respondents background demographics, PhET simulation likes and dislikes, impressions of the impact of PhET simulations on their students' learning, their level of PhET simulation usage, their student populations, how they typically implement PhET simulations, and the goals they want their students to achieve through use of the PhET simulations [7].The survey included multiple choice and long-answer questions.This paper examines only the multiple-choice, multiple-answer responses.
The survey was disseminated widely through numerous email listserves, PhET's newsletter list, and prominent access links on the PhET website.Responses were collected from 11/2012 through 8/2013, and included completed surveys from 2,780 K12 and 504 college educators (about 65% of surveys started were completed).K12 and college level surveys include responses from instructors at all experience levels and across disciplines (physics, chemistry, math, etc.), including international educators.
No incentive was provided to survey respondents and respondents took approximately 10-20+ minutes to complete the survey.The respondents were selfselected, with those educators who value the use of PhET in their instruction (enough to voluntarily complete the survey) likely being the majority of respondents.Because of this self-selection, it is important to note that the survey responses are not broadly representative: These results provide evidence of the existence of certain types of uses, but not necessarily of the distribution of uses among physics educators as a whole.
For the present analysis, we have filtered the responses to include only high school (HS) and college educators in the United States who use the simulations for physics instruction, resulting in 1,233 HS responses and 276 college responses.Because the PhET project began in physics (in 2002), its use is most established within the physics community, although the use among chemistry educators is growing rapidly.Responses from the physics community provide insight from long-term as well as newer users.

RESULTS
We analyze the survey responses from this HS and college physics educator community to examine four questionswho uses PhET simulations, with which populations of students, towards what pedgogical goals, and with what instructional approaches.
Who uses PhET?Among the 1,233 HS physics and 276 college physics educators who responded to the survey (Table 1), we find a wide range of teaching experience, with representation from new teachers (<5 years of experience) to highly experienced teachers (>20 years).Since PhET simulations became available starting in 2002, these data suggest that the simulations have been adopted by educators with prior established teaching practices as well as those new to the profession, and by educators of different ages.
In addition, we find a broad range in the amount of PhET usage by these teachers (Table 1), with similar distributions among the college and HS teachers.The largest fraction of respondents used 6-10 simulations in their teaching (there are about 90 physics simulations across a range of topics).About 20% of respondents are light PhET users (using 5 or fewer simulations in their teaching) and over 10% are heavy PhET users (using 16 or more simulations).
With which student populations?HS and college educators report using PhET simulations with student populations spanning all achievement levels.While usage of PhET with average-achieving students is most commonly reported (90% of HS and 72% of college educators), these educators also report substantial use with under-achieving students (61% and 53%, respectively) as well as with high-achieving students (88% and 60%, respectively).In addition, 38% of the HS educator respondents (or 472 teachers) report using PhET in classes with over 50% minority, low-income, or free or reduced-lunch students.
At the college level, the responding faculty report using PhET with diverse populationsin terms of college major and level.Reported use with science and math majors is the largest overallwith 74% using PhET in introductory courses and 36% in advanced (sophomore or higher) courses.However, 61% also report using PhET with non-science / non-math majors, and 25% report using PhET with pre-service teachers.
Towards what pedagogical goals? Figure 1 shows educators' response to the multiple choice question "When you use PhET simulations, what goals are you typically trying to help students achieve?", indicating also whether the goal is one they are 'typically' or just 'occasionally' trying to achieve.These data demonstrate that PhET simulations are being used by educators to address a variety of goals and that the most popular goals reported are generally shared across HS and college educatorsincluding to engage in exploration and discovery of ideas, and conduct science inquiry.
The goal "to develop enjoyment or interest in science" is reported as a 'typical goal' for PhET use by 51% of HS and 44% of college educators.Consistent with these educators' goal to support student enjoyment of science, 98% of the HS educators and 95% of college faculty report their students 'like' or 'strongly like' the use of simulations when responding to the question "How do your students respond to the use of PhET simulations in your class?".
With what instructional approaches? Figure 2 summarizes the instructional approaches reported by respondents to the multiple choice question "When you use PhET simulations, in what ways do you typically use them?", distinguishing between a 'typical' or 'occasional' way educators use PhET simulations.While these data indicate that both college and HS educators are implementing PhET simulations in a variety of ways in their classroomsdemonstrations, labs, homework, etc.they suggest some possible important differences in implementation by college and HS educators.The HS respondents report a larger percentage of 'typical use' of PhET simulations in hands-on-sim classroom contexts, such as simulationbased activities or labs where students interact with and control the simulation themselves.In contrast, the college faculty report use of the simulations during lectureas a demonstration or as part of an interactive discussionand in homework as more typical, in comparison to the HS educators.

DISCUSSION
Collectively, the results provide substantial evidence that the PhET simulations are successfully achieving the accessibility goals.Respondents are commonly using PhET simulations with diverse populationsdiverse in ability, in background, in school level, and in interest in STEMand toward a wide variety of pedagogical goals.Across educators, PhET simulations are being implemented in many ways to throughout their courses.In addition, we find a diversity among the educator users themselves, with new and experienced educators making use of the simulations.A low barrier to trying an RBIS has been shown to be an important factor in adoption, these survey results provide evidence that PhET's pick-andchoose model of use is resulting in educator uptake, with some teachers using only a few simulations and others making a more substantial curricular investment in using many PhET simulations.The evidence for achievement of PhET's pedagogical goals are mixed.The results provide evidence that some teachers are intending to use the PhET simulations towards the same pedagogical goals the simulation designs attempt to supportwith the most substantial alignment between PhET and educators being 'development of conceptual understanding'.
One of the most central pedagogical goals of PhET simulations is to actively engage students in productive scientific exploration.As seen in Figure 1, a significant fraction of both college (65%) and HS (52%) educators report sharing this as a 'typical' goal.To most deeply engage students in scientific exploration or science inquiry through a simulation requires opportunities for hands-on-sim experiences for studentswhere the students are exploring, asking questions, sense making, discovering relationships, etc.However, when examining the instructional approaches used with PhET, the responding college faculty reported much lower 'typical' uses of PhET in in-class activities or labsapproaches that would provide students with opportunities for scientific exploration with the simulation.The reason for this mismatch between college faculty goals and implementation is unclear.College faculty do assign the simulations as an optional resource at a higher rate than high school teachersand perhaps their intention is for students to engage in independent exploration.

CONCLUSION
Findings from a broad survey of high school and college physics educators provide evidence that PhET simulations are flexible tools used by a broad range of educators, with diverse student populations, to support a wide range of pedagogical goals, and using a variety of instructional approaches.While survey response showed that a significant fraction of these college and high school educators report using PhET simulations towards many of the intended goals, some educators are not, for instance, fully leveraging the simulations for their intended use as tools for engaging students in science exploration and inquiry.Future work is needed to better understand the factors that influence educators' choices for their goals and approaches to simulation use, and to identify approaches to addressing barriers to pedagogically effective use.

FIGURE 2 .
FIGURE 2. Reported instructional approaches used with PhET simulations by high school and college physics teachers.Reports of 'typical way I use PhET' and 'occasional way I use PhET' are represented by darker and lighter shades, respectively.Shown are exact survey phrasing in the college survey; the K12 survey substituted "In Class:" for "In Lecture:" items.

TABLE 1 .
US High School and College Respondents FIGURE 1. Reported pedagogical goals motivating the use of PhET simulations by high school and college physics teachers.Reports of 'typically a goal for my PhET use' and 'occasionally a goal for my PhET use' are represented by darker and lighter shades, respectively.The '*' indicates the principle pedagogical goals considered in the design of most PhET simulations.