Social network analysis of a physics faculty online learning community

We analyze the results of two surveys administered to a Faculty Online Learning Community teaching a common physics curriculum designed primarily for pre-service elementary teachers. We use Social Network Analysis to represent the faculty network and compare members’ closeness, a measure of how closely connected a person is with every other person in their network, to their reported experience in the community. We find that participants’ self-efficacy, as well as their teaching and sense of benefitting from the community, are predictors of their centrality in the network as measured by closeness with other participants


I. INTRODUCTION
A community of practice is a group of people with a common interest that come together to fulfill both individual and group goals in a spirit of learning, knowledge generation and sharing, and collaboration [1]. Participation takes place at different levels, including legitimate peripheral participation, and learning is framed as moving toward core participation [2]. Though such communities exist for all kinds of practices, this study will focus on a group of educators participating in a Faculty Learning Community (FLC).
An FLC is typically a group of about ten faculty from various disciplines that meets regularly to discuss their work and learn from each other with the goal of professional development [3]. More recently, faculty online learning communities (FOLCs) have been used as geographically distributed, discipline-specific FLCs that serve to support and provide resources for faculty implementing Research-Based Instructional Strategies (RBIS) to enhance student learning. FOLCs have been shown to be effective means of professional development for faculty, especially in terms of transforming instruction through the adoption of Research-Based Instructional Strategies [4,5].
In this analysis, we will represent the network that exists between members of a FOLC based on how they describe their connections with other members in response to a survey. By considering this network along with their self-described impacts resulting from participation in the FOLC, we hope to answer the following research question: What relationships, if any, exist between participants' place in the network and the impacts they report from participating in the community?
Therefore, we will look for relationships between participants' centrality and the way they responded to questions on the final survey. We believe that our results will help the designers of future FOLCs decide how to use time and resources to prioritize certain impacts for participants.

II. BACKGROUND & THEORY
A. Next Gen PET FOLC The Next Generation Physics and Everyday Thinking (Next Gen PET) FOLC is a community of practice for physics faculty using a common curriculum (Next Gen PET) to teach physics and physical science to pre-service elementary teachers [4].
Next Gen PET is a guided-inquiry, physical science curriculum where students use data to explore phenomena and develop scientific concepts. The instructor acts as a facilitator, which requires a significant shift if they are used to instructor-centered, lecture-based practice; it is expected that adopters will need considerable pedagogical skill at facilitating guided-inquiry learning [6,7]. The Next Gen PET FOLC was designed to support faculty in implementing the curriculum.
The FOLC began in 2017 and initially consisted of about 40 physics faculty that taught physics or physical science courses for pre-service elementary teachers. It was led by 10 physics faculty that were recruited and trained for leadership because they had experience using Next Gen PET or one of its predecessors. Participants were grouped in clusters of about 5-13 people and would convene one or two times a month online via videoconference for at least an hour, with two to three leaders facilitating each cluster meeting. The FOLC operated for four academic years with about 50 active participants (including participants and cluster facilitators) at any time.
Participants were grouped in different clusters over the course of their participation in the FOLC based on schedule availability. They were also able to participate in programming beyond the cluster meetings, like a FOLC-wide virtual conference and a Slack space with channels for clusters, cluster facilitators, and the entire FOLC. Because of this, the average FOLC member would have had many opportunities to interact with most, if not all, other FOLC members.

B. Social Network Analysis
Social Network Analysis (SNA) was developed from elements of network theory, primarily by social scientists, for use in social science research [8]. It includes using quantitative methods to characterize a group of people by analyzing the connections between the people, certain qualities of those people, and sometimes by the qualities of the connections themselves. With it, we can describe characteristics of individuals by examining their place within the larger network structure, and we can describe characteristics of the network itself [9].
SNA incorporates elements of graph theory, particularly in representing networks visually. Graphical representations of social networks, often referred to as simply graphs, feature nodes (or people) as solid shapes, and edges (or connections) as lines connecting them [9].
SNA has been used previously in education research and physics education research [10] to investigate gender and ethnicity as they relate to students' position in a network [11], students' social interactions in laboratory contexts [12], and students' persistence in physics as it relates to their position in a network [13], to name but a few examples.

A. Data Collection
The data for this analysis come from two surveys: a "final survey" and an "SNA survey." The surveys were administered to current and former members of the FOLC by the project's evaluation team in May 2021 and June 2021. Both surveys were sent to all 61 current and former FOLC members, and of these, 44 (72 percent) and 42 (69 percent) completed the final and SNA surveys, respectively.
The final survey asked respondents to reflect on their experience in the FOLC in the previous year and on their entire experience in the FOLC, since it was coming at the end of the project. The survey included 10 Likert-scale questions that sought to measure the extent and types of impacts the respondents experienced. For example, participants were asked: To what extent has each of the following happened as a result of participating in the Next Gen PET FOLC? [not at all, minimally, moderately, to a great extent] • I have gained confidence in my teaching.
• I have gained knowledge about pedagogical techniques. • I have saved time in preparing and implementing my course. The purpose of the SNA survey was to record self-reported connections between FOLC members after joining the FOLC. The survey asked respondents whether they "had conversations with" or "collaborated with" every other current and former FOLC member after joining the FOLC. Respondents saw a list of names of every other FOLC participant and were able to check boxes describing their connections.

Data Preparation & Network Development
We administered the SNA survey electronically and organized the raw data into an edgelist, with columns representing the first node (the survey respondent), the second node (the person with whom they are indicating a connection) and a variable representing whether that connection was a conversation or collaboration. Since the survey asked respondents about two different types of social connections, we split the data into a conversation network and a collaboration network. We treated the networks as undirected, meaning that a connection is represented and treated the same regardless of which node(s) indicated the connection.
Then, we cleaned the connection data so that only people that completed both the SNA survey and the final survey would be included in the networks. This means that the networks do not include any connections where at least one member did not complete either survey, so our networks are limited representations of the actual networks that exist among FOLC members. We removed 413 of the original 1202 connections, leaving 66 percent of the connections from the raw data.
By cleaning the data such that only members that completed both surveys were included in the networks, we can ensure that every node being analyzed had equal opportunity to be included in the network, both as a reporter and as someone that could be reported in a connection. We were also able to ensure that it would be possible to compare all nodes according to their survey responses.
We also removed data from respondents that completed very different versions of the final survey than most other respondents. Since the survey went to all members past and present, some former participants skipped all questions related to their experience in the FOLC in the past year, which comprised many of the responses we examined. In total, we removed 7 nodes in the conversation network and 6 nodes in the collaboration network, leaving 37 and 34 nodes in these networks, respectively.

Attribute and Composite Definitions
After cleaning and splitting the connection data, we used an R [14] package called igraph [15] to turn the edgelists into graph objects, allowing us to generate lists of nodes in the networks and add attributes for those nodes. In this case, the attributes include the answers to each question on the final survey that could be captured numerically.
The final survey consisted of nearly 20 questions. Some were free-response, and thus not included as attributes for the network, and many had multiple parts, sometimes over 10.
Treating each numerical (Likert scale) response as a separate attribute would mean nearly 70 attributes per person, so we developed composite variables of question parts to make the attributes more manageable.
The designed composites, which we will also refer to as impact composites (ICs) were developed concurrently with the final survey and aim to group together items that address similar topics. For example, one question contained 10 subquestions that asked respondents to rate their agreement with various statements about impacts on teaching after participating in the FOLC. These sub-questions form the "Teaching impacts" composite, so a respondent's average numerical response to these sub-questions is recorded as a single attribute for their "Teaching impacts" composite.
We measured the reliability of the ICs by calculating Cronbach's alpha for each of the related items of each IC. Cronbach's alpha is a coefficient ranging from 0 to 1 that indicates the degree to which different variables seem to be measuring the same thing [16]. Thus, a higher coefficient indicates more reliable ICs. The coefficients associated with all of our ICs were near or exceeding 0.8, so we conclude that the designed composites are reliable and reasonable ways to group the items.
Descriptions of the ICs and some examples of their related questions can be found in Table I.

Statistical Methods
A primary goal of this analysis is to look for relationships between FOLC participants' centrality and the way they responded to questions on the final survey. Centrality is a family of measures that capture the prominence of individual nodes within a network. There are many ways to measure centrality, though some of the most common ones are degree, betweenness, and closeness [17].
Degree is the total number of connections a node has. Betweenness is the number of shortest paths between a pair of other nodes that pass through the node in question. Closeness is a node's average distance to other nodes in the network, as measured by the smallest number of edges connecting them. Therefore, it may be seen as a measure of how closely connected a person is with every other person in their network [17]. We use closeness as a measure of centrality in this analysis and will show in subsequent analyses why we believe it is a reasonable measure of centrality to use. Social networks violate the most basic assumption of frequentist statistics-that the data are independent. In a social network, centrality measures like closeness are necessarily interdependent, so statistical methods that do not assume independence of data are useful, like Bayesian methods [18].
Using Bayesian methods, we seek to establish the belief in the validity of a given hypothesis. More specifically, we revise our beliefs about the validity of a particular hypothesis (or multiple hypotheses) according to data [19]. Here, we use Bayesian regression to compare our beliefs in the validity of multiple models (hypotheses) which suggest that participants' closeness is predicted by any combination of ICs.
We compared the intercept-only model, that is, the model that suggests that the closeness of one node is best predicted by the average closeness of all nodes, to models that suggest that a node's closeness is best predicted by some combination of its ICs, where all possible combinations of anywhere from 1 to 10 of the 10 ICs can comprise a combination (there are 1023 such combinations).
We calculated Bayes factors for each model to understand how well that model predicts closeness compared to a model which uses average closeness as the predictor. [19].

IV. RESULTS
Representations of the conversation and collaboration networks can be found in Fig. 1. The nodes are arranged according to the Fruchterman-Reingold algorithm, which treats nodes as like charges with a repulsive force between them and edges as spring-like attractors. The algorithm finds an arrangement that minimizes the energy in the system, which usually means that nodes with more connections are closer to the center [20]. We note that, although facilitators and participants had different responsibilities, they engaged in the community in similar ways, as reflected by similar average closeness values of 0.00600 and 0.00544, respectively.
Our primary result concerns the relationship between cen-trality and final survey responses. Bayesian regression that used closeness as the centrality measure with which to compare models suggested that a model of the Self-efficacy IC was over eight times more likely to predict a node's closeness in the conversation network than a model of the average closeness only. It also suggested that the combination of 'Teaching impacts', 'Community benefits', and 'Selfefficacy' together was five times more likely to predict closeness than average closeness only. These results and others are summarized in Table II. For the collaboration network, the same combination of 'Teaching impacts', 'Community benefits', and 'Self-efficacy' together was 20 times as likely to predict closeness as the average closeness model, though SE alone as a predictor was not as significant as it was in the conversation network. Other models that were significantly (at least 10 times) more likely to predict closeness included 4 or more ICs and are summarized in Table III.
Other studies have suggested that a Bayes factor between 3 and 20 constitutes "positive evidence [19,21]." Therefore, there is positive evidence that the above ICs, mainly 'Selfefficacy', followed by 'Teaching impacts' and 'Community benefits', are predictors of centrality as measured by closeness. In other words, we have evidence that a FOLC participant that is better-connected with other FOLC participants FIG. 1. The Next Gen PET FOLC conversation (left) and collaboration network (right), arranged using the Fruchterman-Reingold algorithm. In these graphs, node size represents degree, or total number of connections a node has, and color represents a node's role as participant or facilitator in the FOLC. is more likely to report higher impacts related to their selfefficacy, teaching, and valuation of their community after participating in the FOLC. We reserve discussion of the implications of Bayes factor differences for future work.

V. DISCUSSION & CONCLUSION
Results of a Bayesian regression suggest that 'Selfefficacy' is the best predictor of closeness in the conversation network among all ICs and combinations of ICs. 'Teaching impacts' and 'Community benefits' are also good predictors of closeness. For the collaboration network, we see that the same ICs, as well as 'Attitude toward participating in cluster meetings' and 'Sense of community', in various combinations, are good predictors of closeness.
Since closeness is a measure of how well connected a person is with other people in a group, in the case of this FOLC, a person's closeness may relate to the depth and breadth with which they participated socially in the FOLC. Then, our results suggest that participants that are more active in the community see greater impacts from participation in their selfefficacy, teaching, and perhaps other areas. These results may be particularly useful to those organizing similar FLCs-they may choose to design interventions to increase connections among participants if improved self-efficacy and teaching are primary goals. These results also are consistent with the motivation for creating FLCs in the first place-that participation in community-based learning interventions is an effective means of professional development.
It is unclear from our evidence whether the increased impacts are the result of a person's higher closeness or their higher closeness is the result of increased impacts, only that they are related. In other words, it may be that a person with higher self-efficacy becomes closer with others in the FOLC, or alternatively that a person with higher closeness perceives more benefits from the community-we can not say one way or the other.
We note that this analysis is limited by the size of our dataset. The community itself is a relatively small network, and as noted earlier, our dataset does not represent the full FOLC network. This limitation became apparent when we tried to perform a principal component analysis on the full attribute (final survey response) list-we had too few data points and so we used the designed composites to group the attributes instead.
We plan to extend this study qualitatively by analyzing the experiences of a few participants that were especially central or peripheral in the network. In doing so, we hope to obtain a more illustrative picture of what a participant's centrality might suggest about their experience in the FOLC.