How inclusiveness of learning environment predicts female and male students’ physics grades and motivational beliefs in introductory physics courses

In this study, we adapted a prior identity framework to investigate how students’ perception of the inclusiveness of the learning environment (including sense of belonging, peer interaction and perceived recognition) in an introductory physics course predicts their course grades and physics motivational beliefs (including self-efficacy, interest and identity) at the end of this course. We found signatures of inequitable and non-inclusive learning environment in that female students’ mean scores for sense of belonging, peer interaction and perceived recognition were all lower than male students’ in the course. In addition, we found that female students had lower average course grades than male students. Using structural equation modeling, we found that students’ perception of the inclusiveness of the learning environment predicts their self-efficacy, interest, identity and grades at the end of the course even after controlling for students’ gender, motivational beliefs and grades in a previous course as well as their high school GPA and SAT math scores. In particular, students’ perceived recognition, e.g., by instructors and teaching assistants, played a major role in predicting students’ physics identity, and students’ sense of belonging in physics played an important role in explaining the change in students’ physics self-efficacy. Our findings can be helpful for creating an inclusive and equitable learning environment in which all students can excel.

In physics, researchers have found that self-efficacy is an important motivational characteristic of students in order to excel [4,5,7,8]. Self-efficacy is the belief in one's capability to be successful in a particular task, course, or subject area [30,31], and it has been shown to influence students' engagement and performance in a given domain [15][16][17]32]. Another important motivational characteristic of students is interest, which is defined by positive emotions accompanied by curiosity and engagement in a particular discipline [33,34]. According to Eccles's expectancy-value theory [35,36], interest is paired well with self-efficacy as connected constructs that predict students' academic outcomes and career aspirations. In addition, science or physics identity is another motivational characteristic that can influence students' career decisions and outcome expectations [1][2][3][37][38][39][40][41]. Students' physics identity is related to whether they see themselves as a physics person [1][2][3]37,38,41].
To achieve equity in learning, not only should all students have equitable opportunities and access to resources, they should also have an equitable and inclusive learning environment with appropriate support and mentoring so that they can engage in learning in a meaningful and enjoyable manner and the learning outcomes should be equitable. By equitable learning outcomes, we mean that students from all demographic groups (e.g., regardless of their gender identity or race/ethnicity) who have the prerequisites to enroll in the course have comparable learning outcomes. The STEM learning outcomes include student performance in courses and their motivational beliefs because regardless of the performance, the motivational beliefs can influence students' short and long-term retention in STEM disciplines.
Unfortunately, studies have shown that there are gender disparities in both motivational beliefs and performance in some STEM disciplines, e.g., physics [2,4,5,7,[10][11][12]42]. Therefore, investigating the factors that influence physics motivational beliefs and grades in introductory physics courses taken by physical science and engineering majors can play an important role in understanding women's underrepresentation in those disciplines. Here we describe a study focusing on how the perception of the inclusiveness of the learning environment predicts students' physics selfefficacy, interest, identity and grades in a calculus-based introductory physics course. Our findings can be useful for developing an inclusive and equitable learning environment.

II. BACKGROUND AND FRAMEWORK
The well-known science identity framework by Carlone and Johnson [1] includes three dimensions: competence ("I think I can"), performance ("I am able to do"), and recognition ("I am recognized by others"). In more recent studies, interest was added to this picture and the relation between gender and physics identity was mediated by interest, competency belief, and perceived recognition [43,44]. These studies reveal that individuals' identity in science is impacted by their perceived recognition from others.
Similarly, students' self-efficacy and interest have also been found to be influenced by their interaction with others [31,34]. According to Bandura's social cognitive theory, one factor that contributes to the development of self-efficacy is social persuasion experiences [31]. In Hidi and Renninger's four stages model of interest development [34,45,46], people's interest in a discipline is triggered and maintained by external factors first, but then it becomes an individual interest and finally becomes a well-developed interest. In prior work [47], students' perceived recognition is not only the strongest predictor of identity, it also predicts self-efficacy and interest.
In addition to perceived recognition, some studies have shown that students' interactions with peers and students' sense of belonging are also important aspects of their perceptions of the inclusiveness of the learning environment [6,[48][49][50][51][52][53][54][55][56]. For example, if students have a higher sense of belonging, they may approach others in the academic environment more often and with more positive attitudes, building better interactions with others and reporting higher perceived recognition from others [57]. However, there are few quantitative studies about the effect of inclusiveness of the learning environment on students' physics motivational beliefs and academic performance and the roles played by each component of the inclusiveness of the learning environment. Thus, to better understand how the inclusiveness of the learning environment influences students learning outcomes and how to foster an inclusive and equitable learning environment, further study is needed.
In this study, we investigated how students' perception of the inclusiveness of the learning environment (including perceived recognition, peer interaction and sense of belonging) in an introductory physics course (physics 2) predicts their self-efficacy, interest, identity and grades at the end of this course by controlling for students' gender, high school GPA and SAT math as well as their self-efficacy, interest and grades in a previous introductory physics course (physics 1). As shown in Fig. 1, the thirteen constructs are divided into three groups: what we control for, perception of the inclusiveness of learning environment, and outcomes. Students' gender, SAT math, high school GPA (HS GPA), and their self-efficacy, interest and grades in physics 1 (SE 1, Interest 1, and Grade1) are constructs that we control for. Outcomes include students' self-efficacy, interest, identity and grades in physics 2 (SE 2, Interest 2, and Grade 2). Perceived recognition (Perceived Recog), peer interaction (Peer Int) and sense of belonging (Belonging) constitute the perception of the inclusiveness of the learning environment in physics 2.
In this study we first estimated gender differences in student's motivational beliefs and grades in both physics 1 and physics 2. Then, we used structural equation modeling (SEM) to investigate how the perception of the inclusiveness of the learning environment predicts students' learning outcomes at the end of physics 2, and what role each inclusiveness of the learning environment construct plays. Finally, we estimated how much variance in students' learning outcomes is explained by the model.

III. RESEARCH QUESTIONS
Our research questions regarding the calculus-based introductory physics courses 1 and 2 taken by engineering and physical science majors in their first year of college studies at a large state-related university in the US are as follows: RQ1. Are there gender differences in students' academic performance and motivational characteristics and do they change from physics 1 to physics 2?
RQ2 How do the components of the perception of the inclusiveness of the learning environment (including sense of belong, peer interaction and perceived recognition) predict students' self-efficacy, interest, identity, and grades in physics 2, controlling for students' gender, high school GPA, SAT math, and their self-efficacy, interest and grades in physics 1?
RQ3. If gender does not moderate any path in the model shown in Fig. 1, how does gender mediate (a) the factors that were controlled for, (b) the perception of the inclusiveness of the learning environment after controlling for students' high school GPA, SAT math, and their self-efficacy, interest and grades in physics 1, and (c) the learning outcomes after controlling for everything else in the model in Fig. 1?

IV.METHODOLOGY
In this study, we collected the motivational survey data at the end of each course in a two-term college calculus-based introductory physics sequence (including physics 1 and physics 2) in two consecutive school years. These courses are taken mostly by students majoring in engineering, physical sciences, and mathematics. The paper surveys were handed out and collected by TAs during the last recitation class of a semester. The demographic data of students-such as gender, ethnicity or race-were provided by the university. Students' names and IDs were de-identified by an honest broker, so researchers could analyze students' data without having access to students' identifying information. There were 697 students participating in the survey in both physics 1 and physics 2 (233 female students and 434 male students). We recognize that gender identity is not binary. However, because students' gender information was collected by the university, which offered binary options, we did the analysis with the binary gender data available.
In this study, we considered female and male students' physics self-efficacy, interest, peer interaction, perceived recognition, sense of belonging, and identity. In particular, students' self-efficacy and interest in physics 1 and physics 2 were measured at the end of each course, and their perceived recognition, peer interaction, sense of belonging and identity were measured at the end of physics 2. This is because only after the course can students answer these survey questions based on their real experience in the course such as their interaction with peers, TAs and instructors. The survey questions were adapted from the prior motivational surveys [43,44,[58][59][60][61] and were revalidated in our prior work [7,47]. The validation and refinement of the survey involved use of one-on-one student interviews with students, exploratory and confirmatory factor analyses (EFA and CFA), Pearson correlation between factors and Cronbach alpha [62].
In our survey, we had four items for self-efficacy (Cronbach alpha = 0.79 for self-efficacy in physics 1 and Cronbach alpha = 0.81 for self-efficacy in physics 2 with Cronbach alpha results over 0.70 indicating acceptable internal consistency [62]). Students had four options on a Likert scale for each item which corresponded to 1 to 4 points. We also had four items for interest; each item involved a four-point Likert scale (Cronbach alpha = 0.82 for interest in physics 1, Cronbach alpha = 0.84 for interest in physics 2). Physics identity corresponds to students' belief about whether they see themselves as a physics person [3]. Students could choose from strongly disagree, disagree, agree, and strongly agree and they corresponded to 1 to 4 points [63], respectively.
In addition, perceived recognition, peer interaction and sense of belonging are the perception of the inclusiveness of the learning environment constructs. Unlike self-efficacy, interest and identity, these three constructs are directly related to students' experience in the course. Perceived recognition included three items which represent whether a student thinks other people see them as a physics person [2,3,37] (Cronbach alpha = 0.86). Peer interaction includes four items and represents whether students have a productive experience when working with peers (Cronbach alpha = 0.92). Sense of belonging is about students' feelings of whether they thought they belonged in the physics class [50], and it included five items that each had a 5-point Likert scale: "not at all true, a little true, somewhat true, mostly true and completely true" (Cronbach alpha = 0.87).
First, we calculated the mean score for each construct for each student. Then we used a t-test [64,65] to compare students' self-efficacy and interest in physics 1 and physics 2 and to compare responses for female and male students. Finally, we used Structural Equation Modeling (SEM) [66] to analyze predictive relationships among the constructs. The SEM includes two parts: CFA and path analysis.
To validate the items on our survey, we performed the CFA for each construct. The model fit is good if the fit parameters are above threshold. In CFA, Comparative Fit Index (CFI) > 0.9, Tucker-Lewis Index (TLI) > 0.9, Root Mean Square Error of Approximation (RMSEA) < 0.08 and Standardized Root Mean Square Residual (SRMR) < 0.08 are considered as acceptable and RMSEA < 0.06 and SRMR < 0.06 are considered as a good fit [67]. In our study, CFI = 0.933, TLI = 0.918, RMSEA = 0.054 and SRMR = 0.038, which represents a good fit. Thus, there is additional quantitative support for dividing the constructs as proposed. In addition, the CFA results show that all of the item loadings are above 0.5 and most of them are above 0.7, which means that our constructs extract sufficient variance from the items.
To analyze the relations among the constructs, we performed the full SEM. Apart from CFA, SEM gives regression coefficients b for paths between each pair of constructs and the value of each b is a measure of the strength of that relationship. Compared with a multiple regression model, the advantage of SEM is that we can estimate all the regression links for multiple outcomes and factor loadings for items through CFA simultaneously [67]. Before performing the gender mediation models, we first tested the gender moderation relations between each pair of constructs using multi-group SEM. Results showed that in our model, strong measurement invariance holds and there is no difference in any regression coefficients by gender, which allowed us to perform the gender mediation analysis using SEM [67]. After performing the full SEM, we analyzed the role played by each inclusiveness of the learning environment construct in predicting students' learning outcomes and how much variance in each outcome construct is explained by the model.

V. RESULTS AND DISCUSSION
In response to RQ1, we find that female students had significantly lower scores in all six motivational constructs with medium effect sizes (as shown in Tables I and II) [65]. In particular, female students' average scores pertaining to perceived recognition and physics identity indicate that on average, female students did not think others see them as a physics person, and they did not see themselves as a physics person either.
When we compared students' self-efficacy and interest in the two courses, we found that both male and female students' self-efficacy and interest decreased from physics 1 to physics 2; however, the effect sizes are relatively small compared with the effect sizes of the gender differences in these constructs. Table III shows students' high school GPA, SAT math scores, and grades in physics 1 and physics 2. As we can see, female students had significantly lower grades than male students in both physics 1 and physics 2, while there was no statistically significant gender difference in SAT math scores, and female students even had a higher average high school GPA than male students. Although high school GPA includes many courses and may have issues such as grade inflation, our results show that female students have comparable overall high school preparation as male students.

Table I.
Descriptive statistics for physics self-efficacy and interest in physics 1 and physics 2 by gender. Table II. Descriptive statistics for perceived recognition, peer interaction, belonging and identity in physics by gender. Table III. Descriptive statistics of students' high school GPA, SAT math, and grades in physics 1 and physics 2.
To answer questions RQ2 and RQ3, we ran the full SEM model in which perceived recognition, peer interaction and sense of belonging constitute the perception of the In response to RQ2, we find that students' perception of the inclusiveness of the learning environment predicts their selfefficacy and interest at the end of physics 2 even after controlling for their self-efficacy and interest at the end of physics 1. According to Fig. 2, sense of belonging is the largest predictor of self-efficacy. Although identity is predicted by self-efficacy, interest and perceived recognition, perceived recognition, which is the only inclusiveness of the environmental factor of these three, is the largest predictor.
Regarding RQ3, we find that although there were large gender differences in students' self-efficacy, interest and identity in physics 2, Fig. 2 shows that gender does not directly predict these constructs. Thus, Fig. 2 reveals that the gender differences in students' self-efficacy, interest and identity in physics 2 shown in Tables I and II were mediated by the different components of the perception of the inclusiveness of the learning environment. To further understand how much variance in students' learning outcomes is explained by our model, we calculated the coefficient of determination ! (fraction of variance explained) for each of the four outcome constructs. According to our results, the ! value of Grade 2 is 0.40, which means that the model explains 40% of the variance in Grade 2. For the motivational outcomes, there is 85% of the variance in SE 2, 82% of the variance in Interest 2, and 72% of the variance in Identity explained by the model. Thus, our model explains students' learning outcomes reasonably well.

VI. SUMMARY AND CONCLUSION
In this study, we included peer interaction, perceived recognition, and sense of belonging as three components of students' perception of the inclusiveness of learning environment and found that they predicted students' physics identity, self-efficacy, interest and grades in an introductory physics course after controlling for students' gender, motivational beliefs and grades in a previous course as well as their high school GPA and SAT math scores. We found signatures of inequitable and non-inclusive learning environment in that female students' mean scores for sense of belonging, peer interaction and perceived recognition were all lower than male students' in the course, which could contribute to the gender disparities in students' learning outcomes shown in the results. In particular, we found female students had lower grades than male students even though there was no significant gender difference in SAT math scores and female students actually had a higher average high school GPA than male students. In addition, we found that although there were large gender differences in students' self-efficacy, interest, identity and grades in physics 2, gender does not directly predict these outcomes. This means that the gender differences in these constructs were mediated by the perception of the inclusiveness of the learning environment.
Thus, in addition to being driven by prior differences, students' learning outcomes are also influenced by their perception of the inclusiveness of the learning environment. These findings suggest that an inclusive and equitable learning environment can help improve students' performance and motivational beliefs in physics especially for students who already had lower motivational beliefs, e.g., because of societal stereotypes and biases about who can excel in physics. Some social-psychological interventions such as ecological-belonging intervention focusing on reducing students' stereotype threats have been shown to reduce gender gaps in students' performance [68,69]. Thus, as instructors, we need to keep in mind how the societal stereotypes and biases in physics impact the stereotyped groups and take the responsibility to develop an inclusive and equitable learning environment in which students from all demographic groups, e.g., women and men, have opportunities and access to resources, feel recognized and affirmed, and have comparable learning outcomes.

ACKNOWLEDGEMENT
This work was supported by NSF award DUE-1524575.