Physics is one of the STEM fields with the highest attrition rates and a lack of diversity in graduate education. The results of retention rates of underrepresented racial minority (URM) students from the American Physical Society- Bridge Program (APS-BP) indicate a potential correlation exists between experiencing a supportive departmental environment and retention. We examine whether physics graduate URM and non-URM students' self-reports show meaningful differences in supportive components at their programs. We also examine students' self-reports of experiencing supportive departmental components between institutions that are and are not affiliated with the APS-BP. This study was based on physics graduate students' responses to a survey regarding their experiences of departmental structures. In this paper, we use 254 responses from U.S. citizen students enrolled at 20 research-intensive graduate programs. Performing multivariate analysis of variance and calculating effect sizes, we find URM students report experiencing supportive components in their programs at higher levels than non-URM students, except for financial support that includes having covered tuition for the entire program, and being offered health benefits. Looking at other available data in the larger survey to better understand this outcome, we find non-URM students receive graduate assistantships at a higher rate than URM students (89.7% vs. 62.5%, respectively), and URM students are more likely to either obtain a loan or self-support (working outside of campus) their graduate education compared to non-URM students (25% vs. 12%, respectively). Overall, our results suggest that physics graduate programs have practices in place to better support URM students on aspects related to their progress in the program, but are not paying similar attention to their financial needs. Finally, we find the APS-BP affiliated institutions provide more student support than non-affiliated institutions.