The ways in which physics majors make career decisions is a critical, yet understudied, aspect of the undergraduate experience. Such decisions are important to students, physics departments, and administrators. In this project, we specifically examine how students develop interests and intent to pursue specific subfields of physics by interviewing 13 physics majors from all years of study. The interviews examined factors that led students to choose their most preferred and least preferred subfields. Interviews leveraged the framework of Social Cognitive Career Theory, a model that describes how several constructs such as self-efficacy, learning experiences, and outcome expectations relate to decision-making. Findings highlight the differences in decision-making between upper-division students and beginning students. For instance, we see how popular culture and popular science provide an initial learning experience about certain subfields, such as astronomy and astrophysics, which strongly affect beginning students' perceptions of that subfield. Initial exposure to biology and chemistry in high school or early undergraduate classes often negatively affected students' interests in fields like biophysics or chemical physics. Data also suggests a splitting between students with respect to their outcome expectations of a desirable career in science. While some students prioritize using science to help people, others prioritize discovery of new knowledge though science, and some are in between. Students in both groups form perceptions about subfields that do not align with their identities and hence make decisions based on these perceptions. For instance, a student who prioritizes helping others through science may be quick to reject astrophysics as a subfield choice as they do not think that astrophysics can help people enough.