Mathematics is a critical part of much scientific research. Physics in particular weaves math extensively into its instruction beginning in high school. Despite much research on the learning of both physics and math, the problem of how to effectively include math in physics in a way that reaches most students remains unsolved. In this paper, we suggest that a fundamental issue has received insufficient exploration: the fact that in science, we don't just use math, we make meaning with it in a different way than mathematicians do. In this reflective essay, we explore math as a language and consider the language of math in physics through the lens of cognitive linguistics. We begin by offering a number of examples that show how the use of math in physics differs from the use of math as typically found in math classes. We then explore basic concepts in cognitive semantics to show how humans make meaning with language in general. The critical elements are the roles of embodied cognition and interpretation in context. Then, we show how a theoretical framework commonly used in physics education research, resources, is coherent with and extends the ideas of cognitive semantics by connecting embodiment to phenomenological primitives and contextual interpretation to the dynamics of meaning-making with conceptual resources, epistemological resources, and affect. We present these ideas with illustrative case studies of students working on physics problems with math and demonstrate the dynamical nature of student reasoning with math in physics. We conclude with some thoughts about the implications for instruction.