To characterize the evolution of student understanding better than what is possible by pre-and post-testing, we posed simple conceptual questions several times per week to separate, randomly selected groups of introductory physics students. This design avoids issues of retesting and allows for tracking of student understanding of a given topic during the course with a resolution on the order of days. Based on the data from over 1600 students over five quarters, we found significant and interesting changes in performance on time scales of days and weeks. We found that the response curves of performance versus time can be divided into four categories: flat (no change), step-up, step-up and decay, and step-down. We examined changes on a 1 day time scale and found that changes in performance did not coincide with relevant traditional lectures or laboratories, but coincide with due dates of relevant on-line homework that provides immediate feedback. This method is well suited to measuring the effect of particular instructional events as well as providing insight into the nature of student understanding and learning.