Classroom response systems can be powerful tools for teaching physics. Their efficacy depends strongly on the quality of the questions. Creating effective questions is difficult and differs from creating exam and homework problems. Each classroom response system question should have an explicit pedagogic purpose consisting of a content goal, a process goal, and a metacognitive goal. Questions can be designed to fulfill their purpose through four complementary mechanisms: directing students' attention, stimulating specific cognitive processes, communicating information to the instructor and students via classroom response system-tabulated answer counts, and facilitating the articulation and confrontation of ideas. We identify several tactics that are useful for designing potent questions and present four "makeovers" to show how these tactics can be used to convert traditional physics questions into more powerful questions for a classroom response system.