A new question methodology has been developed and used with voting machines in large physics lecture classrooms. The methodology was tested by comparing student performance in voting machine and non-voting machine lecture sections during three consecutive electricity and magnetism quarters of introductory calculus-based physics. Data from The Conceptual Survey of Electricity and Magnetism and common examination questions indicates that students using voting machines achieved a significant gain in conceptual learning, and that voting machines reduced the gap between male and female student performances on tests. Surveys indicated that students were positive about the use of voting machines and believed that they helped them learn. The surveys also suggested that grading voting machines responses and/or overusing voting machines may lower student enthusiasm.