PERC 2025 Abstract Detail Page
Previous Page | New Search | Browse All
| Abstract Title: | Analyzing Undergraduate Problem-Solving in Physics Through Interaction With an AI Chatbot |
|---|---|
| Abstract Type: | Contributed Poster Presentation |
| Abstract: | Providing individualized scaffolding for physics problem solving at scale remains an instructional challenge. We investigate (1) students' perceptions of a Socratic AI chatbot's impact on problem‐solving skills and confidence and (2) how the specificity of students' questions during tutoring relates to performance. We deployed a custom Socratic AI chatbot in a large‐enrollment introductory mechanics course at a Midwestern public university, logging full dialogue transcripts from 150 first‐year STEM majors. Post‐interaction surveys revealed median ratings of 4.2/5 for knowledge‐based skills and 4.0/5 for overall effectiveness. Transcript analysis showed question specificity rose from approximately 10–15% in the first turn to 75 % by the final turn, and specificity correlated positively with self-reported expected course grade (Pearson r = 0.43). These findings demonstrate that AI-driven Socratic dialogue not only fosters expert-like reasoning but also generates fine-grained analytics for physics education research, establishing a scalable dual-purpose tool for instruction and learning analytics. |
| Footnote: | This work is supported in part by U.S. National Science Foundation grant 23000645. Opinions expressed are those of the authors and not of the Foundation. |
| Session Time: | Poster Session C |
| Poster Number: | C-8 |
| Contributed Paper Record: | Contributed Paper Information |
| Contributed Paper Download: | Download Contributed Paper |
Author/Organizer Information | |
| Primary Contact: |
Syed Furqan Abbas Hashmi Purdue University West Lafayette, IN 47906 Phone: 7657469547 |
| Co-Author(s) and Co-Presenter(s) |
N. Sanjay Rebello, Purdue University |
Contributed Poster | |
| Contributed Poster: | Download the Contributed Poster |
