PERC 2024 Abstract Detail Page
Previous Page | New Search | Browse All
| Abstract Title: | Performance of freely available vision-capable chatbots on the test of understanding graphs in kinematics |
|---|---|
| Abstract Type: | Contributed Poster Presentation |
| Abstract: | In this paper, we evaluate the performance of three freely available vision-capable chatbots – Copilot, Gemini, and Claude 3 Sonnet – on the Test of Understanding Graphs in Kinematics (TUG-K). Our analysis highlights a performance gap between these freely available chatbots and the state-of-the-art, subscription-based ChatGPT-4. We also report largely unclear patterns of performance of the tested chatbots on different types of tasks. We discuss the implications of our findings for using chatbots in educational contexts, point out potential challenges for educational equity, and provide some ideas for future research that could help us better understand the patterns in the chatbots' performance on tasks that involve the interpretation of graphical input. |
| Session Time: | Poster Session 2 |
| Poster Number: | B91 |
| Contributed Paper Record: | Contributed Paper Information |
| Contributed Paper Download: | Download Contributed Paper |
Author/Organizer Information | |
| Primary Contact: |
Giulia Polverini Uppsala University Uppsala, Sweden 756 44 Phone: 0793536805 |
| Co-Author(s) and Co-Presenter(s) |
Bor Gregorcic, Uppsala University |
