PERC 2024 Abstract Detail Page
Previous Page | New Search | Browse All
| Abstract Title: | PhysicsLlama: Bridging the Educational Divide with a Contextualized Large Language Model |
|---|---|
| Abstract Type: | Symposium Talk |
| Abstract: | Large Language Models (LLMs) are adept at generating high-quality texts for various queries. Despite their linguistic prowess, these models often need to catch up when responding to scientific queries, particularly those in physics education that demand students' scientific reasoning and deep contextual understanding. Such limitations render them less effective for tasks requiring higher-order physics problem-solving skills, thus diminishing their utility in educational contexts where students seek assistance with physics problems. To address these challenges, we develop PhysicalLlama, a novel contextualized foundation LLM pre-train on an extensive dataset compiled from over 1000 high-impact physics education articles and thousands of student written responses in physics. This innovative approach equips PhysicalLlama with a rich factual and procedural knowledge foundation specifically tailored to address the intricate demands of physics education. To evaluate PhysicalLlama's performance rigorously, we fine-tune the model on student-written responses associated with ten physics constructed response assessment items. This fine-tuning process is designed to enhance further the model's understanding and responsiveness to the nuances of physics education. Subsequently, we tested PhysicalLlama against a diverse set of 2,000 student responses, examining its performance with and without prior fine-tuning. This comparative analysis extends to its performance against two commercial LLMs (GPT-4 and Claude 3) and one public LLM (Mistreal 7B-instruct), offering a comprehensive perspective on PhysicalLlama's capabilities relative to the current state-of-the-art. Our findings will shed light on PhysicalLlama's potential as a transformative tool in physics education. By bridging the significant institutional gaps in this field, PhysicalLlama stands poised to redefine the landscape of educational assistance in physics, offering students unprecedented support and insight. |
| Parallel Session: | Applications, Opportunities and Challenges of Large Language Models in Physics Education |
Author/Organizer Information | |
| Primary Contact: |
Ehsan Latif University of Georgia |
| Co-Author(s) and Co-Presenter(s) |
Xiaoming Zhai |
