PERC 2022 Abstract Detail Page
Previous Page | New Search | Browse All
| Abstract Title: | Using IBM’s Watson to automatically evaluate student short answer responses |
|---|---|
| Abstract Type: | Contributed Poster Presentation |
| Abstract: | Recent advancements in natural language processing (NLP) have generated interest in using computers to assist in the coding and analysis of students' short answer responses for PER or classroom applications. We train a state-of-the-art NLP, IBM's Watson, and test its agreement with humans in three varying experimental cases. By exploring these cases, we begin to understand how Watson behaves with ideal and more realistic data, across different levels of training, and across different types of categorization tasks. We find that Watson's self-reported confidence for categorizing samples is reasonably well-aligned with its accuracy, although this can be impacted by features of the data being analyzed. Based on these results, we discuss implications and suggest potential applications of this technology to education research. |
| Session Time: | Poster Session 3 |
| Poster Number: | III-67 |
| Contributed Paper Record: | Contributed Paper Information |
| Contributed Paper Download: | Download Contributed Paper |
Author/Organizer Information | |
| Primary Contact: |
Jennifer Campbell University of Illinois Urbana-Champaign Urbana, IL 61801 |
| Co-Author(s) and Co-Presenter(s) |
Katie Ansell, University of Illinois Urbana-Champaign Tim Stelzer, University of Illinois Urbana-Champaign |
Contributed Poster | |
| Contributed Poster: | Download the Contributed Poster |
