home - login - register

PERC 2022 Abstract Detail Page

Previous Page  |  New Search  |  Browse All

Abstract Title: Using IBM’s Watson to automatically evaluate student short answer responses
Abstract Type: Contributed Poster Presentation
Abstract: Recent advancements in natural language processing (NLP) have generated interest in using computers to assist in the coding and analysis of students' short answer responses for PER or classroom applications. We train a state-of-the-art NLP, IBM's Watson, and test its agreement with humans in three varying experimental cases. By exploring these cases, we begin to understand how Watson behaves with ideal and more realistic data, across different levels of training, and across different types of categorization tasks. We find that Watson's self-reported confidence for categorizing samples is reasonably well-aligned with its accuracy, although this can be impacted by features of the data being analyzed. Based on these results, we discuss implications and suggest potential applications of this technology to education research.
Session Time: Poster Session 3
Poster Number: III-67
Contributed Paper Record: Contributed Paper Information
Contributed Paper Download: Download Contributed Paper

Author/Organizer Information

Primary Contact: Jennifer Campbell
University of Illinois Urbana-Champaign
Urbana, IL 61801
and Co-Presenter(s)
Katie Ansell, University of Illinois Urbana-Champaign
Tim Stelzer, University of Illinois Urbana-Champaign

Contributed Poster

Contributed Poster: Download the Contributed Poster