home - login - register

PERC 2021 Abstract Detail Page

Previous Page  |  New Search  |  Browse All

Abstract Title: Understanding the Efficacy of Machine Learning for Coding Practices
Abstract: Qualitative data coding is a powerful research tool for analyzing text statements; however, having human researchers manually code can be time-consuming and hindering to a project with a large data set. Modern machine learning programs have advanced such that they currently play large roles in automating data categorization and may be useable as a method of semi-automating categorization in PER: human coders can prepare a moderate sample of the data for training, and machine learning software can be left to sort the remaining data, leaving only complex responses to be finally resolved by humans. We have attempted this process using data from a study seeking to code several hundred answers to a single free-response question. Our preliminary work provides Google's AutoML API with a minimal training sample. In this poster, we compare the accuracy and efficiency of their software to human coders' interrater reliability to demonstrate how we need to continue developing our training sets moving forward.
Abstract Type: Contributed Poster Presentation
Session Time: Poster Session 2 Room C
Poster Number: 2C-14

Author/Organizer Information

Primary Contact: Jennifer Campbell
University of Illinois Urbana-Champaign
Urbana, IL
and Co-Presenter(s)
Sean Golinski, University of Illinois Urbana-Champaign
Joseph Kuang, University of Illinois Urbana-Champaign
Alex Nickl, University of Illinois Urbana-Champaign
Katie Ansell, University of Illinois Urbana-Champaign
Tim Stelzer, University of Illinois Urbana-Champaign