Conference Proceedings Detail Page
|
Assessing a combined human coding and natural language processing method for qualitative analysis in physics education research
As artificial intelligence becomes increasingly integrated into education research, it is critical to assess the capabilities and limitations of such tools in complex disciplinary contexts. This study explores the effectiveness of natural language processing (NLP) for topic identification using the BERTopic package to analyze undergraduate quantum mechanics syllabi. By comparing AI-generated topics with human-coded themes from a corpus of syllabi from 50 US institutions, we find that while NLP reliably identifies broad and structural elements of syllabi (e.g., policies, textbook references), it is limited in its ability to capture the nuanced and context-dependent content typical of upper-level physics instruction. The AI-derived topics were often underrepresented and diverged from expert human interpretation, particularly in areas requiring deep disciplinary knowledge. Our findings support the use of NLP as a complementary tool for theme generation and cross-validation, but emphasize the continued necessity of expert human analysis for rigorous and meaningful educational research.
Physics Education Research Conference 2025
Part of the PER Conference series Washington, DC: August 6-7, 2025 Pages 112-117
ComPADRE is beta testing Citation Styles!
Record Link
<a href="https://www.per-central.org/items/detail.cfm?ID=17126">Buzzell, A, R. Barthelemy, and T. Atherton. "Assessing a combined human coding and natural language processing method for qualitative analysis in physics education research." Paper presented at the Physics Education Research Conference 2025, Washington, DC, August 6-7, 2025.</a>
AIP Format
A. Buzzell, R. Barthelemy, and T. Atherton, , presented at the Physics Education Research Conference 2025, Washington, DC, 2025, WWW Document, (https://www.compadre.org/Repository/document/ServeFile.cfm?ID=17126&DocID=6054).
AJP/PRST-PER
A. Buzzell, R. Barthelemy, and T. Atherton, Assessing a combined human coding and natural language processing method for qualitative analysis in physics education research, presented at the Physics Education Research Conference 2025, Washington, DC, 2025, <https://www.compadre.org/Repository/document/ServeFile.cfm?ID=17126&DocID=6054>.
APA Format
Buzzell, A., Barthelemy, R., & Atherton, T. (2025, August 6-7). Assessing a combined human coding and natural language processing method for qualitative analysis in physics education research. Paper presented at Physics Education Research Conference 2025, Washington, DC. Retrieved February 11, 2026, from https://www.compadre.org/Repository/document/ServeFile.cfm?ID=17126&DocID=6054
Chicago Format
Buzzell, A, R. Barthelemy, and T. Atherton. "Assessing a combined human coding and natural language processing method for qualitative analysis in physics education research." Paper presented at the Physics Education Research Conference 2025, Washington, DC, August 6-7, 2025. https://www.compadre.org/Repository/document/ServeFile.cfm?ID=17126&DocID=6054 (accessed 11 February 2026).
MLA Format
Buzzell, Alexis, Ramón Barthelemy, and Timothy J. Atherton. "Assessing a combined human coding and natural language processing method for qualitative analysis in physics education research." Physics Education Research Conference 2025. Washington, DC: 2025. 112-117 of PER Conference. 11 Feb. 2026 <https://www.compadre.org/Repository/document/ServeFile.cfm?ID=17126&DocID=6054>.
BibTeX Export Format
@inproceedings{
Author = "Alexis Buzzell and Ramón Barthelemy and Timothy J. Atherton",
Title = {Assessing a combined human coding and natural language processing method for qualitative analysis in physics education research},
BookTitle = {Physics Education Research Conference 2025},
Pages = {112-117},
Address = {Washington, DC},
Series = {PER Conference},
Month = {August 6-7},
Year = {2025}
}
Refer Export Format
%A Alexis Buzzell %A Ramón Barthelemy %A Timothy J. Atherton %T Assessing a combined human coding and natural language processing method for qualitative analysis in physics education research %S PER Conference %D August 6-7 2025 %P 112-117 %C Washington, DC %U https://www.compadre.org/Repository/document/ServeFile.cfm?ID=17126&DocID=6054 %O Physics Education Research Conference 2025 %O August 6-7 %O application/pdf
EndNote Export Format
%0 Conference Proceedings %A Buzzell, Alexis %A Barthelemy, Ramón %A Atherton, Timothy J. %D August 6-7 2025 %T Assessing a combined human coding and natural language processing method for qualitative analysis in physics education research %B Physics Education Research Conference 2025 %C Washington, DC %P 112-117 %S PER Conference %8 August 6-7 %U https://www.compadre.org/Repository/document/ServeFile.cfm?ID=17126&DocID=6054 Disclaimer: ComPADRE offers citation styles as a guide only. We cannot offer interpretations about citations as this is an automated procedure. Please refer to the style manuals in the Citation Source Information area for clarifications.
Citation Source Information
The AIP Style presented is based on information from the AIP Style Manual. The AJP/PRST-PER presented is based on the AIP Style with the addition of journal article titles and conference proceeding article titles. The APA Style presented is based on information from APA Style.org: Electronic References. The Chicago Style presented is based on information from Examples of Chicago-Style Documentation. The MLA Style presented is based on information from the MLA FAQ. Assessing a combined human coding and natural language processing method for qualitative analysis in physics education research:Know of another related resource? Login to relate this resource to it. |
ContributeRelated MaterialsSimilar Materials |
Save to my folders
