PERC 2021 Abstract Detail Page
Previous Page | New Search | Browse All
| Abstract Title: | Balancing content of computerized adaptive testing for the Force Concept Inventory |
|---|---|
| Abstract: | As a method to shorten the test time of the Force Concept Inventory (FCI), a previous study suggested the use of Computerized Adaptive Testing (CAT). CAT is the process of administering a test on a computer, with items (i.e., questions) selected based upon the responses of the examinee to prior items. As a step to develop a CAT–based version of the FCI (FCI-CAT), the previous study examined the optimal test length of the FCI-CAT such that accuracy and precision [which were measured in terms of root-mean-square error (RMSE)] of Cohen's d would be comparable to that of the full FCI for a given class size. The objective of this paper is to address an issue in the previous study and, in so doing, to improve the FCI-CAT. We consider content balancing ensuring that the same set of concepts assessed in the original test is covered in the CAT administration for each respondent. To balance content in CAT, the percentage of items to be administered from each subgroup is defined in advance. Doing so ensures that items from each subgroup are administered. We conducted a Monte Carlo simulation to analyze how implementing an algorithm of content balancing affects the RMSE of Cohen's d. As a result, we found that, for a class size of 40, the increase of the RMSE due to content balancing is 6%-7% for test lengths of 2-5 items and less than 1% if the test length is larger than 13 items. This result indicates that for a sufficiently large test length (say, larger than 13 items), content balancing does not compromise the accuracy and precision of the FCI-CAT. Hence, we recommend that the FCI-CAT incorporate content balancing, provided the test length is larger than 13 items. |
| Abstract Type: | Contributed Poster Presentation |
| Session Time: | Poster Session 2 Room C |
| Poster Number: | 2C-1 |
| Contributed Paper Record: | Contributed Paper Information |
| Contributed Paper Download: | Download Contributed Paper |
Author/Organizer Information | |
| Primary Contact: |
Jun-ichiro Yasuda Yamagata University Yamagata, Non U.S. 990-8560 Phone: +81-23-628-4030 |
| Co-Author(s) and Co-Presenter(s) |
Michael M. Hull, University of Vienna. |
