Probing for Bias: Comparing Populations Using Item Response Curves

Posted by : on

Category : science   coauthor


Probing for Bias: Comparing Populations Using Item Response Curves

Paul Walter, Edward Nuhfer, and Crisel Suarez

We introduce an approach for making a quantitative comparison of the item response curves (IRCs) of any two populations on a multiple-choice test instrument. In this study, we employ simulated and actual data. We apply our approach to a dataset of 12,187 participants on the 25-item Science Literacy Concept Inventory (SLCI), which includes ample demographic data of the participants. Prior comparisons of the IRCs of different populations addressed only two populations and were made by visual inspection. Our approach allows for quickly comparing the IRCs for many pairs of populations to identify those items where substantial differences exist. For each item, we compute the IRC dot product, a number between 0 and 1 for which a value of 1 occurs when the IRCs of the two populations are identical. We then determine whether the value of the IRC dot product is indicative of significant differences in populations of real students. Through this process, we can quickly discover bias across demographic groups. As a case example, we apply our metric to illuminate four SLCI items that exhibit gender bias. We further found that gender bias was present for non-science majors on those items but not for science majors.


About Crisel Suarez
Crisel Suarez

Hi I am Crisel Suarez!

Email : criselsuarez[at]gmail[dot]com

Website : https://criselsuarez.github.io/

About Crisel Suarez

¡Hola! My name is Crisel Suarez. I am a Physics PhD candidate at Vanderbilt University and Smithsonian Astrophysical Observatory Predoctoral Fellow . I grew up in Mexico City, Mexico and Austin, TX, USA. I am fully bilingual in both Spanish and English. I earned my Bachelor’s of Science and majored in Mathematics and minored in Global Studies and Physics at a small liberal arts school called St. Edward’s University . I was a McNair Scholar at St. Edward’s University and pursued all types of research; from Computer Science looking at the expected values of properties in the game of Monopoly (with Dr. Michael Kart ), Mathematics looking at the tetrominoes ( TETRIS pieces) and how they form gaps and make you lose at TETRIS (with Dr. Jason Callahan ), and Science Education in quantifying science literacy in undergraduates students (with Dr. Paul Walter . I also took a graduate Nanotechnologies and Nanofabrication certification from the Nanomanufacturing Systems Center (NASCENT) at University of Texas at Austin.

After graduating from St. Edward’s University, I went to pursue my Masters of Science in Physics at Fisk University as part of the Fisk-Vanderbilt Masters to PhD program . At Fisk, I was part of Dr. Arnold Burger’s Materials Science and Applications Group (MSAG) where I studied the chemical composition of high energy crystal scintillators using Vanderbilt’s University high performance computing ACCRE facilities.

During the summers as a Masters student, I had the opportunity to work with Dr. Christopher S. Moore at the Center for Astrophysics | Harvard & Smithsonian in the Solar Stellar X-ray Group as part of the Fisk-Vanderbilt-CfA Latino Initiative Program (now CfA Research Experiences in Astronomy, Technology, & Engineering (CREATE) program. I worked on analyzing beautiful high energy events in the Sun called solar flares . As part of PhD work at Vanderbilt University, I continued to examined ~30 soft x-ray (SXR) flares observed with the Miniature X-ray Solar Spectrometer (MinXSS-1) and derived changes in temperature, volume emission measure and elemental abundances as a function of time with parametric spectral fits. I continue to research solar flares and look for Quasi-Periodic Pulsations (QPPs) in flares with different instruments including Hinode/X-ray Telescope (Hinode/XRT) , Solar Dynamic Observatory/Atmospheric Imaging Assembly (SDO/AIA) , Geostationary Operational Environmental Satellite (GOES) , Miniature X-ray Solar Spectrometer- 1 and 2 (MinXSS-1&2) and Dual Aperture X-ray Solar Spectrometer (DAXSS). I have also used the Hydrodynamic Radiation (HYDRAD) model to synthesize Extreme Ultraviolet (EUV) and soft x-ray (SXR) emission in solar flares.

I also enjoy engaging in science outreach events, promoting research, curiosity, creativity and mentoring the next generation of scientists!

When I have some rare free time, I like going to museums, shows, trying new foods, and traveling!

Star