No history yet

Advanced IRT Models

Beyond Difficulty

The simplest Item Response Theory models focus on just one thing: item difficulty. They tell us the ability level a person needs to have a 50/50 shot at getting an answer right. But not all questions are created equal, even if they have the same difficulty level. To get a clearer picture, we need to add more parameters.

How Well Does an Item Discriminate?

Imagine two math problems with the same difficulty. One is a straightforward algebra question. The other is a word problem that's poorly phrased and confusing. Students with high math ability might get tripped up by the confusing language, not the math itself. The first question does a much better job of separating students who know algebra from those who don't.

This separating power is called item discrimination. An item with high discrimination is very effective at telling the difference between people with low and high ability. The probability of answering correctly shoots up sharply as a person's ability increases. An item with low discrimination is less effective; ability level doesn't change the probability of a correct answer as much.

The Two-Parameter Logistic (2PL) model adds this discrimination parameter to the equation.

Pi(θ)=eai(θbi)1+eai(θbi)P_i(\theta) = \frac{e^{a_i(\theta - b_i)}}{1 + e^{a_i(\theta - b_i)}}

A higher aia_i value means a steeper curve and better discrimination. A value near zero means the item is not useful for measurement. In theory, a negative aia_i would mean that lower-ability individuals are more likely to get the item right, which usually points to a flawed question.

What about multiple-choice questions? Even with zero knowledge, a test-taker can still guess the right answer. The 2PL model doesn't account for this.

The Guessing Game

The Three-Parameter Logistic (3PL) model introduces a parameter for guessing. This parameter, often called the pseudo-guessing parameter, represents the probability that a person with very low ability will get the item correct by chance.

For a four-option multiple-choice question, you might expect this probability to be 0.25. However, it's often lower because some answer choices might be obviously wrong, even to a low-ability person.

Pi(θ)=ci+(1ci)eai(θbi)1+eai(θbi)P_i(\theta) = c_i + (1 - c_i) \frac{e^{a_i(\theta - b_i)}}{1 + e^{a_i(\theta - b_i)}}

So far, we've only discussed items that are either right or wrong. But many assessments use more complex scoring, like awarding partial credit or using rating scales.

Handling Multiple Response Categories

Items with more than two possible scores (e.g., 0, 1, 2) are called polytomous. To analyze them, we need different models. Two of the most common are the Graded Response Model and the Partial Credit Model.

Graded Response Model (GRM) This model is designed for items with ordered categories, like a Likert scale where responses range from "Strongly Disagree" to "Strongly Agree." The GRM doesn't model the probability of choosing a specific category directly. Instead, it models the cumulative probability of responding in a certain category or higher.

For a 5-point scale, it would calculate the probability of scoring 1 or more, 2 or more, 3 or more, and so on. The probability of getting a specific score (e.g., exactly 3) is then found by subtracting two of these cumulative probabilities.

The GRM assumes that as a person's underlying trait level increases, their probability of selecting a higher-rated category also increases. Each transition between categories has its own difficulty-like threshold.

Partial Credit Model (PCM) This model is used when partial credit is awarded for an item. Think of a multi-step math problem where a student can earn points for completing each step correctly. The PCM models the probability of earning each possible score (0, 1, 2, ... k points).

Unlike the GRM, the PCM focuses on the probability of getting a certain score over the next lowest score. It estimates a 'step difficulty' for moving from a score of 0 to 1, from 1 to 2, and so on. These models help us understand not just the overall item difficulty, but the difficulty of mastering each step required for a full score.

By moving beyond simple right/wrong scoring and incorporating parameters for discrimination, guessing, and multiple response types, these advanced IRT models provide a much richer and more accurate understanding of how people interact with assessment items.