Mastering Knowledge Assessment
Cognitive Biases in Framing
The Power of Framing
The way a question is worded can subtly nudge you toward a specific answer, even if it's not the one you'd choose based on your knowledge alone. This is the core of the Framing Effect, a cognitive bias where people react to a particular choice in different ways depending on how it's presented.
Consider a multiple-choice question on a public health policy exam:
Option A (Gain Frame): "If Program A is adopted, 200 people will be saved. If Program B is adopted, there is a 1/3 probability that 600 people will be saved, and a 2/3 probability that no people will be saved. Which of the two programs would you favor?"
Option B (Loss Frame): "If Program C is adopted, 400 people will die. If Program D is adopted, there is a 1/3 probability that nobody will die, and a 2/3 probability that 600 people will die. Which of the two programs would you favor?"
Mathematically, the outcomes are identical. Program A and C are the same, as are B and D. Yet, most people choose Program A (the sure thing) when it's framed as a gain, and Program D (the risky option) when it's framed as avoiding a loss. Test designers can use this to make a specific answer feel more appealing.
When you see options presented in terms of gains versus losses, pause. Rephrase the question in your own words to remove the emotional framing before choosing an answer.
Anchors and Estimates
Our brains often latch onto the first piece of information offered when making decisions, a bias known as anchoring This initial piece of information, the "anchor," influences subsequent judgments, even if it's completely irrelevant. In assessments, this can skew your response on numerical or scaled questions.
For instance, a performance review might ask, "Given that the team's average satisfaction score last year was 7 out of 10, how would you rate your satisfaction this year?" The mention of "7" acts as an anchor. You're now more likely to give a score closer to 7 than you would have if the question was simply, "How would you rate your satisfaction on a scale of 1 to 10?"
This can also appear in questions asking for numerical estimates. A question like, "Was the Eiffel Tower completed before or after 1850?" anchors you to the year 1850, making your subsequent guess of the actual year (1889) closer to 1850 than it might have been otherwise.
Mental Shortcuts That Lead Astray
When faced with multiple answer choices, we often rely on mental shortcuts. The availability heuristic is one such shortcut, where our minds judge the likelihood of an event by how easily examples come to mind. If you just read an article about shark attacks, you'll likely overestimate the probability of being attacked by a shark. In a test, this means you might be more inclined to select an answer that relates to a recent or vivid example from your studies, even if another answer is more accurate but less memorable.
When designing summative assessments, instructors should remember to:
Clarify assessment goals (in alignment with course learning goals) Develop clear assessment criteria Communicate assessment goals and criteria to students Communicate allowed/disallowed uses of artificial intelligence tools (in light of assessment goals) If appropriate, use a rubric Communicate assessment format and other parameters in advance to reduce student anxiety/cognitive load, and focus cognitive resources on the assessment task itself Offer practice opportunities focused on the assessed skills/content.
This ties into the challenge of identifying leading questions, which are framed to suggest a particular answer. A self-assessment might ask, "Do you agree that our streamlined new process is a significant improvement over the old, clunky one?" This question doesn't just ask for an opinion; it frames the new process as superior and the old one as inferior, making it harder to disagree. A neutral question would be, "What are your thoughts on the new process compared to the old one?"
Dunning-Kruger effect
noun
A cognitive bias in which people with low ability at a task overestimate their ability. It is related to the cognitive bias of illusory superiority and comes from the inability of people to recognize their lack of ability.
Finally, we must contend with our own self-perception during tests. The describes our tendency to overestimate our competence in areas where we are actually weak. This can lead to a dangerous overconfidence, causing you to spend too little time on questions you find "easy" or to stick with an initial, incorrect answer without double-checking.
To counteract this, get into the habit of metacognition. As you go through an assessment, consciously rate your confidence in each answer. Flag any question you're even slightly unsure about and return to it later with a fresh perspective. This forces you to separate your feeling of knowing from your actual knowledge.
Let's check your understanding of these cognitive traps.
A public health program is described to one group as "saving 200 out of 600 people" and to another group as "400 out of 600 people will die." People are more likely to support the first option, even though both outcomes are identical. This demonstrates which cognitive bias?
A question asks, "Was the population of ancient Rome more or less than 500,000 people?" followed by, "What is your estimate of ancient Rome's population?" The number 500,000 influences the subsequent estimate. What is this initial number acting as?
Recognizing these patterns in how questions are framed and how our own minds work is a critical skill. It allows you to engage with the content of an assessment objectively, ensuring you're demonstrating your knowledge, not just reacting to psychological nudges.