Bloom's Taxonomy is a useful curriculum planning framework. It gives teachers and curriculum designers a shared vocabulary for describing the cognitive demands of different types of tasks: remembering, understanding, applying, analyzing, evaluating, creating. In curriculum design, it is genuinely helpful for making sure a unit does not consist entirely of recall questions or, conversely, that students are not asked to analyze before they can recall.
Where Bloom's Taxonomy becomes insufficient is in adaptive sequencing. The taxonomy describes the type of cognitive operation a question requires. It does not describe the relationship between two specific questions for a specific student with a specific response history. Knowing that question A is an "applying" level question and question B is an "analyzing" level question tells you nothing about whether question B is an appropriate next question for a student who just got question A wrong in a particular way.
This distinction matters because adaptive engines make sequencing decisions at the level of individual question transitions, and those decisions need to be grounded in something more precise than taxonomy-level difficulty labels.
What Difficulty Actually Means in an Adaptive System
In a static curriculum, difficulty is a property of a question in isolation. A hard question is hard for most students. An easy question is easy for most students. The question has a difficulty rating, and that rating is essentially fixed.
In an adaptive system, difficulty is a relationship between a question and a student's current knowledge state. A question about fraction multiplication is trivially easy for a student who has strong fraction understanding, and impossible for a student who has not yet mastered fraction equivalence. The same question has different effective difficulty for different students at different points in their learning history. This means that a static difficulty tag on a question is only the starting point; it needs to be adjusted for each student's specific context.
The framework we use internally distinguishes three components of effective question difficulty for a given student: intrinsic difficulty (the question's inherent cognitive demand, roughly what the curriculum author assigned), prerequisite load (how many prerequisite concepts does this question require, and how confident is the engine that this student has acquired them), and format novelty (is this question type one the student has seen before, or is the format itself a source of friction).
Intrinsic difficulty is the Bloom's-level assessment. Prerequisite load is the graph-theoretic calculation: trace all ancestors of this concept in the content graph, check the student's mastery state for each ancestor, weight by how closely each ancestor is linked to the current question. Format novelty is the operational question: a student who has only seen multiple-choice questions all semester will experience an open-response question as harder than its intrinsic difficulty implies, simply because the response format is unfamiliar.
The Prerequisite Graph Is the Core Problem
Most adaptive learning systems that use content graphs define prerequisite relationships between concepts, but few define them at the question level. The common approach is: concept B requires concept A. If a student has mastered concept A, they can be given questions about concept B. This is correct but too coarse for quality adaptive sequencing.
Different questions within the same concept can have very different prerequisite profiles. Consider a unit on linear equations. Some questions test the ability to identify whether two expressions are equivalent, which primarily depends on arithmetic and variable notation. Other questions test solving for an unknown when the unknown appears on both sides, which requires more steps and depends on understanding inverse operations. Both are "linear equations" questions. But their prerequisite profiles are different, and a student who fails the second type of question may have a problem with inverse operations specifically, not with linear equations in general.
When we tag questions in our content library, we try to identify the specific prerequisite subconcepts each question depends on, not just the parent concept. This takes more initial work from curriculum authors, but it makes the engine's remediation decisions substantially more precise. Instead of routing a student to "review linear equations," the engine can route them to "review the specific sub-skill where their response pattern suggests a gap."
How We Calibrate Empirically
Initial difficulty tags on questions are author-assigned estimates. They are educated guesses, and in our experience, they are systematically biased toward underestimating difficulty. Curriculum authors know their subject well and tend to underestimate how much prior knowledge a question implicitly assumes.
We recalibrate difficulty estimates using response data from students who interact with the questions. The calibration approach is straightforward: for each question, we track the response distribution across students who had similar prerequisite mastery profiles when they encountered it. If a question was tagged as "medium difficulty" but 70% of students with strong prerequisite mastery are getting it wrong on first attempt, it is probably harder than its tag suggests. We adjust the estimate downward.
This calibration loop requires enough response data to be statistically meaningful, which is a bootstrapping problem. Early in a program's deployment, we have very little data per question. We run a warm-start phase where the engine uses conservative estimates and collects data before making significant difficulty adjustments. The calibration stabilizes after roughly 80-100 response observations per question per comparable student cohort. Programs that run multiple semesters see substantially more accurate difficulty calibration than programs in their first term.
The Discrimination Problem
Item Response Theory, the psychometric framework used in standardized test design, distinguishes between item difficulty (the overall probability of a correct response) and item discrimination (how well the item differentiates between students at different ability levels). A question with high discrimination is one where high-ability students get it right at much higher rates than low-ability students. A question with low discrimination is essentially noise: it does not tell you much about the student's knowledge state regardless of whether they answer correctly.
We care about discrimination because low-discrimination questions are actively harmful in an adaptive system. If a question does not reliably distinguish between students who know the material and students who do not, using it to make a sequencing decision produces a noisy signal. The engine makes a branching choice based on garbage data, and the student pays the cost in terms of inappropriate remediation or premature advancement.
Our calibration process flags questions with discrimination estimates below a threshold as unreliable and reduces their weight in sequencing decisions. They are still available in the content pool, but the engine does not rely on them as primary signals for knowledge state assessment. This flagging is one of the more useful pieces of quality feedback we can give curriculum authors: "this question is not reliably diagnostic, and here is the response pattern that tells us why."
Format Considerations That Are Often Ignored
We are not saying format is as important as content knowledge for learning outcomes. It is not. But format novelty is a real source of difficulty inflation that adaptive systems often do not account for.
In K-12 math specifically, students who have been trained on multiple-choice formats often struggle with open-response formats not because they lack the underlying knowledge but because they lack practice with the response format itself. An adaptive engine that interprets an incorrect open-response answer as evidence of content-knowledge failure may trigger unnecessary prerequisite review when the real issue is format unfamiliarity.
Our current mitigation is to track format exposure per student and apply a format-novelty discount to difficulty estimates when a student encounters a new format type for the first time. The discount reduces the confidence with which the engine interprets the response as a signal about content knowledge. For the first two or three exposures to a new format, the engine requires stronger evidence of consistent failure before triggering a prerequisite branch. After that, the novelty effect is assumed to have worn off and the standard difficulty estimate applies.
Where Calibration Is Still Inadequate
The component of difficulty we model least well is what teachers call "conceptual load from context." A math problem embedded in a word problem about baseball is harder for a student who does not understand baseball than the same calculation presented abstractly. The content knowledge required is identical, but the context wrapping creates additional cognitive demand that varies by student background. We do not model this at all. Our difficulty estimates assume the student can parse the context fluently. When that assumption fails, our estimates are off in ways that the calibration loop is slow to catch.
The longer-term direction for difficulty calibration is to incorporate more of the student's response process, not just the outcome. Response time, the pattern of changes a student makes before submitting a final answer, and the sequence of errors made in multi-step problems all carry information about where the difficulty is coming from. We have some of this data. We are not yet using it fully in the calibration model. That is a 2026 development area, not a current capability.