Spring semester 2025 was our first full semester with paying programs using Adaptcourse in live classrooms. That changes how you think about what to build and what to hold. When a teacher is looking at the dashboard at 7:45 in the morning before class starts, the pressure on the product is real and immediate. This post is our honest accounting of the semester: what we shipped, what the usage data told us, and what we got wrong.
What We Actually Shipped
The question-level branching logic that we had architected in late 2024 got its first real stress test across three pilot programs this spring. The core behavior is: when a student answers incorrectly, the engine has to decide whether to remediate the prerequisite, loop on the current concept at a lower difficulty, or flag for teacher review. We had the logic implemented, but the thresholds were calibrated on synthetic data. Real students turned out to be more variable than our test sets.
The biggest change we made in February was to the prerequisite gap detection scoring. We had been using a hard cutoff: two consecutive wrong answers on a concept triggered a prerequisite trace. But what we observed was a pattern we now call "unlucky run" where a student who genuinely understands a concept gets two wrong in a row due to fatigue or a badly worded question. Branching them to prerequisite review at that point was disruptive and, honestly, a little demoralizing. We moved to a weighted signal that incorporates response time alongside accuracy, which reduced premature prerequisite branches by about 30% in the pilot cohorts.
We shipped the teacher dashboard's "class pulse" view in March. The core idea was simple: show the teacher, at a glance, which students are moving forward, which are stuck in a prerequisite loop, and which the engine has flagged as needing human attention. The initial version had too many toggles. Teachers were not using the filters. We stripped it back to three states, added a visual density indicator per student tile, and adoption in the morning pre-class window went up significantly across the three programs.
What the Data Told Us
We track two primary engagement metrics internally: session depth (the number of questions a student completes before exiting a session) and re-entry rate (whether a student who exits a session comes back within 24 hours). These are leading indicators for us, not outcome metrics, but they are useful proxies for whether the pacing feels right.
Session depth improved across all three programs through the semester, which we attribute mainly to the prerequisite branching refinement. The pattern we saw was consistent: students who hit a prerequisite branch early in a session had much lower session depth than students who moved through on-grade content. Getting the branching threshold right was the single highest-impact engineering change of the semester.
What surprised us was the re-entry rate. We expected it to climb as students got more comfortable with the product. It did for most students, but there was a distinct cohort, roughly 12-15% of the total across programs, where re-entry dropped off sharply around week six of the semester. Looking at the session logs for those students, the pattern was clear: they had been routed into prerequisite remediation loops that were not resolving. The engine was cycling them through the same material because the prerequisite content itself contained gaps we had not mapped. That is a content graph problem, not a sequencing problem. We started addressing it in April with a content graph audit pass, but it is ongoing work.
Teacher Feedback: What They Actually Said
We did structured check-ins with the lead teachers in each pilot program at week four and week twelve. The feedback split into two categories with very little overlap.
The feedback that was enthusiastic focused almost entirely on prerequisite gap visibility. Several teachers said something close to: "I knew some students were behind, but I did not know what specifically they were missing." That visibility, delivered at the question level rather than the quiz level, was genuinely new for them. One teacher mentioned that she had changed her small-group intervention strategy based on what the dashboard showed, grouping students by shared prerequisite gaps rather than by the chapter they were currently on.
The frustrated feedback was almost all about the content graph itself, not the engine behavior. Teachers wanted to add questions, adjust difficulty tags, and flag questions they thought were misleading. We had not built that workflow yet. Exporting a CSV and sending it to us was not a reasonable ask for a classroom teacher mid-semester. This is the clearest product gap we are carrying into the fall. We are building a lightweight content editing interface for the summer, with the goal of having it live for fall pilots.
What We Got Wrong
We are not saying this semester was a failure, but honesty about what did not work is more useful than a success story.
The onboarding process for teachers was too long. We had designed it assuming teachers would have 60-90 minutes before semester start to read documentation and configure their course graph. In practice, most teachers had 15-20 minutes across two fragmented sessions. We need a "start in 20 minutes" path that gets a teacher to a working course with sensible defaults, and a separate path for deeper customization. We are rebuilding the onboarding flow this summer.
We also underinvested in admin-level visibility. The programs that ran through school-level pilots had a curriculum coordinator or IT lead who wanted progress reports they could share upward. Our export formats were not formatted for that audience. Adding a structured PDF export for end-of-unit summaries is on our roadmap for the fall release.
One thing we genuinely got right: we did not try to build a student-facing "progress" display this semester. There is a design temptation to show students their learning path, reward streaks, or gamification elements. We resisted it, and I think that was correct. The engine behavior is not legible enough yet at the student level. When a student sees themselves being branched to prerequisite review, we need the explanation for why to be clear and constructive, not a badge system. We will revisit this in fall 2025, but only once we have confidence in the branching logic quality.
What Is Coming This Summer
The summer development focus is on three things: the content editing interface for teachers, the admin reporting export, and a refactor of the session persistence layer that we have been putting off. Session persistence is an unglamorous infrastructure issue, but we have had enough student progress rollbacks from dropped connections to make it a priority before we scale to more students in fall.
We are also running a quiet beta of the LTI integration with a partner platform through June and July. Integration quality varies significantly across LMS environments, and we want controlled conditions before we open it to anyone who asks. If you are running a program that uses Canvas, Schoology, or Moodle and want to be part of that beta, reach out directly.
The progress report format will change next semester. We are committing to publishing a fall 2025 update in December that focuses specifically on outcome data from programs that ran a full semester, not just engagement proxies. That will require careful framing: we are not claiming causality, we are describing what we observed. But the data from this spring gives us enough signal to report on meaningfully by December.