How it works
Two passes per step. The gap between them is the bug.
What we learned building it
"Fail if any label is missing" made Kane check every label on the first page, before clicking anything. The literal pass is now an ordered list of single clicks, and strict mode stops on the first one that's missing.
Asked to click "API Keys", Kane opened Settings, then Account, then Developers, then Tokens, looking for it. It failed correctly here, but it shows how a single lenient pass hides drift.
Asked to store its own click path, Kane read a value off the page instead: "Settings→General" for a run that clicked Team → Invite member. Errata reads the clicks from Kane's sealed evidence pack.
GitHub's activity filter shows "All activity", but its accessible name is "Filter by activity type: All". Errata keeps every name Kane recorded, so a passing step is never flagged over an aria-label.
Kane reports infra_error.* when the browser crashes or the machine runs out of memory. Errata retries once, then marks the step unchecked, never broken.
On a moved menu Kane calls its literal failure an "agent misstep", because from its side the label simply isn't there. Errata decides drift from the goal pass, not the triage.
GitHub's file view has a "Code" toggle, and the repository has a "Code" tab. Given only the label, Kane clicked the tab, and the next step started on the wrong page. Kane now also gets the doc's own sentence ("above the file content") as context.
Errata asked Kane to note the page heading after each step. A raw text file has none, so a correct click failed. The heading is optional now. Both bugs came from the public run and are listed there as false alarms.
24 published tutorials from GitHub Docs, Wikipedia, Grafana, Plausible and OpenStreetMap: no confirmed drift. That's a real result too. The demo app shows what drift looks like when it's there.
The honesty rules
A rewrite may only use UI labels that appear in Kane's actual_path or the heading it landed on. The check is in code and covers every bold or quoted label in the sentence itself, not just the labels the agent lists, so a rewrite that invents a label is rejected before Kane ever runs it.
Shell commands, file edits and "check your email" steps are marked not checkable and shown as such, never counted as passed. When Kane itself can't run (sign-in, network, Chrome), the step is unchecked and not counted as a finding.
Recorded runs are labelled as replays everywhere they appear. Every flag in the public run is reviewed against the live page and the evidence, with the reviewer named, and false alarms are published.