How it works

Two passes per step. The gap between them is the bug.

Literal--mode action Goal--mode testing Rewriteagent · guard Prove--mode action fails actual_path passes accurate fails broken passes drifted · fixed proof fails · Kane's reason goes back to the agent · ≤ 3 loops Every arrow is a real kane-cli run or a check in code. Nothing is shown to a reader until the last box passes.

What we learned building it

One rule per click, not one rule for all.

"Fail if any label is missing" made Kane check every label on the first page, before clicking anything. The literal pass is now an ordered list of single clicks, and strict mode stops on the first one that's missing.

Strict mode still explores.

Asked to click "API Keys", Kane opened Settings, then Account, then Developers, then Tokens, looking for it. It failed correctly here, but it shows how a single lenient pass hides drift.

"Store what you clicked" reads the screen.

Asked to store its own click path, Kane read a value off the page instead: "Settings→General" for a run that clicked Team → Invite member. Errata reads the clicks from Kane's sealed evidence pack.

One button, three names.

GitHub's activity filter shows "All activity", but its accessible name is "Filter by activity type: All". Errata keeps every name Kane recorded, so a passing step is never flagged over an aria-label.

A crash is not a broken doc.

Kane reports infra_error.* when the browser crashes or the machine runs out of memory. Errata retries once, then marks the step unchecked, never broken.

Kane's triage is a signal, not a verdict.

On a moved menu Kane calls its literal failure an "agent misstep", because from its side the label simply isn't there. Errata decides drift from the goal pass, not the triage.

The same label, twice.

GitHub's file view has a "Code" toggle, and the repository has a "Code" tab. Given only the label, Kane clicked the tab, and the next step started on the wrong page. Kane now also gets the doc's own sentence ("above the file content") as context.

Not every page has a heading.

Errata asked Kane to note the page heading after each step. A raw text file has none, so a correct click failed. The heading is optional now. Both bugs came from the public run and are listed there as false alarms.

Well-kept docs stay accurate.

24 published tutorials from GitHub Docs, Wikipedia, Grafana, Plausible and OpenStreetMap: no confirmed drift. That's a real result too. The demo app shows what drift looks like when it's there.

The honesty rules

A rewrite may only use UI labels that appear in Kane's actual_path or the heading it landed on. The check is in code and covers every bold or quoted label in the sentence itself, not just the labels the agent lists, so a rewrite that invents a label is rejected before Kane ever runs it.

Shell commands, file edits and "check your email" steps are marked not checkable and shown as such, never counted as passed. When Kane itself can't run (sign-in, network, Chrome), the step is unchecked and not counted as a finding.

Recorded runs are labelled as replays everywhere they appear. Every flag in the public run is reviewed against the live page and the evidence, with the reviewer named, and false alarms are published.