System details
| Aspect | Rule-based summary | AI walkthrough |
|---|---|---|
| What it is | Sentences written by rules from the trace (web/src/lib/explain/summary.ts) | A general-purpose model given the trace and a fixed prompt (explain-v1) |
| Models | None | Claude Haiku 4.5 (default) or Claude Sonnet 5.5, or an OpenAI model id the visitor types |
| Output | Two or three sentences | JSON: a summary, a step-by-step walkthrough citing step numbers, a note on pruning, four claimed numbers and caveats |
| Determinism | Deterministic | Not guaranteed. Haiku runs at temperature 0, Sonnet 5.5 at low effort with default sampling, OpenAI models with their defaults. The settings are stored with every call |
| Where it runs | In the browser | At the provider, called from the browser with the visitor's key |
| Needs a key | No | Yes, the visitor's own |
Intended use
Helping a reader follow one search: why the tree compared these nodes, why it could skip those regions, and why the count is what it is.
Out of scope: explaining anything other than the trace it is given, giving advice about businesses or places, and any decision about a person.
Inputs
Only the facts of one finished search: the insertion order and file name, the tree's size and height, the query point (as clicked or searched, but not the place name), the counts (including how many comparisons the first walk down the tree took, so that later comparisons can be told apart as checks across split lines), up to 150 numbered steps (depth, split axis and key, distances, outcome) and, for a nearest search, the answer's distance and up to five trading names. The panel shows this payload in full before it is sent.
Evaluation
| Check | How | Result |
|---|---|---|
| Schema | The answer must parse as JSON and fit the Zod schema | Enforced on every call; failures are reported as invalid output |
| Grounding | Four claimed numbers must equal the trace's, and every cited step must exist | Shown with every answer (for example 5/5) and stored in the audit log |
| Explanation quality | The visitor accepts, edits or rejects each answer | Recorded per call; not aggregated by the author |
| Systematic evaluation | The harness at /ai-eval (DR-006) runs the prompt over 20 fixed, seeded searches: the share fully grounded with a Wilson interval, each check's pass rate, the median latency, and a paired comparison of two runs (difference with Newcombe's score interval, exact McNemar test) | Run by visitors with their own keys and kept in their browsers. I have published no results: there is no budget for API calls |
The grounding checks test agreement with the trace, not clarity or correctness of the reasoning in the prose. Twenty cases give wide intervals: even 20 grounded answers out of 20 leave a Wilson lower bound of 83.9%.
Known failure modes
- Restating a number from the trace wrongly, or citing a step that does not exist (caught by the checks).
- Describing a step correctly but drawing a wrong general lesson from it (not caught; left to the reader).
- Treating raw-degree distances as metres.
- Over-reading a truncated trace: radius searches can have thousands of steps, and only the first 150 are sent (the trace says so, and the prompt asks for a caveat).
- Refusals, truncated answers, rate limits and network or CORS failures, all shown as errors and logged (a refused or truncated answer keeps its token count and text in the log).
Ethical considerations
- Keys: stay in the visitor's browser and are sent only to the chosen provider.
- Privacy: the query point is sent, so a visitor who searched their own address sends its coordinates. The place name is not sent. Trading names are public data.
- Transparency: outputs are labelled "AI-generated" with the model name; every call is logged without the key and can be reviewed and exported.
- Governance: these practices are informed by the Australian Government's policy for the responsible use of AI in government (DTA), the EU AI Act transparency principles and the NIST AI Risk Management Framework. This is not a claim of compliance with any of them.