html-docs desktop · design note · v2
Why Mission Control feels laggy, and a three-template protocol — plan, execution, summary — that renders agent status instantly while giving the agent a high-bandwidth, low-token channel to keep it fresh. Templates carry the basics; freeform HTML islands keep custom visualization first-class; and the whole protocol is harness-neutral — Claude Code, Codex, OpenCode, anything that can write a file.
Latency isn't one bug — it's a chain of throttles, each reasonable alone, that compound into a UI that trails the agent by 10–15 seconds. The measured budget, from the current code:
| Stage | Delay | Where |
|---|---|---|
| Fleet card "step" line | ≤ 10 s throttle | canvas-activity.mjs · FLEET_MS |
| Activity feed patch | ≤ 1.5 s throttle | canvas-lib.mjs · patchRegionThrottled |
| Diff meter | ≤ 3 s throttle | canvas-activity.mjs:217 |
| Desktop file watcher | + 400 ms stability + 500 ms debounce | fleet.ts:61-68 |
| Fleet refresh cost | full re-discovery, every poke | fleet.ts · list() re-parses transcripts |
| Deliverable ("stage") | minutes — agent hand-writes HTML | canvas skill · curl per region |
| Summary ("debrief") | only at Stop | canvas-stop.mjs |
The deepest problem is the last three rows. Status the hooks can derive (activity, diff) is merely throttled — but anything with meaning (what phase are we in, what's the current step called, what's the result) either waits for the agent to hand-author HTML through curl, or is inferred from transcript mtimes. The agent's channel to the UI is expensive (hundreds of tokens of HTML) so it's used rarely, and everything downstream is a workaround for that scarcity.
Make a single structured state file the source of truth, and make every renderer — the desktop fleet card, the session detail view, the cloud canvas doc — a deterministic template over that state. No LLM in the render path. The agent and the hooks both write the same file; the agent sends tiny JSON patches (tens of tokens), hooks auto-fill the mechanical parts, and the templates absorb both.
The agent's bandwidth problem disappears: updating "current step" costs one short Bash line, not a rewritten HTML document.
Extends the existing .claude/canvas/state/<sessionId>.json (which already holds activity, chat, tasks, region keys). New fields are additive and versioned; old sessions render exactly as before.
{
"v": 1,
"phase": "plan" | "execute" | "summary",
"headline": "Adding PDF export to /convert", // agent-set, one line — becomes the fleet card step
"plan": {
"goal": "PDF in + PDF out on the convert page",
"steps": [ // hook-filled from TodoWrite, agent may override
{ "id": "s1", "text": "Accept .pdf uploads", "status": "done" },
{ "id": "s2", "text": "Render output via Chromium", "status": "doing" },
{ "id": "s3", "text": "Verify + commit", "status": "todo" }
],
"approach_md": "Reuse renderPdf + withPrintCss…", // freeform room
"questions": ["Keep print-edition variant out of scope?"]
},
"execute": {
"currentStep": "s2",
"checks": [ { "name": "tsc", "status": "pass" },
{ "name": "smoke: /convert", "status": "running" } ],
"blockers": [],
"note_md": "Chromium path needs --no-sandbox on Vercel",
"activity": [ …existing… ], // hook-filled (unchanged)
"diff": { "files": 3, "add": 184, "del": 52 } // hook-filled (unchanged)
},
"summary": {
"outcome": "shipped" | "partial" | "blocked",
"result_md": "PDF upload + output live on /convert…",
"artifacts": [ { "label": "commit adfc2d9", "ref": "git" } ],
"verification": ["tsc clean", "renderPdf smoke test → %PDF-"],
"followups": ["Wire print-edition variant"],
"blocks": [ // optional custom-viz islands — see §5
{ "type": "html", "title": "Coverage map", "html": "<svg …>" }
]
}
}
Ownership rule, to keep writers from fighting: hooks only ever write their designated fields (activity, diff, steps-from-TodoWrite, timestamps). The agent owns everything else via merge patches. If the agent has written plan.steps itself, the hook's TodoWrite projection defers.
Each template is fixed structure for the basics (auto-updated, always fresh) plus one freeform markdown slot (the agent's room to say more). Tags on each block show who keeps it updated.
Plan · round 2
Adding PDF export to /convert
Goal agent
PDF in + PDF out on the convert page, reusing the docs' export pipeline.
Steps hook · TodoWrite
Approach agent · freeform md
Reuse renderPdf + withPrintCss from the in-doc export; new server action skips the document lookup since /convert hasn't saved a doc yet.
Open questions agent
Keep the LLM print-edition variant out of scope for v1?
Replaces: plan-file markdown mirror + TodoWrite rail (both survive as inputs).
Executing · step 3 of 4
Render via headless Chromium server action
Checks agent
Working note agent · freeform md
Chromium needs --no-sandbox on Vercel; local falls back to system Chrome.
Activity hook · every tool call
The step rail collapses to a one-line header here; the full rail stays one click away.
PDF upload + PDF output live on /convert
Result agent · freeform md
Upload tab accepts .pdf (signed upload + extraction); new PDF output format renders through the same Chromium pipeline as the in-doc export.
Verification agent
Follow-ups agent
Wire the print-edition (LLM re-typeset) variant into convert.
The Stop hook stamps outcome/diff/duration automatically if the agent hasn't — a summary always exists, even on a crash.
Templating everything would trade one problem for another — the canvas's whole charm is that an agent can draw. The middle ground is a three-tier contract, where each tier trades expressiveness for guaranteed freshness:
| Tier | Rendering | Guarantee |
|---|---|---|
| Card / fleet grid | structured fields only (headline, phase, steps, diff) | always fresh, never waits on custom HTML |
| Phase panels | template blocks, with html islands inline | basics auto-update; islands render verbatim |
| Stage (deliverable) | fully freeform HTML — unchanged from today | unlimited; the hero surface stays the agent's canvas |
Concretely, each phase section accepts an optional blocks array with a small vocabulary the renderer understands — steps, checks, md, metric, table, and crucially html: a verbatim island for anything custom (an inline SVG chart, a wireframe, a heat map). Islands sit inside the template, so a session with a beautiful custom diagram still has a live step rail above it. And if the agent wants the whole panel, a phase can declare "render": "custom" with its own HTML — a full takeover of the detail view, while the card tier keeps rendering from structured fields, so the fleet grid never goes stale even when a panel is bespoke.
Rule of thumb the system prompt teaches: status is structured, deliverables are drawn. The templates aren't a fence — they're the floor that's always there when the agent doesn't spend tokens on more.
One tiny CLI (installed by the hooks at .claude/canvas/bin/canvas) that deep-merges a JSON patch into the state file and touches nothing else. The PostToolUse hook sees the write and re-renders templates immediately.
# set the phase + headline (≈20 tokens)
canvas '{"phase":"execute","headline":"Wiring Chromium render"}'
# flip a check (≈15 tokens)
canvas '{"execute":{"checks":[{"name":"tsc","status":"pass"}]}}'
# land the summary (≈60 tokens)
canvas '{"phase":"summary","summary":{"outcome":"shipped",
"result_md":"PDF in/out live on /convert",
"followups":["print-edition variant"]}}'
| Channel | Cost per update | Freshness |
|---|---|---|
| Today: agent curls region HTML | 300–1500 tokens | once or twice a session |
Proposed: canvas JSON patch | 15–60 tokens | every step transition |
| Hooks (activity, diff, steps) | 0 tokens | every tool call |
The system prompt append (already threaded via systemPromptAppend in types.ts) teaches the protocol in a few lines: "keep headline current; flip phase when you move; land summary before stopping." Because each update is cheap, the agent can afford to do it at every transition — that's the bandwidth.
Nothing in the core is Claude-specific: the state file is JSON on disk, the templates are pure functions, and the canvas patch CLI is just a binary. That makes the schema — not the hooks — the protocol. Harness-specific code shrinks to an adapter whose only job is translating native events into state patches. The desktop already has this seam: AgentAdapter in agents/types.ts, with claude and a gated codex skeleton registered in agent-manager.ts.
| Harness | Adapter mechanism | What it feeds |
|---|---|---|
| Claude Code | hooks (SessionStart / PostToolUse / Stop) — today's code, refactored to emit state patches | activity, diff, TodoWrite→steps, turn status |
| Codex | codex exec --json event stream — the adapter process parses events and writes patches itself (per-turn process model) | activity, turn status, token/cost |
| OpenCode | plugin subscribing to its event bus, writing the same patches | activity, steps, turn status |
| Anything else | lowest common denominator: a generic tailer for stdout/JSONL logs, plus the canvas CLI — any agent that can run a shell command gets the full semantic channel with zero integration | headline, phase, checks, summary |
Two consequences worth naming. First, the canvas CLI is the universal port: even a harness with no adapter at all can drive the semantic layer just by being told about the command in its instructions — the same few prompt lines work verbatim in Claude Code, Codex, and OpenCode. Second, the directory contract should shed its Claude-specific path: the protocol root becomes vault-level (e.g. .mission/state/), with the current .claude/canvas/state/ kept as a read alias during migration. Mission Control's FleetService then watches one neutral directory regardless of which harness produced the session, and per-harness quirks (turn boundaries, todo tools, cost fields) never leak past the adapter.
fleet.ts): state JSONs are single atomic writes — drop awaitWriteFinish for the state dir and cut the debounce to ~100 ms. Keep the 400 ms window only for transcript files.list() re-discovery only on add/unlink. This is the biggest perceived-latency win.headline / current step from state directly — the 10 s FLEET_MS throttle stays only for the cloud fleet doc PATCH; the local card updates instantly.SessionState v1 types + deep-merge + template functions (incl. the block vocabulary) in one shared module used by both hooks and desktop (canvas-lib.mjs stays the hooks' entry; desktop imports the templates via shared/).canvas patch CLI + system-prompt teaching lines. Hooks start writing phase/steps/timing into the new fields alongside the old ones.headline.v: 1. Stage stays freeform forever — it's tier 3 by design.exec --json (the skeleton in codex-adapter.ts already gates on detect()); neutral .mission/ root with the legacy path as read alias.Net effect: the mechanical layer (activity, diff, step states) updates within ~100 ms of each tool call with zero agent tokens; the semantic layer (phase, headline, checks, summary) updates at every transition for the price of a short Bash one-liner; and rendering is a pure function everywhere, so no surface ever waits on a model.