Rubber Duck
SOURCE SNAPSHOT · 2026-07-26 02:34 UTC · LOCAL / NO GIT HEAD
Project field guide

A second mind inside the doc.

Rubber Duck is a voice-first brainstorming partner for Google Docs and HTML Docs. It listens, reasons, searches the current document, writes into a protected session area, and asks before touching the rest. This is the complete map—from microphone to model to provider-safe document change. E-README E-ARCH

2document providers behind one client contract
5Realtime tools exposed to the voice partner
4protected lanes: ideas, decisions, questions, next steps
2reasoning modes: realtime conversation + deep think
23/24automated cases passing at snapshot time

01 / Mental model

The browser is the body. The API is the trust boundary.

The product is split so the browser can see the active Google Doc and use the microphone, while the server keeps long-lived OpenAI credentials out of the extension. Shared TypeScript contracts keep both halves speaking the same language. E-ARCH

Rubber Duck system cutaway A spoken idea travels from the user and Chrome side panel to the OpenAI Realtime service, invokes tools in the extension or backend, and reaches Google Docs or HTML Docs only through guarded provider clients. CHROME side panel APP STATE session · transcript · UI VOICE SESSION WebRTC · tool dispatch DOC CLIENT provider-neutral contract CONTEXT current-doc paragraph search RUBBER DUCK API Fastify trust boundary AUTH EXCHANGE Google identity → 1h JWT REALTIME SECRET short-lived client credential DEEP THINK structured reasoning result SESSION STORE memory or PostgreSQL OPENAI Realtime voice model gpt-realtime-2.1 Deep reasoning model gpt-5.6-sol · medium DOCUMENT HOSTS Google Docs REST API HTML Docs region API Document source of truth revision-guarded writes VOICE / JWT WEBRTC DIRECT PROVIDER READ / GUARDED WRITE
YELLOW CURRENT = the live conversation path · SOLID INK = service calls · DASHED INK = the direct browser-to-Docs path. E-MANIFEST E-VOICE E-API
Key architectural choice

Google Docs and HTML Docs credentials stay in the extension; OpenAI access stays behind the project API. The server is not a document proxy and never receives document-provider credentials. E-ARCH

02 / One idea’s journey

What actually happens when you press Start.

Session startup is a handshake across Chrome, Google, the project API, and OpenAI. The side panel owns the lifecycle and shuts itself down if you switch documents. E-APP

Identify the active supported document

The content script recognizes Google Docs URLs plus HTML Docs owner, long-share, and clean-share URLs. It sends provider, document ID, origin, and any share token to the background worker.

CONTENT + BACKGROUND

Acquire Google identity, then an app token

Chrome Identity yields a Google token. The API verifies the Google user and issues a one-hour app JWT subject to email/domain policy.

EXTENSION → API

Select a provider client and install the protected scaffold

Google Docs creates session/category named ranges. HTML Docs places session/category markers inside a stable editable region. Both satisfy the same document-client contract.

DOCUMENT PROVIDER

Mint a short-lived Realtime credential

The extension requests a client secret from the project API, then establishes the voice connection directly to OpenAI over WebRTC.

API → OPENAI

Inject document context without speaking it aloud

The current snapshot is sent as silent context. During a live session, the app checks the revision every five seconds and reinjects context when it changes.

VOICE SESSION

Listen, respond, and call tools

The Realtime agent can reread, search, append a protected note, prepare a proposal, or ask the API for deeper reasoning. Each tool call is timed and audited.

REALTIME TOOLS

End cleanly and retain a minimal session record

Microphone and voice resources close; the recent conversation is summarized; transcript, proposals, metrics, and tool audit are sent to the session endpoint. The UI also offers deletion.

APP + STORE

The active document is polled about every 1.5 seconds. A document change ends the running session instead of allowing the agent to keep writing against a stale document identity. E-APP

03 / Chrome extension

Five layers live inside the side panel.

The extension is a Manifest V3 React app, but its responsibilities are deliberately split between Chrome shell code, orchestration state, a provider-neutral document contract, provider implementations, and voice transport. E-MANIFEST E-PROVIDER

Chrome shell

manifest.json, background.ts, content.ts, and messages.ts register the panel, watch navigation, recognize both providers, and bridge active-document state.

Session orchestration

App.tsx selects the provider client and owns status, snapshots, credentials UI, session ID, transcript, proposals, microphone state, polling, errors, and cleanup.

Voice partner

voice.ts configures WebRTC, transcription, voice activity detection, model instructions, tool schemas, context injection, and rollover.

Document contract

document-client.ts defines one interface for read, protected-section setup, note append, proposal creation, and approved apply.

Provider safety clients

google-docs.ts uses named ranges and revisions; html-docs.ts uses stable regions, session markers, exact anchors, and pre-patch rereads.

Extension permissions

Chrome permissions are identity, sidePanel, storage, and tabs. OAuth requests openid, email, and full Google Documents access. Host permissions cover Google Docs, HTML Docs (including local port 3000), Google OIDC, OpenAI, and the local or deployed Rubber Duck API. E-MANIFEST

Sensitive scope

The full Google Documents scope is powerful because the Google provider must read and edit the active doc. HTML Docs uses a per-document share token or an optional account API key stored in Chrome. A real rollout needs a clear privacy story, OAuth verification work, and tight allowlisting.

04 / Voice + tools

Conversation is realtime. Actions are explicit tools.

The voice partner runs on gpt-realtime-2.1 with the marin voice, semantic VAD, near-field noise reduction, audio output, and gpt-4o-mini-transcribe. Its system instructions frame it as a concise senior partner and forbid claiming a write succeeded unless a tool reports success. E-VOICE E-OPENAI

ToolPurposeEffect boundarySafety posture
read_document Refresh the document snapshot when the agent needs ground truth. Active-provider read READ ONLY
append_session_note Add a normalized bullet to Ideas, Decisions, Open Questions, or Next Steps. Protected named range AUTO-WRITE
propose_document_edit Create a before/after replacement for the user to inspect in the side panel. Local proposal only APPROVAL
deep_think Escalate a difficult question to a stronger reasoning model and return structured insights. Authenticated API call NO DOC WRITE
search_context Rank current-document paragraphs against query terms. In-memory snapshot READ ONLY
Why two models?

The Realtime model keeps conversation fluid and can use tools. The deeper model—gpt-5.6-sol at medium reasoning—is called selectively for harder synthesis and returns strict JSON: answer, insights, and suggestedNotes. E-OPENAI

05 / Document safety

Named ranges define where autonomy stops.

Rubber Duck has two write lanes across both providers. One is narrow and automatic; the other is broad and gated. The provider-neutral contract preserves that invariant while each host gets its own conflict logic. E-PROVIDER E-DOCS E-HTML-DOCS

LANE A · TOOL RESULT APPLIES DIRECTLY

Protected Brainstorm section

  • Google: root and category named ranges
  • HTML Docs: a session marker inside one stable region
  • Ideas
  • Decisions
  • Open Questions
  • Next Steps

A note is whitespace-normalized and deduplicated. Google inserts a bullet at the named-range boundary; HTML Docs appends escaped HTML inside the matching category span.

⇄
LANE B · USER MUST APPLY

The rest of the document

  • Agent creates an anchored before/after proposal
  • Side panel renders Apply and Dismiss
  • Apply refetches the latest provider state
  • Anchor must still occur exactly once
  • Google maps offsets to Docs locations
  • HTML Docs maps visible text inside one region

Ambiguity, missing anchors, unsupported modes, or unresolved revision conflicts fail closed.

How a safe replacement works

Proposal is inert

Creating it does not write to either provider. It exists in extension state as before/after text plus an anchor.

LOCAL STATE

User chooses Apply

The client refetches the newest document snapshot rather than trusting the version that produced the proposal.

EXPLICIT ACTION

Anchor is revalidated

The original text must occur exactly once. A missing or repeated anchor is unsafe, so no mutation is attempted.

FAIL CLOSED

The provider maps visible text safely

Google converts plain-text boundaries into Docs structural locations. HTML Docs resolves visible text inside exactly one editable region while preserving surrounding markup.

MAPPING LAYER

Provider-specific guarded update

Google uses a revision-aware batch update and preserves the final structural newline. HTML Docs rereads the exact region immediately before PATCH and retries once if it changed.

PROVIDER API
Native Docs only

Office compatibility mode is rejected by the Google provider because its structure cannot satisfy the same invariants. HTML Docs additionally refuses documents with no editable text region or missing session/category markers. E-DOCS E-HTML-DOCS

06 / Context lifecycle

The document remains the source of truth.

Context is not a one-time prompt dump. The extension maintains a revision-aware snapshot and injects changes silently so the conversation can respond to edits made by the human. E-APP E-VOICE

0:00 · STARTRead doc, ensure named ranges, inject initial context.
EVERY 5sCompare revision IDs; inject context only when changed.
55:00 · ROLLOVERSummarize with deep think or recent 24 turns, then reconnect.
ENDClose voice/media, summarize recent 20 turns, persist session record.

Large document context is capped at roughly 24,000 characters by keeping the first and last 12,000 characters. Search currently lowercases query terms, scores term containment across current-document paragraphs, and returns the best matches. A ContextProvider interface leaves room for broader sources later. E-VOICE E-EXT-NET

Why the 55-minute rollover?

Realtime credentials and long-lived voice sessions are treated as renewable. Rubber Duck carries forward a compact summary instead of assuming one connection can live forever. E-VOICE

07 / Backend + auth

The server proves identity and protects expensive capabilities.

The Fastify API verifies Google identity, enforces the team allowlist, signs one-hour app JWTs, holds the OpenAI key, mints ephemeral Realtime credentials, rate-limits deeper reasoning, and stores end-of-session records. E-API E-API-AUTH

RouteRoleGuard
GET /healthLiveness response for local or deployed checks.Public
POST /v1/auth/exchangeVerify Google userinfo, enforce allowlist, return app JWT.Google bearer token
POST /v1/realtime/sessionMint a short-lived OpenAI Realtime client secret.App JWT
POST /v1/deep-thinkRun structured synthesis with the stronger reasoning model.App JWT + 12/min limit
POST /v1/sessions/:id/endStore the completed session payload with ownership and expiry.App JWT
DELETE /v1/sessions/:idDelete only the caller’s own stored session.App JWT + owner match

Identity chain

  1. Chrome obtains a Google access token.
  2. API calls Google OIDC userinfo.
  3. Email or domain is checked against policy.
  4. API signs a one-hour HS256 app JWT.

Abuse controls

  • Global limit: 120 requests/minute
  • Deep think: 12 requests/minute
  • Strict CORS origin in production
  • Schema validation on request bodies
  • Owner-scoped session deletion
Credential placement

The extension keeps the user’s Google token plus any HTML Docs share token/account key client-side. The backend alone holds the long-lived OpenAI API key. The browser receives only a short-lived Realtime client credential. E-API-AUTH E-HTML-DOCS E-OPENAI

08 / Data + retention

Shared contracts make the session inspectable.

The shared workspace defines the domain shapes used across browser and server: session states, document snapshots, transcript turns, proposals, categories, tool audits, and persisted session payloads. E-SHARED

Core domain types

  • SessionStatusUI lifecycle
  • DocumentSnapshottext + revision
  • TranscriptTurnspeaker + content
  • EditProposalbefore / after
  • ToolAuditEntrylatency + result
  • SessionRecordretained payload

What is saved at session end

The app submits the session identity, document identity, transcript, proposals, summary, timing metrics, and tool audit. The store adds ownership and expiration. A database URL selects PostgreSQL; otherwise the API uses an in-memory store.

Expired sessions are purged every six hours. The default retention window is 30 days, and the DELETE route lets the authenticated owner remove a record earlier. E-STORE

Privacy boundary in plain English

Live document content is read in the extension and supplied to the voice model as context. Deep-think calls send the relevant question and context to the project API, which calls OpenAI. Session-end records can include transcript and tool-audit data. Google and HTML Docs credentials do not become stored session data, and raw workspace source files are not part of this guide’s published artifact.

The exact deployment policy still matters: production should set the allowlist, CORS origin, secrets, database, and retention values intentionally. E-README E-API

09 / Run + deploy

Local development has two moving parts.

The API and extension run separately. The extension expects the local API unless its build-time API base URL points at a deployed service. E-README

# terminal 1 — backend
npm run dev:api

# terminal 2 — extension build/watch
npm run dev:extension

# project verification
npm test
npm run typecheck
npm run build

Local request path

Chrome loads the unpacked extension. The side panel talks to the API on localhost, while Google Docs or HTML Docs reads/writes and the Realtime WebRTC connection originate from the browser. If the API is down, authentication exchange and Realtime startup cannot complete; the extension surfaces a targeted offline diagnostic. E-EXT-NET

HTML Docs setup

Owner pages use an account API key saved in chrome.storage.local. Share URLs supply a narrower document token automatically. When both exist, reads prefer the document token; a write rejected as read-only retries with the saved account key. The credential is sent only to the active HTML Docs origin and never to the Rubber Duck backend. E-HTML-DOCS E-APP

Cloud Run shape

The provided service manifest deploys the API with scale-to-zero, a maximum of 10 instances, concurrency 40, and a 30-second request timeout. OpenAI, JWT, and database values are injected as secrets; CORS and allowed-domain policy are environment configuration. E-DEPLOY

Deployment handoff

After deployment, rebuild the extension with VITE_API_BASE_URL aimed at the Cloud Run API and make sure the extension’s host permissions and production CORS origin agree.

Current verification footprint

The current suite contains 24 cases across API and extension workspaces. At snapshot time, 23 pass and one HTML Docs credential-fallback test fails: retries a read-only share-link write with a saved account key. Coverage also includes health/auth routes, Realtime credentials, the deep-think contract, storage ownership/expiry, both providers’ URL parsing and write edges, context ranking, and offline API recovery. E-TESTS

10 / Failure map

Most failures have a clear boundary.

The fastest debugging move is to identify which handshake failed: active document, Google auth, project API, Realtime transport, Docs mutation, or extension asset loading.

NO ACTIVE DOCUMENT

The content script did not resolve a Google Docs or HTML Docs document ID, or the active tab is no longer a supported document route. Check the URL and reload the extension after rebuilding.

API OFFLINE / FETCH FAILED

The extension cannot reach the configured Rubber Duck API. Start npm run dev:api locally or verify the deployed base URL and CORS configuration.

DYNAMIC IMPORT CHUNK FAILED

A rebuilt extension can leave an already-open side panel pointing at an old hashed asset. The voice code is now loaded eagerly with stable chunk names, but Chrome still needs an extension reload after a new build.

GOOGLE AUTH REJECTED

Google identity still authenticates the Rubber Duck API even during an HTML Docs session. Check OAuth client configuration, scopes, Chrome identity state, and the email/domain allowlist.

HTML DOCS CREDENTIAL REJECTED

An owner URL needs a saved account key. A share token may be read-only; when an account key is also saved, the client retries a denied write using that key.

PROPOSAL WILL NOT APPLY

The anchor may have changed, disappeared, or become ambiguous since the proposal was created. This is a safety stop; recreate the proposal against the latest document.

OFFICE COMPATIBILITY MODE

Convert the file to a native Google Doc. Rubber Duck intentionally declines writes where its index and named-range invariants are not reliable.

SESSION ENDS AFTER TAB SWITCH

This is deliberate. A live session is bound to one document ID and stops rather than carrying its context and write authority into another doc.

VOICE WORKS, WRITE DOES NOT

Inspect the tool audit and provider error. Conversation transport and document mutation are separate paths, so one can succeed while the other is rejected.

The mutation behaviors above are grounded in both clients’ explicit retries and fail-closed checks. The dynamic-import diagnosis reflects the extension’s bundled-asset lifecycle and its current eager voice import/stable chunk configuration. E-APP E-DOCS E-HTML-DOCS

11 / Honest limits

It feels collaborative, but it does not yet have document-native presence.

The current architecture is an agent operating through provider APIs plus a Chrome side panel. That gives it document awareness and controlled edit authority, but neither client implements cursor, selection, or presence state. Google’s REST API also does not expose the native coauthor-presence protocol. E-ARCH

What it cannot show today

  • A native named collaborator cursor in either provider
  • Live selection highlights owned by the agent
  • Google Docs presence/avatar state
  • HTML Docs agent-presence state
  • True simultaneous character-by-character coauthoring

What is feasible in this architecture

  • Voice and transcript presence in the side panel
  • Tool activity such as “reading,” “thinking,” and “drafting”
  • Before/after previews with explicit approval
  • Controlled auto-notes inside the protected section

Google’s document API exposes content and batch edits, not the internal realtime collaboration surface used for human cursors and selections. A visual “Duck cursor” there could only be simulated as an extension overlay. HTML Docs is a more plausible home for first-class agent presence because its surface and API can evolve together, but the current region API still exposes content edits, not live cursor/selection events. The project deliberately avoids inferring either provider’s selection state from rendered DOM.

Other present boundaries

  • Context search covers the current document only; external workspace sources are not connected.
  • Google support is limited to native Google Docs; HTML Docs owner and share routes are separate supported surfaces.
  • There is no Google Meet integration.
  • The in-memory session store is a development fallback, not durable production storage.
  • The product remains a private-team pilot and needs OAuth, policy, observability, and accessibility hardening before broad release.
The product truth

Rubber Duck can act like a thoughtful second person through voice, memory, visible tool state, and approval-aware edits. Today that person lives in the side panel. A truly native Duck cursor requires a presence API in an owned document surface—not just REST writes.

12 / Source ledger

Every architectural claim has a trail.

This guide was reconciled against a private local workspace snapshot taken at 2026-07-26 02:34:12 UTC. The workspace has no Git HEAD, so file fingerprints—not a commit—freeze the reviewed evidence. Credential files, dependencies, generated output, VCS internals, the lockfile, and minified assets were excluded.

Evidence families and SHA-256 fingerprints
  1. E-README
    README.md · lines 1–167
    Product purpose, two-provider setup, privacy, and pilot limitations.
    sha256 750205ddcd9c2e7c9cb8df11e3c95b9e7042e13c414ccaf65dd79cb117cdb186
  2. E-ARCH
    docs/architecture.md · lines 1–46
    System boundaries, provider trust, document invariants, and context lifecycle.
    sha256 9dd1867f9b7368b13b393ebb5e3d39026cd654f971b08ef2dc4c55b31e6de40f
  3. E-SHARED
    packages/shared/src/index.ts · lines 1–100
    Shared domain types, session states, proposal, transcript, and audit contracts.
    sha256 9da299f3a1c5ac847d8268a56c08293a5b0bc960c15203fcf0dc38bbebaf62a7
  4. E-MANIFEST
    apps/extension/public/manifest.json · lines 1–64
    Chrome permissions, OAuth scopes, provider hosts, side panel, and content scripts.
    sha256 64b81b9def6c99454c278a6a3397c24c5a652f9e7dfa314adec99111164374e1
  5. E-EXT-SHELL
    background.ts · content.ts · messages.ts
    Provider-aware active-document detection, side-panel enablement, and messages.
    sha256 d5fbe582…970637 · 570ceaff…d55a · 0adf948d…34d0d
  6. E-APP
    apps/extension/src/App.tsx · lines 1–687
    Provider selection, HTML Docs credential UI, lifecycle, proposals, polling, and microphone cleanup.
    sha256 10d511f113e757fa030d883a92abd6e6d67ce54dd9b03e591e4ddc8916e297d5
  7. E-PROVIDER
    apps/extension/src/lib/document-client.ts · lines 1–32
    Provider-neutral read, protected-section, note, proposal, and apply contract.
    sha256 a1ba889c1ed54308c3cf2e18b86ae89f985b7709ec9b63b700a55d4ed18af2c4
  8. E-VOICE
    apps/extension/src/lib/voice.ts · lines 1–438
    Provider-neutral Realtime configuration, tool contracts, context, rollover, and audit.
    sha256 57409b734537961883f4c884df0c1fd635f7391513fc916e38073525f34dc087
  9. E-DOCS
    apps/extension/src/lib/google-docs.ts · lines 1–433
    Google Docs parsing, named ranges, revision writes, proposal conflicts, and final-newline handling.
    sha256 39e589cee5a4738f3d78cf842fb5be7aa4748e60dbea2235718dc4fd69ccd129
  10. E-HTML-DOCS
    html-docs.ts · html-docs-auth.ts
    HTML Docs shell/regions, share and account credentials, session markers, exact anchors, and pre-PATCH conflict checks.
    sha256 f1b38f96…3b1679 · dc620917…5b103
  11. E-EXT-NET
    apps/extension/src/lib/auth.ts · api.ts · context.ts
    Token exchange, backend calls, offline diagnostics, and current-document search.
    sha256 b5b36ec2…7792b · 53331dff…d03ac · beeab864…42a26c
  12. E-API
    apps/api/src/app.ts · lines 1–173
    Fastify routes, validation, rate limits, CORS, and retention endpoints.
    sha256 d0de3a5893382fc250cf7722f58e6e4212dacac7c9b8d07cf4e7f2b2ccf2b809
  13. E-API-AUTH
    apps/api/src/auth.ts · lines 1–79
    Google userinfo verification, allowlisting, and one-hour app JWTs.
    sha256 b4c26e23ed77fe279013daec44f7340be45854a7152b4bfbd69b1fb126c85282
  14. E-OPENAI
    apps/api/src/openai.ts · lines 1–89
    Ephemeral Realtime credentials and the deeper reasoning contract.
    sha256 344ed9356d59ad7aec25c5be3ff929320c94cc52055f02863ef6d855f4b74072
  15. E-STORE
    apps/api/src/store.ts · apps/api/src/server.ts
    Memory/Postgres storage, ownership deletion, expiry purge, and server lifecycle.
    sha256 47affec8…6a13 · 9be7b7c4…767e41
  16. E-DEPLOY
    deploy/cloudrun-service.yaml · lines 1–44
    Cloud Run topology, secret injection, autoscaling, concurrency, and timeout.
    sha256 fb7111f1085a365ca35e2465eb3c19dcf01290bd329552137cfa5a025a048457
  17. E-TESTS
    apps/**/*.test.ts · 24 cases
    23 passing and 1 HTML Docs credential-fallback test failing at snapshot time; covers API auth/storage, both provider clients, context, URL parsing, and offline recovery.
    aggregate fingerprint not frozen; individual source files remain local

A machine-readable version of this ledger is stored beside this document as docs/rubber-duck-project.sources.json. Only summarized evidence is embedded here; private source code and credentials are not.