PDF Support Layer

Technical Specification for html-docs.com — v1.0, May 25 2026

01Architecture Overview

Key Design Decisions

• Upload-first, process-later. The client uploads the raw PDF to Supabase Storage via a lightweight API route, gets back a job ID and placeholder document ID immediately. Processing happens in a separate long-running function (Vercel maxDuration: 300s).
• Page-by-page vision. Each PDF page is rendered to a high-DPI PNG (2x scale, ~1400x1800px for letter-size), then sent to GPT-4o vision with a prompt that asks for faithful HTML reproduction of that single page. Pages are processed in parallel (batches of 4) to stay within rate limits while maximizing throughput.
• Stitching pass. Individual page HTMLs are merged into a single cohesive document with consistent styling, cross-references, and a unified stylesheet. A final LLM pass normalizes the CSS and resolves any inter-page continuity issues (split paragraphs, table continuations).
• Progress polling. Client polls a status endpoint every 2 seconds. Status includes: current step name, pages processed / total pages, estimated time remaining. No fake timers.

02PDF Page Rendering

// lib/pdf/render.ts
import { createCanvas } from '@napi-rs/canvas'

async function renderPage(page: PDFPageProxy, scale: number = 2.0): Promise<Buffer> {
  const viewport = page.getViewport({ scale })
  const canvas = createCanvas(viewport.width, viewport.height)
  const context = canvas.getContext('2d')

  await page.render({ canvasContext: context, viewport }).promise

  return canvas.toBuffer('image/png')
}

Page Size Detection

Page dimensions are read from page.getViewport() and stored per-page. The final HTML preserves these dimensions via CSS @page rules and max-width on the content container, so the document looks correct at all screen sizes and prints accurately.

03Vision-Based HTML Reproduction

System: You reproduce PDF pages as faithful, semantic HTML+CSS.
Your output must visually match the source image as closely as possible.

Rules:
- Use semantic HTML: h1-h6, p, table, figure, blockquote, ul/ol, etc.
- Reproduce the exact visual layout: column count, spacing, alignment
- Preserve all text verbatim -- do not summarize, rewrite, or omit
- For math/equations: use inline SVG or HTML entity approximations
- For figures: output a <figure> placeholder with [FIGURE_N] marker
  and the caption text. Figures will be injected in post-processing.
- Tables must reproduce the exact cell structure and content
- Use CSS Grid or Flexbox for multi-column layouts
- All styles must be inline or in a single <style> block
- Typography: match the source font style (serif/sans), size ratios,
  weight, and color as closely as possible
- No external dependencies. No JavaScript. No remote fonts.

User: [page image attached]
Page {N} of {total}. Page dimensions: {width}pt x {height}pt.
{extracted_text_for_reference}

Multi-Column Layout Strategy

Academic papers commonly use two-column layouts. The vision model is instructed to detect and reproduce these using CSS Grid:

.two-column {
  display: grid;
  grid-template-columns: 1fr 1fr;
  gap: 1.5rem;
  max-width: 900px;
}
@media (max-width: 720px) {
  .two-column { grid-template-columns: 1fr; }
}

The stitching pass verifies column detection is consistent across pages and normalizes the grid definitions.

Table Extraction

Tables are reproduced by the vision model directly from the page image. The extracted text serves as a validation layer -- if the vision model's table content doesn't match the text extraction, the system flags it for a second pass. Complex tables (merged cells, nested headers) are handled by the vision model's spatial understanding of the image rather than programmatic detection.

Math and Equations

Three-tier approach:

1. Simple inline math: HTML entities and Unicode (superscripts, subscripts, Greek letters, operators)
2. Display equations: Inline SVG generated by the vision model, reproducing the visual appearance
3. Complex notation: The page image section containing the equation is cropped and embedded as an <img> with appropriate alt text

Future: Add MathML or KaTeX rendering as a post-processing option when the user opts in.

04Figure and Image Handling

05Processing Pipeline (Detailed)

Background Processing Architecture

On Vercel, the cleanest approach is a dedicated API route with maxDuration: 300 that acts as the processing worker. The flow:

1. Client calls importPdfDocument server action, which uploads the PDF to Storage and creates a job record in a pdf_import_jobs table (Supabase), then returns immediately.
2. Server action fires a fetch() to the internal processing route (/api/pdf/process) with the job ID. This is fire-and-forget -- the server action doesn't await the response.
3. Processing route does the heavy work (render, vision, stitch, import). Updates the job record at each step.
4. Client polls /api/pdf/status?jobId=X every 2 seconds.

// Supabase table: pdf_import_jobs
{
  id: uuid (PK),
  user_id: uuid (FK -> auth.users),
  document_id: uuid | null,     // set when complete
  storage_path: text,           // path in Supabase Storage
  status: text,                 // 'pending' | 'rendering' | 'converting' | 'stitching' | 'complete' | 'failed'
  current_step: text,           // human-readable status
  pages_total: int,
  pages_processed: int,
  error: text | null,
  created_at: timestamptz,
  updated_at: timestamptz
}

06Scanned PDF / OCR Fallback

07PDF Export (HTML to PDF)

08Error Handling and Resilience

• Function timeout: Processing route has maxDuration: 300 (5 minutes). If it times out, the job record is left in its last-known state. A cleanup cron marks stale jobs (>10 min old, not complete) as failed.
• Vision API failures: Individual page failures are retried once. If a page fails twice, it falls back to the text-extraction redesign for that page only. Other pages still use vision output.
• Corrupt PDFs: pdfjs parse failure returns a clear error. Partially corrupt PDFs (some pages readable) process the good pages and mark the rest as "[Page could not be processed]".
• Memory pressure: Pages are processed and flushed sequentially to keep peak memory low. PNGs are not all held in memory simultaneously.
• Rate limiting: GPT-4o vision has per-minute rate limits. The batch-of-4 parallelism with backoff handles this. Very long documents (50+ pages) may hit rate limits and take longer.

09Migration Path

10Cost Analysis

Roughly 2-5x the cost of the current text-only redesign, but the quality improvement is dramatic. For a product that positions itself as "the way agents share documents," this is table-stakes investment.

11File Structure

lib/pdf/
  extract.ts        # existing -- text + image extraction (keep as fallback)
  render.ts         # NEW -- render PDF pages to PNG via pdfjs + @napi-rs/canvas
  vision-convert.ts # NEW -- send page images to GPT-4o, get back HTML
  stitch.ts         # NEW -- merge per-page HTMLs into cohesive document
  types.ts          # NEW -- shared types for the PDF pipeline

app/api/pdf/
  process/route.ts  # NEW (Phase 3) -- background processing worker
  status/route.ts   # NEW (Phase 3) -- job status polling endpoint

scripts/sql/
  pdf_import_jobs.sql # NEW (Phase 3) -- job tracking table

Spec authored by Amika for html-docs.com. May 25, 2026. Implementation begins with the immediate crash fix, followed by Phase 2 vision pipeline.