Backend Design Note · Résumé Pipeline

Keeping the raw résumé text, not just the structured JSON

The upload endpoint already extracts clean résumé text on its way to producing structured data — then discards it. Here's where that happens and the cheapest durable place to keep it for reuse.

Scope · upload_resume_portal (Flask :4000) Table · main_fyi_users Status · Recommendation, pre-implementation
RECOMMEND

Add a resume_text text column to main_fyi_users and populate it from the markdown that extract_text() already computes — no re-extraction, no new service.

01 The endpoint & current flow

POST /upload_resume_portal is handled by _upload_resume_portal_core() in flask_service/services/positions_service.py:329. It stores the PDF to S3, then runs a two-stage extraction pipeline whose only persisted output is the structured résumé.

S3
Upload PDF to S3
Key resumes/{uuid}/{filename}s3_url. Kept as resume_link.
TXT
extract_text() the artifact you want
Marker API (fallback MarkItDown) → raw markdown text. Expensive: a network round-trip + PDF download. In shared/pdf/extractor.py:68.
LLM
ResumeStructurer().structure(text)
GPT-4o turns the text into a typed ResumeV1 model → structured JSON.
DB
Persist text is dropped here
main_fyi_user_dao.create({user_id, resume_data, resume_link}) writes only the structured JSON to main_fyi_users.resume_data. The raw text goes out of scope.

The gap

The raw text (text inside extract_structured_content) is computed and thrown away. Every downstream use case that wants plain text today has to re-download the PDF and re-run Marker — paying the expensive step again for a value we already had in memory.

02 Where résumé data lands today

ColumnTypeHolds
resume_datajsonStructured ResumeV1 — name, skills, experience, education…
resume_linktextS3 URL of the original PDF
resume_texttext — proposedRaw extracted markdown, unstructured

main_fyi_users is already the canonical 1:1 record per parsed résumé, holding the structured data and the S3 link side by side and linked to users via users_id. The raw text belongs in the same row.

03 Why a text column, over the alternatives

04 Change points

Only the fact that extract_structured_content() returns just the model — hiding the raw text from its caller — needs addressing. Two clean options:

# shared/pdf/extractor.py — surface the text that already exists
def extract_structured_content(file_path, doc_type):
    text = extract_text(file_path)
    ...
    return result, text        # return (model, raw_text)

# — or — call extract_text() once in the service and pass it into the structurer
#     (avoids a second extraction round-trip)

Then persist it and widen the model:

# flask_service/services/positions_service.py
main_fyi_user_dao.create({
    "user_id":     user_id,
    "resume_data": detailed_parsed_data,
    "resume_link": ...,
    "resume_text": text,        # NEW
})

# shared/models/main_fyi_user.py
resume_text: Optional[str] = None

Plus a migration adding resume_text text to main_fyi_users (written to supabase/migrations/ — file only, not pushed).

05 When to go further

Scale option — content-addressed table

If you need to dedup re-uploads of the same PDF or re-run new structurers on old résumés without re-extracting, key a small resume_extractions table by file SHA-256:

(file_sha256, raw_text, extractor, extractor_version, created_at)

Consumers reference it by hash. This matches the hashing pattern the résumé-fit pipeline already introduced on applications. It's more machinery, so reach for it only when dedup / versioning is a real requirement — the column can be promoted to this later.