Résumé Fit — architecture

1The two halves

The whole design rests on one seam: a pure domain core that knows nothing about infra, and a thin adapter that delivers it. Dependencies point inward — the adapter depends on the core, never the reverse.

3How the fit score works

Three questions: which signals, how they make a 0–100 score, and what reaches the report. The model is a scored skill spine with two adjusters, plus two company signals that never hurt a candidate.

The signals

Because skill_match is contextual, not lexical (“built ETL in Python” matches; “Python” in a skills list doesn’t), a separate relevance signal is redundant. evidence and recency stay their own signals — each has its own confidence, tuning knob, and failure mode (a skill can match yet be thinly-backed or stale) — but as × adjusters they only ever qualify a match, so they share the one Skills chip rather than earning their own. Guardrails: missing ≠ negative, Unknown ≠ Low, a standard degree earns nothing.

The formula

fit_score = skill_match × evidence_adj × recency_adj × 100  +  pedigree_bonus
#             skills fit    backed?       current?              opt-in, capped, ≥0

Example — QA role, 3 competencies; Playwright + Selenium match strongly, API testing missed:

0.82 × 1.00 × 0.94 × 100 = 77 → Strong (fit ≥ 75). The card shows Skills 2/3 matched; the hover carries the per-skill evidence + recency and the API-testing miss.

Every number lives in one config file

The model returns categories it can judge reliably (an evidence tier, a recency bucket); config maps each to a factor. So no number is hallucinated — each is a seeded default, then calibrated from the annotation sheet (§9). That is the whole answer to “where does the 0.85 come from”:

# resume_fit/config.py — every tunable number in ONE place
MODELS = {"extract": "cheap-tier",   # résumé → facts, once per résumé
          "score":   "strong-tier"}  # contextual match + confidence

# evidence tier → × adjuster  (≤1: a miss, never a negative)
EVIDENCE_ADJ = {"demonstrated": 1.00, "in_context": 0.92,
                "bare_mention": 0.85, "unsupported": 0.70}
# recency bucket → × adjuster  (freshest wins)
RECENCY_ADJ  = {"this_year": 1.00, "yrs_1_2": 0.94,
                "yrs_3_5": 0.85, "yrs_5_plus": 0.72}

# skill_match aggregation
MISS_PENALTY   = 0.09   # per unmatched required competency
PARTIAL_CREDIT = 0.6    # a partial match vs. a full one

# company signals
LOYALTY_WEIGHT = 0      # informational only — never moves fit
PEDIGREE = {"enabled": False, "bonus_cap": 5,  # opt-in, additive, ≥0
            "tier1": 5, "tier2": 3}

# confidence gate + buckets (0–100)
CONFIDENCE_MIN = "high"                    # only high-conf signals score/show
BUCKETS = {"strong": 75, "potential": 55}  # calibrated, not legacy 85/70
SCORING_VERSION = "v1"                     # bump on change; old results pinned

What reaches the report

Each signal carries show ∈ {chip, detail, hidden} and a derived confidence (evidence spans + rubric, never self-reported). Two gates: only high-confidence signals score & show; and if skill_match itself isn’t high-confidence → “Couldn’t assess”, never a low score.

2What the recruiter sees

One run's output, rendered from fit_result alone — no separate report, no modal. Both surfaces show the same lean chip set: a bucket pill + one aggregate Skills n/m matched chip, reusing the chip shape already in the product (e.g. the Location differs from profile chip). Per-skill detail lives in the hover, never as separate chips.

Position page — ranked session cards

Proposed on the real cards. Each card leads with the bucket pill (● Strong fit / ◐ Potential fit / Weak), then one Skills n/m matched chip (n competencies of m), then any informational chip (e.g. 4 yrs · 1 company loyalty — weight 0). The list sorts Strong→Weak. (Names are placeholders.)

Candidate report — chips on the profile card, per-skill detail on hover

Proposed on the real report (redacted). The bucket pill + Skills 2/3 matched chip sit in the profile-card tag row beside the existing location chip. Hovering the Skills chip expands the aggregate into the per-skill breakdown — each matched competency with its relevance (strong/partial), recency (year used), and an evidence quote, plus the misses and the derived confidence.

4Delivery: Airflow + Fargate

The decision rule, and where Résumé Fit lands:

Résumé Fit scores up to 500+ résumés in the background — heavy parallel LLM. That's task_analysis-shaped, not task-builder-shaped. The team convention is "all LLM work on Fargate," and only Fargate lets a big position (a) not hog app/worker servers and (b) scale out instead of draining on one worker. So: an Airflow DAG that fans scoring out to Fargate (mirroring task_analysis_fargate) — prepare → map (N Fargate) → reduce → notify.

Trade-offs recorded — Airflow + Fargate (chosen) vs a DB queue-runner

Net: background + heavy-parallel-LLM + the "LLM on Fargate" convention → Airflow + Fargate. A queue-runner would win only for interactive, low-latency LLM (task-builder), which this isn't. (Earlier drafts leaned queue-worker by mis-analogizing to task-builder; the 500-scale requirement + Fargate convention correct that — a stable classification.)

5Extraction vs scoring — the caching split

The single most important efficiency: separate the expensive, position-independent extraction (cached forever by resume_hash) from the per-position scoring (re-runs on any context change). Changing competencies re-scores but never re-extracts.

6Data model & migrations

Additive columns only — no new table. Airflow is the queue (run state lives in the DAG run, not the DB); the result is a blob, not a per-signal child table.

Why no fit_jobs table

Airflow is the queue. Run state, retries, per-task progress, and manual re-trigger all live in Airflow — so there's no claim/sweeper/coalesce table to build. Idempotency comes from the DAG run id "{position_id}:{ctx_hash}" and the fit_context_hash stale-set on applications. (A DB queue-runner would need a fit_jobs/fit_items table for exactly this — one of the trade-offs in §2.)

Columns & when they fill

Staleness key: fit_context_hash = hash(competencies + experience + description + evaluation_context). Any of those changing flips it → the stale-set query (fit_result IS NULL OR fit_context_hash ≠ current OR scoring_version ≠ current) re-scores exactly the affected rows and nothing else. The two clauses split the job cleanly: the hash catches context edits, scoring_version catches formula changes — together they are the full “is this result current?” test.

fit_context_hash is the convergence mechanism, not the run id. It records which context version each result was scored against, so under let-finish a soon-stale in-flight run and a fresh run coexist with no locks: the old run’s rows carry the old hash and are simply re-scored by the next run — that is what makes the batch converge to “every row at the latest context.” Because reverting the context reproduces an old hash, it must not key the dag_run_id — a reverted hash would collide with an already-finished run and silently skip newly-added candidates. So the run id is a plain unique handle, the Airflow pool enforces ≤1 run/position, and idempotency comes from prepare finding an empty stale-set (the data is the source of truth, not the run id).

Total & per-signal scores both live in the blob. The total is fit_result.fit_score; each signal keeps its own fit_result.signals[].score (+ confidence + evidence) in the same JSONB. They’re one blob rather than columns because the signal set is dynamic — the Registry adds/removes signals, so a fixed column per signal would need a migration each time; scoring_version records which formula produced the numbers. Trade: the DB can’t ORDER BY fit_score or filter by bucket cheaply. Fine today — ranking happens in reduce (in memory over the batch) and the report reads the blob directly — but if DB-level sort/filter is ever needed (a leaderboard query, “Strong first” in SQL), promote fit_score (and maybe bucket) to a generated, indexed column extracted from the blob.

Lifecycle

Migration mechanics: additive nullable columns on applications+positions only (inherit table RLS) — no new table. One file, forward-only.

7Concurrency & re-runs

What happens when events overlap — you add résumés (a run starts), then change competencies or add more before it finishes. Under Airflow + Fargate:

No mid-run cancel — chosen for simplicity. We’re not at the scale where a superseded run’s wasted scoring matters, so we drop the whole cancellation mechanism (no cooperative ctx_hash check, no ECS StopTask): the in-flight run just finishes and the new context runs after it. The only guard kept is the one-line notify-suppression above. Idempotency doesn’t depend on this choice — it comes entirely from the content-addressed stale-set + per-candidate upserts (the data is the checkpoint; there is no run cursor to lose), so cancel-vs-let-finish is a wasted-compute trade, not a correctness one. Adding candidates (no ctx_hash change) never cancels anything either.

Overlapping events — one flow

With let-finish, a second event during a run is always the same story: the running one finishes, a queued run follows, one ping at the end. The trigger only changes what the queued run re-scores — and whether the first run notifies.

Only cost is wasted compute (run1 finishing against a soon-stale context) — accepted at current scale. Correctness is independent of it: the stale-set (keyed on fit_context_hash) makes the system converge — run1’s rows are just stale and re-scored by run2; per-candidate upserts → no corruption. The recruiter sees "in progress," then one "ready" with the latest context.

8Module skeleton

# domain core — pure, unit-testable, imports NO db/Airflow/Fargate
resume_fit/
  run.py          # run(position_id, reason) — the one entrypoint
  config.py       # model tiers · 12 calibration knobs (defaults) · scoring_version
  schemas.py      # Signal · SignalResult · EvalContext · ResumeFacts · FitResult
  extract.py      # résumé → ResumeFacts (cached by resume_hash)
  registry.py     # @register_signal + active(ctx) selection
  formula.py      # ScoringFormula + SkillSpineFormula
  confidence.py   # evidence-grounded + rubric-derived; optional verify
  rank.py         # aggregate · coverage tiers · buckets · refusal pile
  persist.py      # DAO writes (facts cache · signals[] blob · fit_context_hash)
  signals/
    skill_match.py · evidence.py · recency.py
    company_loyalty.py · company_pedigree.py

# delivery adapter — depends on the core, not vice-versa
airflow/dags/resume_fit_dag.py            # prepare → map (EcsRunTask × N) → reduce → notify; pool=position
airflow/dags/resume_fit_runner/           # Fargate: core.py (calls run() per shard) · runner.py · Dockerfile
flask_service/…                           # add-candidates / position-update paths call trigger_dag(...)
supabase/migrations/<ts>_resume_fit.sql    # applications.* cols + positions.evaluation_context (NO fit_jobs)

9Adding a signal / changing the formula

The extensibility guarantee, concretely. Two tiny surfaces; nothing else moves.

# registry.py — ADD A SIGNAL = one class + one decorator
class Signal(Protocol):
    key: str
    def active(self, ctx: EvalContext) -> bool: ...          # in play for this position?
    def evaluate(self, facts: ResumeFacts, ctx) -> SignalResult: ...

@register_signal
class SkillMatch(Signal):
    key = "skill_match"
    def active(self, ctx): return True
    def evaluate(self, facts, ctx): ...

# formula.py — CHANGE SCORING = swap the object, bump scoring_version
class ScoringFormula(Protocol):
    def combine(self, results: list[SignalResult], ctx) -> FitResult: ...

class SkillSpineFormula(ScoringFormula):   # skill × evidence_adj × recency_adj + bonus
    ...

Tuning the config knobs — where the annotation sheet comes in

The 12 tunable numbers (confidence threshold, band cutoffs, adjuster factors, …) live in config.py with defaults, so the agent runs immediately. To replace a default with a measured value you fill the annotation sheet (specs/024-resume-fit/eval-annotation-sheet.md) — it is not a runtime input, only a calibration tool:

So the loop is sheet → run agent on the same résumés → offline compare → knob values → config.py. The recruiter fills columns blind first (before seeing the agent's answer), then you compare. Details + the full 12-knob list: eval-annotation-sheet.md and decision-log.md → Calibration knobs.