UtkrushtDesign Document

FaceSignals Service — Implementation Plan

Feature: Phase 2 · Offer→Join Identity Continuity (the "selfie verification tool") Design doc: candidate-identity-verification-and-fraud-signals.md · live: https://www.html-docs.com/s/3ab3e4be8c Live (this doc): https://www.html-docs.com/s/c08fb7de01 Status: ✅ Demo shipped 2026-07-15 (Phases A–F done, worked live on dev) · backend PR #650 open (CI green, awaiting review) · next: dev-merge → formal release (see Post-Demo → Production Backlog) Legal gate: ✅ cleared — Utkrusht-legal PR #1 accepted + implemented Scope for now: dev-only (devapp / devrecruiter + dev Supabase) · demo target = a working selfie match for 1–2 real task/test sessions on the dev DB. No prod, no release branch.

What changed vs. the design doc

The design doc wired Phase 2 into fastapi_service (enrolment on the shared schedule_once ThreadPoolExecutor; verify as "a new FastAPI route"). This plan instead builds it as a new, separate service — facesignals_service — in the same repo, following the notifications/ precedent.

Why separate: 1. Native-dep isolation. opencv-python-headless + ONNX model files is the first CV dependency in the backend. Keeping it out of the fastapi_service image means a broken CV build can never block a deploy of the core API (proctoring, tasks, sandboxes). 2. CPU isolation. Face embedding + liveness inference would otherwise share the same pool that runs transcription and Gemini analysis. A separate service isolates that load. 3. Biometric blast radius. "Face embeddings are only ever touched by one small service with service-role-only DB access" is a cleaner boundary for the DPIA/legal story.

The model choice makes this cheap: YuNet (detect) + SFace (embed) run via OpenCV's built-in cv2.FaceDetectorYN / cv2.FaceRecognizerSF, and MiniFASNet (liveness) via cv2.dnn.readNetFromONNX — so the whole stack is one opencv-python-headless wheel (prebuilt aarch64, no source compile) plus ~40 MB of ONNX files.

Architecture

Service shape (mirrors notifications/, but COPYs shared/)

facesignals_service/
├── facesignals.Dockerfile        # mirror notifications.Dockerfile; COPY shared/ + set PYTHONPATH=shared:facesignals_service
├── requirements.txt              # opencv-python-headless, numpy, fastapi, uvicorn, supabase, boto3, pydantic
├── main.py                       # FastAPI app factory (wires routes)
├── gunicorn.conf.py
├── core/                         # HOST-AGNOSTIC — pure functions over image bytes, ZERO web/DB imports
│   ├── detect.py                 # YuNet  (cv2.FaceDetectorYN)
│   ├── embed.py                  # SFace  (cv2.FaceRecognizerSF) → 128-d vector
│   ├── liveness.py               # MiniFASNet (cv2.dnn) → live/spoof score
│   └── match.py                  # cosine + banding (clear / borderline / no-match)
├── models_onnx/                  # vendored ONNX files (YuNet, SFace, MiniFASNet)
├── enrollment.py                 # cover-video → frame → detect+embed → write face_enrollments (uses shared DAOs + S3)
├── routes.py                     # HTTP layer: /links (mint), /links/{token}, /verify
└── static/join.html              # thin join page served by THIS service (webcam selfie → /verify → band)

Shared data layer lives in the backend shared/ tree (reused, not duplicated): - shared/models/face_enrollment.py, shared/models/join_verification.py - shared/daos/face_enrollment_dao.py, shared/daos/join_verification_dao.py

Boundaries — what lives where

Concern Home
Face detect / embed / liveness / banding facesignals_service/core/ (host-agnostic)
Enrolment from cover video facesignals_service/enrollment.py (+ shared/ S3 helpers)
Link mint + verify API + join page facesignals_service (routes.py + static/join.html)
Tables + DAOs/models shared/ + supabase/migrations/ (dev DB)
Recruiter "mint link" button + result chip recruiter-utkrusht — DEFERRED (reuses risk-signal-chips.tsx, depends on PR #288)
Candidate biometric consent item utkrushta-assessment DeviceCheck.tsx — DEFERRED for dev demo

The demo simplification

For tomorrow, facesignals_service serves its own minimal join page. The whole selfie→match loop is one service + the dev DB — no cross-repo frontend, no recruiter typed-client regen, no signed push to the frontends. The polished recruiter chip + candidate consent are post-demo.

Data model (dev DB)

Follows the extension_captures async-job template + report_comments dual-FK/CASCADE precedent. Service-role-only RLS, served only via the facesignals service (never direct anon Supabase). No pgvector (1:1 exact cosine). No raw images stored.

Spec number: 024 (follows 023-ip-intelligence).

Demo scope — deferred / out of scope

Legitimately trimmed because it's a dev-only demo (all called out in the design doc as hardening/launch items): - Recruiter-app integration (mint button, Match/Review/No-match chip, typed-client regen) - Candidate biometric consent screen - Automated 6-month retention purge cron - Per-demographic fairness calibration + threshold tuning - Hardened liveness / deepfake-injection defense - Human-review workflow UI - Anything prod (release branch, prod Supabase, prod Coolify)

§ Implementation TODO — Execution Checklist

Ordered by dependency. 🧑 YOU = needs you (a decision, a signed push, or infra). Everything else is mine.

Milestone A — Prep & decisions

Milestone B — FaceSignals core module (host-agnostic)

Milestone C — Data layer (dev)

Milestone D — Enrolment path

Milestone F — Thin join page (served by facesignals)

Milestone G — Deploy (or run local)

Milestone H — Demo dry run

Deferred (post-demo)

Your action-items at a glance

# You need to When
A1 Confirm the 1–2 demo candidates (with cover video) Before enrolment (D3)
A2 Approve ONNX-in-repo vs git-lfs Before B2
A3 Pick demo host: local-against-dev vs new Coolify app Before G
C4 Sign + push migrations to dev (/db.push) After C3
G2 (If deploying) create Coolify app + env + DNS Milestone G
H1 Join me for the dry run Last

First 3 tasks

  1. A1 — Confirm the demo anchors. I query dev Supabase for 1–2 task/test sessions that already have a cover video recorded; you pick the candidate(s). (Unblocks all enrolment work.)
  2. B1–B4 — Build the host-agnostic core. Scaffold facesignals_service/, vendor the three ONNX models, write detect/embed/liveness/band as pure functions, unit-test locally. (The irreducible engine; no infra needed.)
  3. C1–C4 — Land the data layer on dev. Author the two migrations + DAOs/models, run the /db.push preflight; you push to the dev DB. (Unblocks enrolment persistence + verification.)

Post-Demo → Production Backlog

The demo (A–F) proved the vertical works on dev, driven from a local process against the dev DB. Getting from there to merged-into-dev → formally released means paying back the corners we deliberately cut for speed. Grouped by what unblocks what: [dev] = needed to test on devapp/devrecruiter; [release] = needed before prod.

The only way a verification link is created and handed out is via the recruiter portal (never the open service):

This single flow resolves two things at once: it fixes the "no auth on mint" gap, and the raw-signal-exposure gap — the recruiter gets the raw band/similarity through their authed portal, while the external link-holder only ever gets the softened display. The joiner-vs-recruiter view split falls out of where each party enters the system.

1 · Deploy the service — [dev] (nothing runs on devapp without this)

2 · Recruiter portal integration — [dev] (the flow above; also closes security #1)

3 · Correctness / robustness — [dev→release] (PR #650 review findings)

4 · Compliance & quality — [release] (the load-bearing ones the design doc calls out)

Critical path

Utkrusht — Meaningful Skill Assessments for India