One repo, two processes

00Verdict

01The real question(s)

"Separate service vs monolith" bundles two independent decisions that deserve separate answers:

Almost all of the pain being felt — same DAO logic in both repos, drift, double maintenance — is repo-boundary pain. None of it requires collapsing the process. Utkrushta itself already proves this: it is one repo shipping four independent runtimes (Flask :4000, FastAPI :9000, notifications :5000, Airflow) off one shared data layer.

02Current state — and the hidden coupling

The two repos look independent. They aren't. The most load-bearing fact found in this investigation:

This is the worst version of a service boundary: the deployment is split, but the data model is shared with no enforcement. A real service boundary would own its schema and expose an API. What exists today is a monolith's data layer spread across two repos — which is why it hurts.

03The duplication inventory

Concrete, file-level duplication found between the repos — this is the maintenance tax being paid today:

04Why merge the repo

• Schema and writers reunited. The migrations already live in Utkrushta. Moving the writer next to them makes every schema change a single reviewable PR that updates the migration, the DAO, and all consumers atomically.
• The DAO layer already exists — it just isn't being used. ConversationDAO, GenerationJobDAO, GeneratedScenarioDAO, TemplateDAO are the only new classes needed; task_dao and competency_dao are adopted as-is. The 8 copies of create_client() collapse into BaseDAO.
• The monorepo pattern is proven in-house. Four runtimes already share shared/ via PYTHONPATH, each with its own Dockerfile and a path-triggered CI workflow (fastapi_service/** changes build only the FastAPI image). Adding a fifth service directory is following the paved road, not paving a new one.
• One review culture, one import-safety net. Utkrushta's CI import tests and its "no raw Supabase outside DAOs" rule start covering task generation for free.
• Small team economics. Two repos means two dependency updates, two CI setups, two sets of conventions, two places to grep. Nothing about the task builder's domain justifies that overhead — it shares the product's core tables.

05Why not merge the process

Folding the Task Builder into the Flask or FastAPI process would be a category error. Its workload is nothing like a request/response API:

The decision, side by side

06Target state

07What moves, what doesn't

08Input files — a database row, not a database cluster

What they actually are

297 files (2.7 MB) across ~82 competency directories: a competency_*.json + background_forQuestions_*.json pair per competency. They are not hand-authored source — pipeline stage 01 generates them from the Supabase competencies table plus one OpenAI call, writes them to disk, and stages 02/04 read them back by absolute path. Humans occasionally hand-tune a single field (minutes_range) via git commits.

Options compared

Proposed table

create table competency_input_files (
  competency_id    uuid references competencies,
  proficiency      text not null,
  competency_json  jsonb not null,   -- was competency_*.json
  background_json  jsonb not null,   -- was background_forQuestions_*.json
  source_hash      text,             -- hash of the competency row it derives from
  generated_at     timestamptz not null default now(),
  edited_by        text,             -- preserves the human-tuning workflow
  primary key (competency_id, proficiency)
);

Stage 01 becomes an upsert-if-stale (compare source_hash); stages 02/04 read from the DB — or, as a zero-risk first step, a thin shim materializes temp files so the subprocess interfaces don't change at all. data/generated/ then leaves git entirely.

Deliberately unchanged: task_generation_prompts/ stays as code. Prompt libraries are reviewed, diffed, and reasoned about like source — that's a feature, not a smell. Only the machine-generated cache moves to the DB.

09Migration plan

1. Ship the recruiter MVP against the existing service — in parallel. The feature targets the stable HTTP contract; do not block it on this migration.
2. Move the runtime slice into Utkrushta/task_builder_service/ with its own Dockerfile, requirements, and a path-triggered CI workflow cloned from the notifications pattern. Goal: deploys from the monorepo with zero behavior change. Same TASK_BUILDER_URL, same INTERNAL_PROXY_TOKEN.
3. Adopt the shared data layer. Add ConversationDAO, GenerationJobDAO, GeneratedScenarioDAO, TemplateDAO (+ models); switch the pipeline to the existing task_dao/competency_dao; delete the 8 create_client() copies. Merge github_utils into one shared module.
4. Move input files to competency_input_files (upsert-if-stale in stage 01, DB reads or temp-file shim in stages 02/04). Remove data/generated/ from git; keep 98 MB of task artifacts in S3/GitHub where they already end up.
5. Align Python and archive. Settle on one version (likely 3.12 — verify dspy and claude-agent-sdk first), then archive ngm9/utkrusht-task with a pointer to the new home.
6. (Later, cheap once co-located) Harden the job model: a worker that polls the existing generation_jobs table so in-flight runs survive restarts. The table is already there; only the runner moves out-of-process.

10Risks & caveats — stated honestly

AEvidence appendix

Every claim above traces to files read in both repos on 4 Jul 2026:

• Utkrushta/supabase/migrations/20260530091150_create_conversations.sql — header: "Replaces an in-memory SESSIONS dict in task_builder/server.py"; siblings create generation_jobs · generated_scenarios · templates · task_template_match
• utkrusht-task/task_builder/conversation_repo.py:8 — "The shape mirrors the conversations migration in the Utkrushta backend repo"
• Utkrushta/shared/daos/ — ~45 DAOs incl. task_dao.py · competency_dao.py · base.py; ~40 models in shared/models/; rule & exceptions documented in Utkrushta/CLAUDE.md
• utkrusht-task/generators/task/persistence.py · generators/input_files/generator.py · generators/prompts/db_queries.py · generators/scenarios/repository.py · infra/e2b/supabase_helpers.py · gist_manager.py · trace_ui/server.py · task_agent_preflight.py — the 8+ raw create_client() sites
• utkrusht-task/task_builder/jobs.py — daemon-thread runner, BoundedSemaphore(3), "no cross-process queue" consolidation note; task_builder/runner.py — 5 subprocess stages, stdout scraping
• utkrusht-task/infra/github_utils.py (247 LOC) vs Utkrushta/fastapi_service/github_utils.py (~1,000 LOC) — identical signatures
• utkrusht-task/data/generated/input_files/ — 297 files / 2.7 MB / ~82 competency dirs; paths hard-coded in generators/input_files/generator.py:41 and run_pipeline.py:53; human-tuning note at generator.py:555
• utkrusht-task/task_builder/Dockerfile — python:3.13-slim; Utkrushta service images on 3.11/3.12; notifications precedent: Utkrushta/notifications/ (own Dockerfile, 21 deps, own db layer)
• Scale: Utkrushta ≈ shared 30k · flask 21k · fastapi 13k · airflow 37k LOC; utkrusht-task runtime ≈ task_builder 1.5k · generators 11k · infra 7k · prompts 32k · task_artifacts 98 MB (generated)

Utkrusht · "Task Builder — one repo, two processes" · companion to Create a Task via Chat, from a Position. This document changes no application code; it records the architecture decision and the migration plan for the follow-up implementation.