Immigr8's core product flows and the technical worker process built to solve each one. Ground-truth as of the live system (owned async evaluation pipeline went end-to-end live 2026-06-13).
A staff evaluator opens a submission and clicks “Run Immigr8 Evaluation” → POST /submissions/:id/evaluate. Gated by a global kill-switch (EVAL_PIPELINE_ENABLED) and a per-country allow-list (EVAL_ENABLED_COUNTRIES). It inserts one row into the evaluation_jobs Postgres queue with status queued. A partial-unique index guarantees only one active job per submission — a re-run while a draft is in flight returns the existing job (202 deduped).
The immigr8-worker — a separate Render process running the same apps/api tree (node dist/worker.js) — polls every 5 seconds. It claims the oldest queued job with SELECT … FOR UPDATE SKIP LOCKED, flips it to running, stamps claimed_at and bumps attempts. Jobs stuck running > 10 min are reclaimed to queued; after 3 attempts a job goes failed. A health check emits structured warnings on failure spikes or queue backlog.
For each job, the worker POSTs an r2Key to a Cloudflare Container running PaddleOCR (PP-StructureV3, Apache-2.0) at EXTRACT_ENDPOINT_URL. The container pulls the transcript bytes straight from R2 (immigr8-transcripts, free same-network egress) and returns ExtractedCourse[] — { foreignCourse, grade, yearTaken, gradeLevelHint, ocrConfidence }. The reader is always-warm since 2026-07-14 (IMMIGR8-724): a 24/7 cron keeps the container hot in prod and staging, so evals never begin with the old ~150 s cold start — the 240 s timeout + provisioning-503 retry ladder survives as defense-in-depth for the ~2-minute re-warm window after a deploy or Cloudflare eviction. Multiple documents are merged + de-duplicated; supporting docs are skipped.
Commodity, swappable: reading is not the IP. The earlier Modal plan was superseded by the owned Cloudflare Container (decision 2026-06-08).
evaluateTranscriptDeterministic() maps the structured courses to US equivalents with no LLM in the loop. Per course it consults a tiered source of truth:
@immigr8/shared) — Sandy's curated per-subject mappings + grading scales for the curated countries. → sourceOfTruth=manual, confidence 90.sourceOfTruth=ai_generic, confidence 20, “specialist mapping required.”Then: grade conversion via the Manual's scale bands; credit rules (a course at the student's current grade = 0 credit / in_progress_us, else 1.0 split 0.5 fall + 0.5 spring); STAAR eligibility; and grade placement = highest completed grade + 1.
Sandy's roll-down rule is live (2026-07-14, IMMIGR8-711/722): course ladders — math and LOTE levels — anchor on which records exist, not the row's grade number. Where earlier HS years are absent, the ladder rolls down so the earliest documented year carries the first rung (a 2-year LOTE record is Levels I/II, never II/III), rung 1 is the lowest Manual cell carrying TWEDS HS credit, and unresolvable Manual prose comes back low-confidence + unmapped rather than dressed up as a 90-confidence award (708). A miss beats a wrong award; it does not beat a right one.
Engine v35 is on production (promoted 2026-09-02 at 09b73e2): a spurious-award guard (IMMIGR8-944) tightens award folding for ladder countries — Spain and South Korea records no longer mint an award the documents don't support — and the supported-country set is 38 countries, consistent everywhere (engine SUPPORTED_COUNTRIES, landing copy, and the live /api/stats/public all agree — IMMIGR8-951). The promote put three weeks of accumulated engine + platform work live in one verified step.
A 15-rule validator (IMMIGR8-354) scores every mapped row. BLOCK rows are dropped (grade out of 1–12 range, future year, ascending-grade/year inversion, duplicate course, malformed TWEDS code…); WARN rows are kept with flags attached (low_confidence, low_ocr_confidence < 0.6, missing source quote…). The validator is a second safety net regardless of which engine produced the draft.
persistAiDraft() discards any prior isAiDraft=true rows (a specialist's hand-entered rows are never touched), sanitizes every field to satisfy the DB CHECK constraints, then writes course_mappings + evaluations with isAiDraft=true plus the AI metadata: aiConfidence, validationFlags, aiProvenance, sourceOfTruth.
POST /submissions/:id/approve-ai-draft pre-flight-blocks approval if any row still carries a flag or confidence < 0.70 (unless a platform_admin forces it). On approval it flips isAiDraft true → false on every row + the evaluation. After the flip the record is operationally identical to a hand-typed one; the AI-metadata columns are retained for audit and Path-3 training capture.
/ai-analyze) still exists and writes through the same persistAiDraft(), but the deterministic engine is the live owned path. Losing an API key can never break the core evaluator. (Decision 2026-06-05.)submissionsA district uploads a transcript for Immigr8 staff to evaluate. Status workflow: submitted → assigned → in_review → completed. This is the active path that the Flow 1 pipeline runs on today — evaluation_jobs.submission_id points here.
transcriptReviewsA district user reviews a transcript independently in the portal. Simpler status: draft → in_progress → completed. The table exists and is wired, but is dormant today; the owned pipeline writes through the submissions path. (Per the reconciliation ADR, the reviews path is untouched.)
This is where the volume actually is: 6,439 of 6,470 production submissions arrived this way, because the legacy WordPress form at immigr8.net is still the live intake and will be until Jake calls the cutover. A LaunchAgent on the iMac pulls new Form-1 entries nightly and inserts them directly, keyed on metadata.wp.entryId so re-runs no-op.
It is a MIRROR, and it contacts nobody — never the create route (which would email the district contact and notify every platform_admin, so a 2024 record could email a district in 2026) and never the entitlement path (which would burn a real $75 evaluation credit per record). Records whose identity the matcher cannot resolve uniquely are left for a human rather than guessed, because a wrong match creates a duplicate student.
If it stops, you will be told. The drip runs on one machine, so a watchdog on Render — deliberately not on that machine — watches how stale the newest mirrored record is and pages #immigr8-updates. Intake swings 8.4× between the June trough and the July peak, so its threshold scales to the recent rate instead of a fixed number of days.
immigr8-transcripts, 50 MB max, 1-hour presigned URLs) and all eventually produce a course_mappings + evaluations pair.One submission → N course_mappings + one evaluation. There is deliberately no separate “AI evaluations” table — that would fork every downstream feature into two code paths.
Manual entry calls createForSubmission / upsertForSubmission with isAiDraft:false. The bot calls the identical repos with isAiDraft:true + the AI-metadata columns. The only difference between the two records is that flag.
approveAiDraftForSubmission() sets isAiDraft=false on every draft row + the evaluation. Safety invariants: discard only deletes isAiDraft=true rows (specialist work is safe), the enqueue endpoint dedupes an active job (no race), and AI metadata is retained post-approval for the audit trail + the Path-3 training set.
findOrCreateStudent() runs as a submission is created. A strong match (first-word-of-first-name + last name + exact date of birth) reuses the existing record; a weak match (fuzzy name + DOB within tolerance) is flagged for human review; no match creates a new students row. The chosen path is captured in the activity log with a confidence + source.
Matching never crosses districts. Cross-district student linkage requires a platform_admin action + an audit-log entry — the same RLS posture as submissions.
A pure scoring function (scoreStudentPair) powers a duplicates view that suggests merges; a specialist confirms each one. The dry-run backfill of historical duplicates is human-gated before --apply.
Stripe fires checkout.session.completed → POST /billing/webhook (signature-verified). The Stripe session id is stored for idempotency so a re-delivered webhook can't double-provision.
The handler creates a districts row (name, size band, plan tier — always full_service since the 2026-07-20 relaunch, subscriptionStatus=active, state, contact) and a users row as district_admin. Campus-level signup was retired 2026-07-21 (the route answers 410); signup is district-only, one marketed plan, with a required student-enrollment field validated against the selected size band. Both creations are written to the activity log. ⚠️ Known gap tracked as IMMIGR8-840: provisioning does not yet set annualPlanActive, so a paid annual signup lands METERED until that child of epic 838 ships.
scheduleWelcomeSequence() queues the branded onboarding emails (SendGrid). No human touches the account between payment and a usable login.
A district is UNLIMITED (active annual plan — usage recorded as amount-0 ledger rows, never blocked, counter shows “Unlimited”) or METERED (bought 5/10-packs — packs never expire; at zero, a Stripe buy-more path appears in the submit flow itself). Mode comes ONLY from the explicit districts.annual_plan_active flag (audited admin flip or Stripe provisioning — never inferred from the untrusted WordPress-era tier columns). All 214 districts start METERED with enforcement off; Client Admins see per-user usage for the plan year (July 1 → June 30, UTC — PROPOSED ADR). The old 18-number allowance grid is deleted.
HQ_COMMS_MODE=live); the first campaign (OUTREACH-2026-09) is approved and starts 2026-09-08.
Candidate districts are harvested and sized with free reads only, then the whole run is approved behind a single gate (ONE-GATE: one Jake go releases the sized universe, nothing spends before it). A detached runner enriches in batches while a watchdog enforces an armed auto-halt on quality regression. September run: 3,096/3,096 contacts enriched across 7 states for 1,477 credits — the watchdog never fired.
hq_conference_leadsEvery enriched contact passes a delivery bar before it may enter HQ: three-tier email validation (exact district domain = tier A, corroborated alternate = tier B, everything else quarantined), name↔localpart assertion with accent folding, cross-institution domain checks, same-person merge. The importer enforces ALL-campaigns dedup and hard preconditions (a missing column aborts with zero writes); deletion-suppression is preserved. Live table: 3,308 leads (457 from SEAMLESS-2026-08 + 2,851 from -09), all landing queued — import never contacts anyone.
The sequencer sends the approved, byte-locked copy (3 touches; segment variants for win-back / Rhithm-relationship / prior-contact / registrar; CAN-SPAM footer on every touch) on a deliverability-first ramp: 60/day warmup → 100 → 150 cruise, 8am–5pm recipient-local weekdays, touch 2 at +5 business days, touch 3 at +7 more. Guards: a go-date interlock (live mode refuses before 9/8 — proven), a per-district daily cap (max 2 contacts/district/day, spill to next day), PAUSE file/env, and auto-pause when the bounce readback crosses ≥2 AND >3%. Every send goes through the HQ comms gate — suppression, bounce readback, deletion-suppression and the 45s/80-per-hour throttle enforce regardless of the plan.
Calendar note: 2026-09-08 is a Tuesday (Labor Day is 9/7), so week 1 has four send days; the approved DATE stands.
A daily intake script processes the mailbox reality back into the system: replies pull the person out of the sequence (positives to Jake, any “no” → permanent suppression), unsubscribes are honored same-day, bounces stamp the manual-bounce ledger and flip the lead. The procedure is documented alongside the campaign plan.
training_pair. When the set crosses a decision gate (IMMIGR8-275), those pairs fine-tune an owned Immigr8 model for Phase A reading + the uncovered tail of Phase B — with the deterministic engine + the 15-rule validator staying on as the guardrail that keeps the model honest and auditable. Fully owned, end to end.
evaluation_jobs queue contract