Zakat-Eligibility Triage: System Design

01

The pipeline and its trust boundary

ADR-0001ADR-0003ADR-0004ADR-0006ADR-0007ADR-0010

This is the whole pipeline and the line it is built around. A donor pastes a campaign, one model call extracts a typed record of the facts, and a second maps the text against the eight recipient categories as supported, not supported, or insufficient evidence, with the exact span of story behind every mapping. Citations are the part I was strictest about: the model emits verbatim quotes and the server resolves them to offsets by exact search, so slicing the story at those offsets returns the quote by construction rather than by inspection, and a quote that cannot be located is a hard failure (ADR-0003). The refusal gate underneath is plain code over typed output with no model call, no clock and no network, so the same campaign refuses for the same reasons in the same order (ADR-0006). Four conditions fire it, and each carries the specific question that has to be answered and says who can answer it: the organizer, for what the page leaves out, or a scholar the donor trusts, for where recognised scholars differ (ADR-0010).

The one decision behind this diagram is the red line. The tool never rules. No schema field on the model side accepts a score or a confidence figure, and no table in the database has anywhere to put an outcome, so there is nothing for the system to publish; a schema guard fails the suite if a column named like a status or a verdict ever appears (ADR-0001, ADR-0010). The person who decides is the donor, with the scholar they follow, and nothing they decide is stored. The corpus of past adjudicated cases is still in the repository and still fenced out of every prompt by an import-graph test and a prompt-recording trace test, but the public tool does not show it, because a past ruling shown to a donor reads as an answer (ADR-0004, ADR-0010). Everything in a citation position is either a byte-checked span of the campaign or human-authored corpus text retrieved by id, and the model's own prose is shape-guarded against quotation and chapter-and-verse references (ADR-0007).

02

The evaluation gate

ADR-0001ADR-0009

This is how I check the thing, and it is the part I would want anyone reading this to press on. Eighteen hand-labelled synthetic campaigns run through two scorers that measure different things. The deterministic half counts what a written label can be right about: per-category status agreement, whether a citation slices its own quote back out of the story, exact-set agreement on which refusal conditions fired, and whether every category the label expects a question on got one. The judge half asks a different model, shown the campaign and the record and nothing else, for a pass or fail with a stated reason on four things no label can see. It is blind to the label because the deterministic half already scores agreement and scores it better, and blind to the precedent corpus because a judge holding past decisions would reward a record for resembling them (ADR-0009).

Every judge dimension is pass or fail and never a score, because a judge rating reasoning out of five would reintroduce through the test harness exactly the uncalibrated number ADR-0001 turned down. The timeline is the honest part. The gate went red four times before it went green, and each red caught something real, including one run where the harness charged its own parse failures to the system and reported twelve scholarly rulings that had never happened. A harness that cannot tell a failed measurement from a bad result reports the second when it means the first. One floor moved after all that, in a commit that argued from the report, and that is the only way a threshold is allowed to move here: lowering a bar to turn a red run green is the failure mode the rule exists to make visible.

03

Runtime and integrations

ADR-0002ADR-0005ADR-0011

The runtime is deliberately small. One Vercel project running Next.js App Router with TypeScript end to end, one Neon Postgres database, and Anthropic for the extraction and mapping calls. OpenAI's embeddings are used only by the script that seeds the precedent corpus, which the public tool does not read when it checks a campaign. Models are injected everywhere, so the unit suite runs against mocks with no network, and PGlite boots the real shipped migrations so the tests exercise the schema production runs (ADR-0002, ADR-0005).

What I care about in this picture is which credential each thing holds. The unit job in CI holds none at all, and its build has to stay green with no database URL set, because an unset database is a state the result page reports rather than crashes on. The eval job holds the model key and nothing else, and it fails when that key is missing rather than skipping, since a gate that passes without running is a green build for a run nobody performed. The eval job is also path-filtered, so a docs-only commit does not spend a corpus of model calls to measure nothing. The judge model exists only in CI and never runs in production, which is the whole reason it can afford to be a different model from the subject. Production holds one more secret, the key that hashes a visitor's address for the daily limits, and without it the app runs no checks at all rather than counting raw addresses (ADR-0011).

04

The operations layer this pattern comes from

context, drawn generically

This is where the habits came from, drawn without any of the specifics. I run a personal automation platform with more than thirty scheduled automations on it, and the parts that taught me the most are the boring safety parts. Email-triggered workers ingest mail and attachments written by someone else while holding the access that reads and sends the mailbox, so each one runs inside a bubblewrap sandbox with a tmpfs home and a curated environment, behind an egress allowlist that fails closed. That last word is the one that matters: an unknown destination is refused rather than logged, and if the sandbox cannot start then the worker does not run unconfined. Mail is read and sent through a Google Workspace service account with domain-wide delegation, and a note inside a message body is advisory context for a human rather than an instruction the pipeline follows.

The rest of the panel is the same idea in different clothes. Dead-letter sweepers re-drive jobs stranded by a worker that died mid-flight, up to a bounded number of attempts and then parked, because a job that dies quietly is worse than one that fails loudly. The local retrieval stack answers with citations in about four seconds and is measured against a hand-built 40-question gold set, since a retrieval system with no gold set has no way to tell a better index from a differently wrong one. The overnight build pipeline is eval-gated for the same reason the triage agent is. None of this is novel. It is the habit of writing the fence into the code rather than into the documentation, and that habit is what the zakat triage design is made of.

05

What the evals cannot prove

the limits, stated rather than hidden