Don't pay an LLM to do an if-statement.
Most of what looks like an AI problem is a process problem wearing a costume. The machinery here is deterministic by default — plain code, plain tests, plain logs — and a model only gets called at the point where judgment is genuinely required. That boundary is the whole design.
| System | What it does | Evidence | State |
|---|---|---|---|
| flagstat.us | Knows when U.S. flags fly at half-staff, and why. Official proclamations parsed, cross-checked against a second source, and served over an API. Model classification sits behind deterministic gates, because the failure mode is telling a state its governor ordered something he didn't. | 1,563 tests rev 00252 |
production |
| loteck-evals → | Scores a model against versioned case suites so model choice is a measurement, not a preference. Four scorers, and the LLM judge is the last resort: eleven of the fourteen cases are graded deterministically — by executing the generated code, validating its schema, or matching a rule — and only three ever reach the judge. Open source. | 4 scorers 14 cases |
public |
| mission harness | Runs a queue of jobs overnight against local models. A job only counts as done when it turns a failing test green, and the test is sealed before work starts — a pass earned by editing the test is a trust violation, not a fix. Anything it can't finish inside its budget comes back to me with its working shown. | 109 missions since 29 jun |
running |
| Date | Job | Outcome | Result |
|---|---|---|---|
| 30 sep | specify what flagstat's email webhook should do at ingest when a flag_order email's parsed start_date is more… | qwen3-coder-next 4 attempts |
proposed |
| 30 sep | in flagstat, two functions write a state back to "full-staff" with a hand-rolled dict that leaves stale order… | qwen3-coder-next 1 attempt |
verified |
| 29 sep | in flagstat app/routers/api.py, make the output of state_row() and half_staff_rows() never contain these five… | vox 1 attempt |
verified |
| 29 sep | in flagstat app/routers/api.py, make the output of state_row() and half_staff_rows() never contain these five… | qwen3-coder-next 1 attempt |
verified |
| 26 sep | in flagstat scripts/flagstat_admin.py, make every successful --execute write by cmd_clear, cmd_set and… | vox 1 attempt |
verified |
Twelve runs stopped short and handed the job back — every one of them because it hit its own time or attempt budget, which is the cap doing exactly what it's for. Each came back with its receipt attached, so I could see what it had tried. Every job behind those escalations has since shipped — several on a second pass with a longer leash. The count stays on this sheet on purpose: a machine that never hands anything back isn't checking its work, it's just asserting.