Don't pay an LLM to do an if-statement.
Most of what looks like an AI problem is a process problem wearing a costume. The machinery here is deterministic by default — plain code, plain tests, plain logs — and a model only gets called at the point where judgment is genuinely required. That boundary is the whole design.
| System | What it does | Evidence | State |
|---|---|---|---|
| flagstat.us | Knows when U.S. flags fly at half-staff, and why. Official proclamations parsed, cross-checked against a second source, and served over an API. Model classification sits behind deterministic gates, because the failure mode is telling a state its governor ordered something he didn't. | 940 tests rev 00120 |
production |
| loteck-evals → | Scores a model against versioned case suites so model choice is a measurement, not a preference. Four scorers, and the LLM judge is the last resort: eleven of the fourteen cases are graded by executing the generated code or validating its schema. Open source. | 4 scorers 14 cases |
public |
| mission harness | Runs a queue of jobs overnight against local models. A job only counts as done when it turns a failing test green, and the test is sealed before work starts — a pass earned by editing the test is a trust violation, not a fix. Anything it can't finish inside its budget comes back to me with its working shown. | 52 missions since 29 jun |
running |
| Date | Job | Outcome | Result |
|---|---|---|---|
| 28 jul | make app/tests/test_poller_pa_boilerplate.py pass by editing ONLY app/services/poller.py, keeping… | qwen3-coder-next | proposed |
| 28 jul | make app/tests/test_end_date_review_persist.py pass by wiring the end_date_review signal end-to-end. (1)… | qwen3-coder-next | proposed |
| 27 jul | enrol every installed model for evaluation, not just the hand-listed ones | qwen3-coder-next 2 attempts |
verified |
| 22 jul | correct honoree boundary parsing in proclamation text | qwen3-coder-next 1 attempt |
verified |
| 22 jul | internal project — counted in the tallies above, not named here | qwen3-coder-next 5 attempts |
proposed |
Twelve runs stopped short and handed the job back — every one of them because it hit its own time or attempt budget, which is the cap doing exactly what it's for. Each came back with its receipt attached, so I could see what it had tried. Every job behind those escalations has since shipped — several on a second pass with a longer leash. The count stays on this sheet on purpose: a machine that never hands anything back isn't checking its work, it's just asserting.