Don't pay an LLM to do an if-statement.
Most of what looks like an AI problem is a process problem wearing a costume. The machinery here is deterministic by default — plain code, plain tests, plain logs — and a model only gets called at the point where judgment is genuinely required. That boundary is the whole design.
| System | What it does | Evidence | State |
|---|---|---|---|
| flagstat.us | Knows when U.S. flags fly at half-staff, and why. Official proclamations parsed, cross-checked against a second source, and served over an API. Model classification sits behind deterministic gates, because the failure mode is telling a state its governor ordered something he didn't. | 1,563 tests rev 00252 |
production |
| loteck-evals → | Scores a model against versioned case suites so model choice is a measurement, not a preference. Four scorers, and the LLM judge is the last resort: eleven of the fourteen cases are graded deterministically — by executing the generated code, validating its schema, or matching a rule — and only three ever reach the judge. Open source. | 4 scorers 14 cases |
public |
| mission harness | Runs a queue of jobs overnight against local models. A job only counts as done when it turns a failing test green, and the test is sealed before work starts — a pass earned by editing the test is a trust violation, not a fix. Anything it can't finish inside its budget comes back to me with its working shown. | 109 missions since 29 jun |
running |
| Date | Job | Outcome | Result |
|---|---|---|---|
| 26 sep | specify, for each of CA, NE and MO, whether and how it can become a release-poller backstop source… | qwen3-coder-next 4 attempts |
proposed |
| 26 sep | specify how flagstat should turn Puerto Rico's Spanish mourning decrees (fortaleza.pr.gov /prensa) into flag… | qwen3-coder-next 4 attempts |
proposed |
| 26 sep | in flagstat app/services/llm_classify.py, change _call_gemini so that when a model's response has status_code… | qwen3-coder-next 1 attempt |
verified |
| 26 sep | in flagstat scripts/flagstat_admin.py, make every successful --execute write by cmd_clear, cmd_set and… | qwen3-coder-next 2 attempts |
verified |
| 24 sep | internal project — counted in the tallies above, not named here | qwen3-coder-next 4 attempts |
proposed |
Twelve runs stopped short and handed the job back — every one of them because it hit its own time or attempt budget, which is the cap doing exactly what it's for. Each came back with its receipt attached, so I could see what it had tried. Every job behind those escalations has since shipped — several on a second pass with a longer leash. The count stays on this sheet on purpose: a machine that never hands anything back isn't checking its work, it's just asserting.