Don't pay an LLM to do an if-statement.
Most of what looks like an AI problem is a process problem wearing a costume. The machinery here is deterministic by default — plain code, plain tests, plain logs — and a model only gets called at the point where judgment is genuinely required. That boundary is the whole design.
| System | What it does | Evidence | State |
|---|---|---|---|
| flagstat.us | Knows when U.S. flags fly at half-staff, and why. Official proclamations parsed, cross-checked against a second source, and served over an API. Model classification sits behind deterministic gates, because the failure mode is telling a state its governor ordered something he didn't. | 1,563 tests rev 00252 |
production |
| loteck-evals → | Scores a model against versioned case suites so model choice is a measurement, not a preference. Four scorers, and the LLM judge is the last resort: eleven of the fourteen cases are graded deterministically — by executing the generated code, validating its schema, or matching a rule — and only three ever reach the judge. Open source. | 4 scorers 14 cases |
public |
| mission harness | Runs a queue of jobs overnight against local models. A job only counts as done when it turns a failing test green, and the test is sealed before work starts — a pass earned by editing the test is a trust violation, not a fix. Anything it can't finish inside its budget comes back to me with its working shown. | 109 missions since 29 jun |
running |
| Date | Job | Outcome | Result |
|---|---|---|---|
| 9 sep | how should the forge stop shipping work that passes its oracle but cannot be merged? Two distinct failure… | gemma4:31b | proposed |
| 5 sep | give the build gate a GitHub source, so a GitHub-hosted repo stops being measured only by leftover branches… | qwen3-coder-next 3 attempts |
verified |
| 4 sep | make every promoter lane measure Shad's review budget over the same workspace-wide scope. WIP_LIMIT is one… | qwen3-coder-next 3 attempts |
escalated |
| 4 sep | decide what should determine status precedence in flagstat, using ONLY the seed. The seed's central claim is… | qwen3-coder-next 5 attempts |
proposed |
| 4 sep | fix the event taxonomy in scripts/coverage_gap_report.py so the coverage numbers measure coverage rather than… | qwen3-coder-next 1 attempt |
verified |
Twelve runs stopped short and handed the job back — every one of them because it hit its own time or attempt budget, which is the cap doing exactly what it's for. Each came back with its receipt attached, so I could see what it had tried. Every job behind those escalations has since shipped — several on a second pass with a longer leash. The count stays on this sheet on purpose: a machine that never hands anything back isn't checking its work, it's just asserting.