Series · 3 notes
CI and tests
How we test, where our tests lied to us, and running CI when the usual service stopped.
- Part 1 Deep dive Jun 23 – Jul 2, 2026
A control plane in three days, and nothing protecting it
In one week of June we went from a spec to a deployed control plane, a signed-in customer portal and an agent on our own ERPNext site, with no automated tests. The review that brought the first tests, on July 2, also found that "Schedule tonight, 2 AM" ran immediately.
- Part 2 Deep dive Oct 16 – 19, 2026
When GitHub Actions stopped, CI moved to Cloud Build
On October 16 GitHub Actions stopped starting jobs for our organization. We merged on local test runs, then ran the same CI jobs on Cloud Build; the first run took 1 min 52 s against an estimate of 5 to 10 minutes. Rolling it out found tests that had never run on a CI bench.
- Part 3 Deep dive Oct 19 – 20, 2026
Three flaky tests in two days, and one rule out of them
Three control-plane tests failed on CI for reasons that had nothing to do with the code under test. Each one compared against the clock. We took the clock out of all three assertions, and now we don't assert wall-clock timing in those tests at all.