The self-audit · measured 2026-08-21
We ran our own test on ourselves.
We tell clients most automation breaks within ninety days. It would be dishonest to say that without checking whether our own stack survives it. So we applied the same standard to ourselves and published what came back, including the parts that did not pass.
§ 01 · Why this page exists
Every automation consultancy claims its work is reliable. Almost none of them publish a number that would embarrass them. We looked at sixteen sites in our own category and did not find one.
That is a strange gap, because reliability is the only claim in this business that can actually be measured. So we measured ours, using the same standard we apply on client work, and put the result here whether or not it flattered us. It did not.
The method is below so you can run it against your own stack without hiring anyone.
§ 02 · What we mean by healthy
A workflow counts as healthy only if it passes all five. Any single failure is a failure, because in practice any single one of these is what turns a working automation into a silent one.
01
It still runs
The workflow is active and has a retained execution inside the audit window.
A workflow nobody has proven ran is not an asset. It is an assumption.
02
Failure is announced
An error workflow is attached AND that error workflow is itself published and active.
An attached but unpublished error handler never fires. Failures then arrive as silence, which is worse than an alert.
03
Writes retry
Every node that writes, sends or charges has retry-on-fail enabled.
One timeout against an upstream API should not end a run and lose the record of it.
04
Clean in the window
No errored executions across the retained window.
Intermittent failure is failure. A workflow that works four days in five is not working.
05
Credentials are watched
The authorisations it depends on are monitored for expiry, not discovered on failure.
One expired authorisation can take several unrelated workflows down at once, and usually does.
§ 03 · The result
4 of 31 active workflows passed every criterion.
We ran a version of this in May 2026 and recorded 3 healthy out of 83 audited.
We are not presenting that as a comparison, because we did not write down what healthy meant at the time. The figure survived; the criteria did not. That is exactly the failure this page is about, and it happened to us in the middle of measuring it. The five criteria above are now recorded so the next audit can be compared to this one honestly.
§ 04 · What actually went wrong
Grouped by cause across the retained execution window, 14 August 2026 to 21 August 2026. Specific endpoints, credentials and client systems are deliberately not named here.
- Data shape: a value arrived in a form the next step could not read
- 25
- Expired or revoked credential
- 19
- Logic error in a condition or expression
- 5
- Upstream API error
- 3
- Cause not recoverable from the record
- 1
The window holds 166 execution records. 18 of them log the same fault twice, once by a sub-workflow and once by its parent, which leaves 148 distinct executions and 53 distinct failures. That is a 35.8% failure rate, and both halves of the division are printed here so it can be checked rather than taken on trust.
Thirteen of our 31 active workflows discard successful runs, so successes go missing while failures are kept. That inflates the rate above against us, and we are reporting it as measured rather than adjusting it in our favour. Roughly one week of history is retained, which is its own finding: an audit can only see as far back as the logs allow.
Counted in a verified quiet window with no edits to the estate in progress. An earlier pass was taken while workflows were being changed and the totals moved underneath it. A number measured against a moving target is not a measurement.
§ 05 · What this put on the list
Publishing a bad number is only worth doing if it changes something. This audit is recent, so the honest version of this section is what it put on the list rather than what is already finished. Each item below is the same fix we make on client work, applied to ourselves.
- Verify every error handler is itself active, not merely attached. An error workflow that exists but was never switched on does not fire, which means failures arrive as silence.
- Monitor credential expiry rather than discover it. A single expired authorisation accounts for more failures than any other cause in this audit, across several unrelated workflows at once.
- Require retries on every write, not just the ones that looked fragile.
- Retain successful runs where volume allows, because an audit is only as good as the window it can see.
This page is dated. When we run the audit again, the number here changes, including if it gets worse.
§ 06 · Run it on your own stack
The five criteria are the whole method. You do not need us to apply them.
If you would rather see the shape of your stack first, the self-scoring audit takes two minutes and stays in your browser. If you want us to run the full version against your systems and hand you the findings in writing, that is the Diagnostic.