Case Study · 2026-09-04
Across four AI-native codebases, integration risk scored the same before and after implementation. Review estimates did not. One of the four axes is assessable up front; the other three are contaminated by knowing how the work went.
This is dogfooding, not third-party adoption. All four codebases belong to the team that wrote CIRK. Read it as operational evidence from our own engineering, not as independent validation. The companion analysis of the upstream model is the DD6 case.
The dataset
Not four organisations adopting a standard — four engineering environments inside one team, which is why the rates below differ so widely. The spread is the point: it tracks when the discipline arrived, not how good each team is.
Reusable Django apps and TypeScript packages consumed by every other product: multi-tenancy, plugin loader, workflow engine, messaging.
No end users, but every change propagates to four consumers. High K by construction.
773 commits · 2 171 source files
The product built to enforce this kind of discipline: intake classification, spec governance, agent execution policy.
Adopted CIRK latest of the four, and shows it. Included because it produces the worst number here.
425 commits · 1 364 source files
Generated long-form readings with narrated audio, live in the App Store. The only one with real users and real revenue.
Shipping under load. Highest commit volume and the highest rate of follow-up fixes.
2 548 commits · 958 source files
The first product started after CIRK was already standard practice. Every spec classified at intake, none backfilled.
Not evidence of scale — evidence of what starting under the discipline looks like. Its value grows as implementation proceeds.
implementation has not started
Scored when?
A CIRK vector decides review depth, test requirements and how a change ships. Assigned after the change shipped, it decided none of those. Only the 310 below carry operational meaning.
| Environment | With a vector | Scored at birth | Backfilled | Prospective |
|---|---|---|---|---|
| connectus | 26 | 26 | 0 | 100% |
| myastralmap | 229 | 169 | 60 | 74% |
| core618 | 270 | 86 | 184 | 32% |
| orbit618 | 216 | 29 | 187 | 13% |
| Total | 741 | 310 | 431 | 42% |
Read top to bottom, that column is a timeline. connectus 100% → myastralmap 74% → core618 32% → orbit618 13% is not four teams performing differently. It is one operating model arriving at four different moments — orbit618 last, which is why the governance engine itself scores worst on the discipline it exists to enforce.
Method. For each spec file, find the commit that created it, read the file as it existed in that commit, and check whether the vector was already present. Renamed files whose creation commit is unreachable are excluded. connectus has 26 specs and four source files — it is specified but essentially unbuilt, so its perfect rate reflects starting conditions, not sustained practice.
The finding
310 scored-at-birth vectors against 431 backfilled ones, in the same repositories.
| Axis | Nature | At birth | Backfilled | Delta |
|---|---|---|---|---|
| R Review | experiential | 2.03 | 2.45 | +0.43 |
| C Context | experiential | 2.03 | 2.21 | +0.18 |
| I Iteration | experiential | 1.89 | 1.99 | +0.10 |
| K Integration risk | structural | 1.86 | 1.84 | −0.02 |
The principle underneath
Structural risk can be estimated prospectively.
Experiential risk is contaminated by hindsight.
K asks questions you can answer by looking at a change: how many systems does it touch, which boundary does it cross, who consumes it, what is the blast radius. All of that exists before the code does. R, C and I ask how the work went — and once you know how it went, “how much review did this need” and “how much review did this get” become the same recollection.
| Repository | n (birth / backfill) | R | K |
|---|---|---|---|
| core618 | 86 / 184 | 2.13 → 2.70 | 1.98 → 1.99 |
| orbit618 | 29 / 187 | 2.28 → 2.35 | 2.03 → 1.72 |
| myastralmap | 169 / 60 | 1.88 → 2.03 | 1.71 → 1.77 |
R rises in all three. K has no consistent direction — flat, down, up. The control confirms both.
The rule that falls out
Inheriting backfilled vectors? Trust K. Re-derive R.
Distribution
18 specs at 12 — 2.4%. A band that says “this should have been two specs” firing at that rate is about right. Zero would mean nobody was honest; eighty would mean the spec discipline had collapsed.
Where this sits
CIRK is one link. It only means something because something upstream classified the problem and something downstream measures what execution actually cost.
CIRK classifies risk; it does not price it. Measuring what an agentic change actually costs — input and output tokens, model class, retries, human review minutes — is a separate problem with independent work behind it, including ACEM, a cost estimation model for agentic software engineering, and empirical work on where agent tokens actually go.
We have no proprietary cost unit and do not intend to invent one. What we intend to publish is the join: which problem shapes and which execution vectors predict which costs.
What we have not proven
Whether scoring before the work reduces rework afterwards is the question that decides
whether any of this pays for itself. We measured it as follow-up fix: and
revert: commits referencing a spec ID, split by scored-at-birth versus
backfilled. Read naively, the result says scoring up front triples rework.
That conclusion is wrong. Writing the spec ID into the commit message and scoring the intake up front arrived in these repositories at the same time. Older specs did get fixed — those fixes carry no spec ID and are uncountable. Measuring a discipline through an instrument that arrived with it is circular.
A matched prospective cohort, and only that. Three populations exist in this data and only one of them can carry a causal claim:
Cohort A is what connectus becomes once implementation starts — specs classified on both models before the work, with execution telemetry attached. Until that cohort has aged, the honest claim is that this is operational evidence, not proof, and the numbers above are reported on that basis.
If you are adopting
Data collected 2026-09-04 from four repositories at their dev HEAD. Method and
queries are reproducible from
the case file.