How we build. Five steps, published, because a method you cannot read cannot be verified.
We build custom artificial intelligence tools, and we integrate AI into the systems you already run. The five steps below are the same in both cases. Each one carries the artifact that proves it exists, and you receive the same objects.
From scoping to production, five steps
Each step carries the object that proves it was executed: a file, an exact value, or the instruction that verifies it. A step without an artifact is not a step, it is an intention. The artifacts named here come from the two products we built and have run without interruption since 2022 and 2023; the same sequence is applied to the tools we deliver.
Scope: put a number on the end criterion before the first line
The scope, the deliverables and the gates that decide production are numeric and accepted in writing before the first production commit. Ours, exact values from eval/thresholds.ts: AUC >= 0.70, precision@k >= 0.70 with k = 40, grammar validity 100 %, injection blocking 100 %, deterministic consistency 100 %, all evaluated across 5 seeds (1, 7, 42, 100, 2024). If one threshold fails, the evaluation suite fails and nothing ships.
- Artifact
- The file that carries the thresholds. These are the exact values, not a restatement.
Design: freeze the measuring instrument before building
The test set is written before the first measurement, committed, and never touched again. On our product the corpus holds 50 prompts, 7 of them adversarial and chosen to hurt (out of range bounds, slang and typos, vague quantity, mixed language, a 150-word prompt, an out of scope request, a fully ambiguous prompt), and it was committed before the first model call. It has not moved since.
- Artifact
- The history of the corpus file, with no rewrite: the date of the first commit, and the absence of any edit since.
Build: publish the baseline, even when it is bad
You measure before any fix and you write the number down. Ours: 40 out of 50, that is 80 %, gate not met. That number was published before the fixes, with the three root causes identified. It is the most useful line in the whole file, and the one no competitor has any incentive to write.
- Artifact
40/50baseline before fixes
The number as it came out, before the first fix.
Ship: measure against a control group, not against an impression
You compare against a group that received nothing. In our code: deterministic holdout, two-proportion z-test for binary retention, Welch t-test for continuous outcomes, and the result is only written at the end of the window. The move into production happens in increments, with a rollback that takes one instruction for as long as the window stays open.
- Artifact
- The directory that carries the control computation and the two statistical tests.
Operate: abstain in writing when the data is not enough
Below the significance or sample threshold, the system answers “not enough data”, never a weak number. Two parameters carry that rule in our code: k-anonymity at 5 for comparisons, and a minimum of 30 per group, treated and control, to claim an effect. Over the last twelve months of operation the rule fired on a little under one week in five, and those weeks carry no number at all.
- Artifact
- The two parameters, exactly as they are named in the code.
- Exact sentence present in the code
How a build runs, six commitments
- Scoping comes before code. The scope, the deliverables and the end criterion are written and signed before the first line.
- You get a build ledger, module by module, with the command that verifies each line.
- The first thing we hand you is a list: the reasons the requested tool can fail, sorted into blocking, major, minor.
- The tool does not ship if it does not hold the thresholds set together before starting.
- At each milestone you get the list of what is not ready and why, in green, amber and red.
- If the measurement says the tool is not worth what it costs, the deliverable is a report telling you not to build it, and it is billed at the same price.
The sixth commitment, as it was actually handed over
An accounting firm asked us for a bank reconciliation agent. The baseline, taken on their own entries and on their frozen corpus, came out at 62 out of 100 against the gate of 95 accepted at scoping. The three root causes sat in the quality of the bank labels upstream, not in the model: no amount of work on the tool would have moved the number. The deliverable was a report telling them not to build it, billed at the price agreed.
The report names what has to be fixed upstream for the question to be worth asking again, and it puts a number on the gain that would still be reachable once that is done. No excerpt is published here: the document describes the system of a company that is not ours.
The communication cadence, and nothing beyond what can be held
One business day response. A written update at every milestone. No fixed weekly meeting: an announced cadence that slips destroys more trust than it creates, and we would rather announce what we hold under load.
Exit conditions
- Stop at the end of any milestone, with no penalty.
- Client repository from day one.
- Retention of title until payment in full.
The deposit commits both sides at the same moment, and it has an exact counterpart: you exit at the end of any milestone, and everything produced sits in your repository from day one, including the build ledger and the test cases.
Building new, or integrating into what already runs
These five steps were written and executed on products where we wrote every line, with full access to the code, the test server and the inference budget. On a build from scratch, that is still the case. When the work is integrating AI into a system you already run, the chain exists before we do: we survey it, access has to be negotiated, and part of the system does not belong to us.
It is the same method and it is not the same gesture.
So scoping an integration starts with a survey of what is already there: what the system writes, what it keeps, who reaches it, and where a measurement can sit without disturbing production. That survey is a deliverable in its own right, it is dated, and it decides the thresholds instead of inheriting them. We say so before you ask, and that is also why the “inconclusive measurement” verdict exists.
Ways out of this page
The full measurement, initial failure included, is published on the proof page, together with the two products that carry it. The catalogue says what we build, and at what scope.
Six written questions first, then real slots on the same page. Free, 30 minutes.
The full report, including its initial failure at 40 out of 50.