Consortio run trace: no winner without proof

← Back to Consortio

Why this run

I chose this run because its target was a product repository I own and its outcome is useful rather than flattering. Candidate code was produced, but the proof did not support delivery. That makes the human-controlled boundary visible.

The source is run observation mowzaw8f-by6frc, generated on 8 May 2026. I removed absolute filesystem paths and internal requirement identifiers, shortened commit hashes, grouped repeated path aliases, and omitted raw model prose. The objective, candidate progression, check statuses, proof-failure counts, disposition, and absence of exports are retained.

Recorded run

Field	Recorded value
Target	Masterful CV repository
Base commit	fd6474f
Candidate family	One Codex candidate, revised across rounds
Disposition	incomplete
Turn failures	0
Proof failures	17
Export artifacts	0

The task was to refine the builder's Applications pane into an AI-assisted job-application workbench. The reference direction was a dense three-pane workflow, but the implementation still had to respect the product's existing data model, controls, responsive behaviour, and design conventions.

Candidate states

This was not a comparison between independent models. The observation records one candidate family and two distinct commit states:

State	What changed	Static analysis	Candidate tests
Round 0 · 344fcf1	Application generation and progress panels, preview/mobile surfaces, tests, and design documentation	Passed	Failed
Rounds 1–4 · aa80188	Retained that scope, added the editor header, and broadened application copy across 22 locale files	Passed	Failed

The later rounds and review artifacts repeatedly referenced the same second commit. That is evidence of iteration, not evidence of multiple independent solutions.

Grouped changed-file manifest delta

The observation retained changed-file manifests rather than the raw source patch. This is the faithful, grouped delta between the first and second recorded states:

+ src/components/applications/application-editor-header.tsx
+ src/i18n/locales/en-GB/applications.json
+ 21 sibling applications.json locale files
  application generation, preview, mobile-tab, test, and design-document paths remained in scope

The grouping removes repetitive locale paths; it does not invent a source-code diff. A public artifact should distinguish those two things.

Verification evidence

The positive evidence was narrow: static analysis was recorded as passing for both distinct commit states.

The blocking evidence was stronger:

• Candidate-level testsOk was false for both states.
• The final recorded browser-test proof failed while running the Applications editor, visual, sidebar, and dark-mode end-to-end specifications.
• Across four quality-gate cycles, 16 dark- and light-theme visual-analysis or visual-judge results remained unproven. The analysis expected a valid capture manifest with evidence bindings and theme metadata; without it, the visual judge could not run.
• No turn failure was recorded. The failure was in the proof boundary, not in whether an agent returned a message.

That separation matters. “The agent completed its turn” and “the change is ready to deliver” are different facts.

The disagreement that mattered

There was no preserved model-to-model disagreement in this run, and presenting one would be misleading. The meaningful conflict was between two kinds of evidence:

• the candidate changed relevant code and passed static analysis;
• the external test and visual-proof record did not establish that the result worked or matched the requested design.

Consortio should make that conflict hard to overlook. A plausible implementation summary cannot overrule a failed command or missing visual evidence.

Operator decision

The recorded disposition was incomplete, and the exports array was empty. My decision is therefore: do not export or apply a candidate from this run.

That decision does not prove the code was unusable. It says the run did not establish enough evidence to accept it. The candidate states remain useful for diagnosis or a later, explicitly scoped attempt.

Remaining risk

At the end of the run, a reviewer still could not answer four important questions from the evidence:

1. Did the complete Applications workflow pass in the browser?
2. Did the light and dark layouts meet the requested visual direction?
3. Were the locale changes semantically correct rather than mechanically broad?
4. Would another candidate family have produced a smaller or better-proven change?

A responsible run report keeps those unknowns visible instead of converting them into a confidence score.

What this artifact proves—and what it does not

It proves that this prototype run preserved intent, candidate revisions, distinct verification facts, proof gaps, and an empty export outcome. It also demonstrates a no-winner decision grounded in evidence.

It does not prove multi-model comparison, a correctness guarantee, product adoption, readiness for production, or that the later Masterful CV codebase came from this candidate. It is one sanitized run trace from a private CLI prototype.

Read the Consortio product page or the design note, What I want from agent tooling.