01 / 10
What this talk is

Three free models.
One that beats all three.

81
model A
80
model B
79
model C
→
▲
coordinated
Composite Intelligence — take several free, open models, none of them frontier-class, and coordinate them so the group reaches a capability tier above its best member. No bigger model. No training run. Near-zero cost. The title is the whole bet: can you assemble intelligence instead of buying it?
DDD Perth · the honest field report · 10 slides, ~20 min
02 / 10
Why you should care

The model you rely on
can be taken away.

Fable
A model people built on — pulled from everyone, overnight. If your product's brain is something you rent and don't control, it can vanish on someone else's schedule.
∑
The composite is resilient by construction: many free, swappable parts. One disappears? Swap it. Your capability is something you assemble and own, not something you can lose.
Buy frontier capability
$$$$
A bigger training run, a vendor you depend on, a price only a few can pay — and an off-switch you don't hold.
Assemble it from free parts
¢
Coordinate open models you can host or swap. A sovereign path to frontier-class AI — a nation, a lab, a team — without the frontier price tag or the dependency.
03 / 10
The finding

The winning answer is
already in the room.

82
best single
model
86
Opus 4.8
(frontier)
91
panel ceiling
(any member right)
+8.8
above best
single
196 questions, 14 disciplines. On 90.8% of them at least one of three free models is correct — +8.8 above the best single, and above Opus 4.8 (86.2%, identical blind conditions). The diversity needed to beat the frontier is genuinely sitting in a panel of free models. axis zoomed 60–95 to show the gap.
04 / 10
The uncomfortable part

…but the group
won't say it.

0
best single
= the bar
−0.5
category
routing
−1.9
majority
vote
−0.4
optimised
weights
+8.8
ceiling
(unclaimed)
Every simple way of combining them — voting, weighting, routing to a specialist — lands at or below the best single model. The knowledge is in the panel; counting votes can't extract it. The gap between what the group knows and what it says is the entire problem — and the prize.
05 / 10
A real negative result

Coordination can
make it worse.

+6.7
3 different lineages, comparable strength. Errors decorrelate → the vote beats the best member.
vs
−6.3
2 of 3 share a lineage. They make the same mistake, agree on the same wrong answer, and out-vote the stronger model.
Same technique, opposite sign. The design rule falls straight out of the data: diverse lineages, no dominant member. A correlated sub-majority will drag a vote below its own best member every time.
06 / 10
We caught ourselves

Our best result
was a bug.

First headline
+7.6
"Voting wins, significantly!" 🎉
→
After we fixed it
−1.9
Voting actually loses. The lift never existed.
The cause: our strongest model is a verbose reasoner — it gets cut off before writing "ANSWER: X". Our parser then mis-read a stray letter from its half-finished maths (scoring it 15% on truncated answers vs 89% when it finished), and rate-limited calls were counted as wrong. Both undercounted the best model → inflated the apparent gain from voting. We found it in our own review, re-measured, and the lift vanished. Showing this is the talk.
07 / 10
How you earn a number

Same model, same test:
±7 points apart.

Where the swing comes fromWhat it really isWhat we did
Question samplewhich questions you drawfrozen seed ✓
Provider routingone "model" silently served from different backends & quant levels — a mixture, not a modelpin the provider ✓
Answer parsingscoring a truncated-but-correct reply as wrongstrict + re-ask ✓
Real LLM noisetemp 0 still isn't bit-exactirreducible — but only ±1–2, and we measure it
"LLMs are just random" is mostly an excuse for un-pinned infrastructure. Pin the provider · freeze the parsing · split reliability from accuracy · test the difference paired (same run, so luck cancels). Kill the contamination and the swing collapses to ±1–2 — then a lift is believable. Boom. Next.
08 / 10
Two numbers, not one

Not 58% smart.
87% smart, 72% reliable.

58
naïve score
(the wrong number)
=
87
intelligence
(when it answers)
72
reliability
(% it answers)
A model that truncates 28% of the time isn't dumb — it's unreliable. Mash the two together and you punish your best thinkers and pick the wrong panel. We report coverage and accuracy separately; under 90% coverage a model is flagged and can't silently join a panel. Capability you can't reliably access isn't capability you can ship.
09 / 10
Where the unclaimed wins are

The AI gets promoted.

🛠️
Worker
Every model just answers; a dumb harness tallies votes. Caps at the ceiling — can't beat the best member.
routing · voting
✍️
Editor ← first to clear it
An AI reads every draft, finds the error, writes the answer. Critique-fusion: +0.6 — the first method past the best single.
synthesis
🧠
Manager
An AI runs the process — who to ask, how many rounds, when to stop. The moonshot at the top of the ladder.
orchestrator (open)
Same models the whole way up — only the AI's job in the coordination changes: worker → editor → manager. Selection (worker) is hard-capped at the ceiling. Synthesis is the only regime that can manufacture a correct answer no single model gave — and it's barely solved. That's the open frontier this points at.
10 / 10
The takeaway

Frontier-class,
assembled — not bought.

✅
What's proven
The winning answer is in a free-model panel +8.8 above the best single, above a frontier model. The prize is real.
🔬
What's honest
Simple voting doesn't capture it. The first synthesis lift (+0.6) is small; at 196 questions we can prove the ceiling but not yet the small lift — that needs more data, and we say so.
🧭
What's next
Climb the ladder to orchestration; test across rival labs and harder benchmarks. The unclaimed wins are all above the ceiling.
The old way up a tier: train a bigger model — $$$$. This way: coordinate a diverse panel and put the intelligence in the coordinator — ¢. A sovereign path up the capability ladder, and an honest map of how far it currently reaches.
Read the paper Every bug we caught →
Composite Intelligence · aus.wick.pics · every experiment reproducible, every caveat on the record.