OpenBench: kimi-k3 across 3 harnesses

kimi-k3

Non-comparable provenance: harness_version mixed within group; harness_version mixed within group; harness_version mixed within group; harness_version_source mixed within group; harness_version_source mixed within group; harness_version_source mixed within group. Inspect source rows before publishing.

Correctness

Bars show solve rate; whiskers show Wilson 95% confidence intervals.

Efficiency

Lower-right is better: more correct and leaner. Diamonds identify cursor and opencode; their token totals are self-reported.

Speed

Lower-right is better: more correct with a lower median wall time among solved runs.

Arm (harness × model)	Solved/countable	Solve rate @cap	Solve rate finished	Wilson 95% CI	Med wall	Total tokens/solve (incl. cache reads)	Uncached in/solve	Output/solve	Cache-read/solve	Token basis	Excluded / unmatched / duplicates
grokbuild × kimi-k3	32/45	71.1%	84.2%	56.6–82.3%	114.9s	356,363	39,663	16,672	300,028	self-reported	0 / 0 / 0
opencode × kimi-k3	31/45	68.9%	81.6%	54.3–80.5%	148.5s	—	44,070	22,736	498,143	proxy-measured, self-reported	0 / 0 / 0
pi × kimi-k3	25/39	64.1%	71.4%	48.4–77.3%	95.8s	—	29,633	15,229	223,478	proxy-measured, self-reported	6 / 0 / 0

Total tokens/solve (incl. cache reads) is computed uniformly from the split fields: uncached input + output + cache read. The vendor `tokens` field is never used. Split fields use each harness's own classification and are NOT cross-comparable. Badge bases: unmetered / self-reported / proxy-measured.

Harness	kimi-k3
grokbuild	71.1%
opencode	68.9%
pi	64.1%

Methodology & limitations

Method

Denominators are countable cells. Infrastructure and rate-limit failures are excluded; other failures, including timeouts, remain in the denominator.
Correctness uses the canonical OpenBench aggregation and Wilson 95% confidence intervals. Where timeouts occur, the report shows both accuracy at the configured cap and accuracy among finished cells.
Efficiency is descriptive: median wall time among solved cells and total measured tokens divided by solves. Token basis tags identify vendor/CLI-reported versus proxy-metered values.
Harness defaults are deliberately not clamped because they are part of the product being evaluated. Sampling and provenance should be inspected in the source JSONL.

Limitations

Results cover only the included tasks, trials, model deployments, harness versions, and timeout caps; they do not establish a universal ranking.
Mixed versions or caps, incomplete cells, and differing token protocols can weaken comparability. CLI-reported and proxy-metered token values should not be treated as interchangeable.
Cost appears only for models with a configured price and includes uncached input and output according to the canonical OpenBench price calculation; subscription pricing and cache-specific pricing are not inferred.
Long-horizon work, multi-agent modes, and unsupported model/harness combinations may not be represented.