Open-weight models

How fast are open-weight models catching up?

Once a capability appears in a downloadable model, it cannot be withdrawn. The gap is one measure of lead time.

Published gap ranges (months behind frontier)

Bars show the published range. Labels are as reported; we do not place undated estimates on a time axis.

Gap estimates

General · Dec 2025 report

4 to 8 months

Caveat: Based on external benchmarks. The future trajectory is uncertain.

Cyber · Most of 2025

6 to 10 months

Caveat: Reported retrospectively in the Jul 2026 post. Range results are weaker evidence than narrow tasks.

Cyber · Jul 2026

4 to 7 months

Caveat: Range results are weaker evidence than narrow tasks. Gap narrowed from 6 to 10 months.

Model pairs

Frontier modelOpen-weight modelLagBenchmarkSource and caveat
Claude Opus 4.6 (Feb 2026)GLM-5.2 (Jun 2026)4 monthsAISI narrow cyber tasks (70 tasks)AISI, How far behind the frontier are leading open weight models on cyber? (17 Jul 2026)

Comparable average success on AISI's narrow cyber tasks; AISI also names GPT-5.3-Codex as comparable. Lag is measured between release dates. Lab results, not attack rates.

Claude Opus 4.5 (Nov 2025)DeepSeek V4-Pro5 monthsAISI narrow cyber tasks (70 tasks)AISI, How far behind the frontier are leading open weight models on cyber? (17 Jul 2026)

Comparable average success on AISI's narrow cyber tasks. Lag is measured between release dates. Lab results, not attack rates.

Claude Opus 4.5 (Nov 2025)GLM-5.2 (Jun 2026)Not publishedAISI cyber range "The Last Ones"AISI, How far behind the frontier are leading open weight models on cyber? (17 Jul 2026)

AISI says GLM-5.2 reaches as far as Opus 4.5, released less than 7 months before it; no exact lag is given, so none is shown. Ranges lack active defenders and are weaker evidence than narrow tasks.

GPT-5.4 (Mar 2026)GLM-5.3 (Aug 2026)Not publishedSamaritan Research Cyber ECI (composite index)Samaritan Research, Open-weight models trail closed models in cyber capabilities by 6 to 7 months (16 Oct 2026)

GLM-5.3 scores above GPT-5.4 and below Claude Mythos Preview (Apr 2026). Samaritan gives a 6 to 7 month gap to its 8 Oct data cutoff, not a release-to-release lag, so none is shown. GLM-5.3's score rests on three CAISI results. Composite index, not comparable with AISI's task-based lags.