SWE-bench Verified — cost vs performance
Best current row per canonical model in the group “resolved · board=Verified · system=mini-SWE-agent” (42 models) against ESTIMATED memory at 4-bit, 8K context (GB). The dashed line is the Pareto frontier: no model is both better and cheaper than a point on it.
open / restricted weights closed Pareto frontier (2 models)Estimated
Frontier models 2
Methodology. Points are the best current row per canonical model in comparability group 'resolved · board=Verified · system=mini-SWE-agent'. Price = cheapest current offer across providers (the provider shown). Pareto frontier maximises the score and minimises x; exact ties are all kept. memory_estimate is an estimate (see /methodology). Only the selected comparability group is plotted; points under other configurations are not mixed in. Price = cheapest current offer across providers at the time of the last crawl. Nothing is estimated except where marked.