@kentcdodds/invisible-software

benchmark,Model version,date,score
SWE-bench Verified (Epoch runs),glm-5.2_max,2026-06-16,78.7
SWE-bench Verified (Epoch runs),qwen3.7-max,2026-05-19,77.27272727272727
SWE-bench Verified (Epoch runs),deepseek-v4-pro_max,2026-04-24,77.63975155279503
SWE-bench Verified (Epoch runs),gemini-3.5-flash_high,2026-05-19,79.33884297520662
SWE-bench Verified (Epoch runs),qwen3.6-max-preview,2026-04-20,76.65289256198346
SWE-bench Verified (Epoch runs),glm-5.1,2026-04-07,74.17
SWE-bench Verified (Epoch runs),qwen3.6-plus,2026-03-31,57.85123966942148
SWE-bench Verified (Epoch runs),kimi-k2.6,2026-04-20,76.65289256198346
SWE-bench Verified (Epoch runs),gpt-5.5-pre-release_xhigh,2026-04-23,80.57851239669421
SWE-bench Verified (Epoch runs),claude-opus-4-7_max,2026-04-16,83.47107438016529
SWE-bench Verified (Epoch runs),gpt-5.4-2026-03-05_high,2026-03-05,76.85950413223141
SWE-bench Verified (Epoch runs),gpt-5.3-codex_high,2026-02-05,74.79338842975206
SWE-bench Verified (Epoch runs),gemini-3.1-pro-preview-customtools,2026-02-19,75.6198347107438
SWE-bench Verified (Epoch runs),claude-sonnet-4-6,2026-02-17,75.20661157024794
SWE-bench Verified (Epoch runs),gemini-3-flash-preview,2025-12-17,75.41322314049587
SWE-bench Verified (Epoch runs),claude-opus-4-6,2026-02-05,78.71900826446281
SWE-bench Verified (Epoch runs),gpt-5.1-2025-11-13_high,2025-11-13,67.97520661157024
SWE-bench Verified (Epoch runs),kimi-k2.5,2026-01-27,73.7603305785124
SWE-bench Verified (Epoch runs),gpt-5.1-2025-11-13_high,2025-11-13,65.9090909090909
SWE-bench Verified (Epoch runs),glm-5,2026-02-11,72.07792207792207
SWE-bench Verified (Epoch runs),gemini-2.5-pro,2025-06-17,57.55693581780539
SWE-bench Verified (Epoch runs),gemini-3-pro-preview,2025-11-18,72.93388429752066
SWE-bench Verified (Epoch runs),gpt-5.2-2025-12-11_high,2025-12-11,73.7603305785124
SWE-bench Verified (Epoch runs),o3-2025-04-16_medium,2025-04-16,62.31884057971014
SWE-bench Verified (Epoch runs),claude-opus-4-1-20250805,2025-08-05,73.34710743801654
SWE-bench Verified (Epoch runs),gpt-4o-2024-11-20,2024-11-20,30.99173553719008
SWE-bench Verified (Epoch runs),claude-opus-4-6,2026-02-05,75.6198347107438
SWE-bench Verified (Epoch runs),gpt-4.1-2025-04-14,2025-04-14,48.54166666666666
SWE-bench Verified (Epoch runs),gpt-5-2025-08-07_high,2025-08-07,73.55371900826447
SWE-bench Verified (Epoch runs),claude-opus-4-20250514,2025-05-22,70.66115702479338
SWE-bench Verified (Epoch runs),claude-opus-4-5-20251101,2025-11-24,76.65289256198346
SWE-bench Verified (Epoch runs),claude-sonnet-4-5-20250929,2025-09-29,71.28099173553719
SWE-bench Verified (Epoch runs),gpt-5-2025-08-07_medium,2025-08-07,71.48760330578511
SWE-bench Verified (Epoch runs),claude-3-7-sonnet-20250219,2025-02-24,60.9504132231405
SWE-bench Verified (Epoch runs),gpt-5-mini-2025-08-07_medium,2025-08-07,64.6694214876033
Aider Polyglot,gpt-oss-120b_high,2025-08-05,41.8
Aider Polyglot,Qwen3-235B-A22B,2025-04-29,59.6
Aider Polyglot,DeepSeek-V3.2-Exp,2025-09-29,70.2
Aider Polyglot,DeepSeek-V3.2-Exp_thinking,2025-09-29,74.2
Aider Polyglot,gpt-5-2025-08-07_low,2025-08-07,81.3
Aider Polyglot,gpt-5-2025-08-07_medium,2025-08-07,86.7
Aider Polyglot,gpt-5-2025-08-07_high,2025-08-07,88.0
Aider Polyglot,o3-2025-04-16_medium,2025-04-16,76.9
Aider Polyglot,grok-3-mini-beta_low,2025-04-09,34.7
Aider Polyglot,openhands-lm-32b-v0.1,2024-03-26,10.2
Aider Polyglot,Kimi-K2-Instruct,2025-07-12,59.1
Aider Polyglot,grok-4-0709,2025-07-09,79.6
Aider Polyglot,o3-pro-2025-06-10_high,2025-06-10,84.9
Aider Polyglot,gemini-2.5-pro-preview-05-06,2025-05-06,76.9
Aider Polyglot,gemini-2.5-pro-preview-06-05,2025-06-05,79.1
Aider Polyglot,gemini-2.5-pro-preview-06-05_32K,2025-06-05,83.1
Aider Polyglot,gemini-2.5-flash-preview-05-20_23K,2025-05-20,55.1
Aider Polyglot,gemini-2.5-flash-preview-05-20,2025-05-20,44.0
Aider Polyglot,claude-sonnet-4-20250514,2025-05-22,56.4
Aider Polyglot,claude-sonnet-4-20250514_32K,2025-05-22,61.3
Aider Polyglot,claude-opus-4-20250514,2025-05-22,70.7
Aider Polyglot,claude-opus-4-20250514_32K,2025-05-22,72.0
Aider Polyglot,Qwen3-32B,2025-04-29,40.0
Aider Polyglot,Llama-4-Maverick-17B-128E-Instruct,2025-04-05,15.6
Aider Polyglot,gpt-4.1-mini-2025-04-14,2025-04-14,32.4
Aider Polyglot,gpt-4.1-nano-2025-04-14,2025-04-14,8.9
Aider Polyglot,chatgpt-4o-03-27-2025,2025-03-27,45.3
Aider Polyglot,gemini-2.5-flash-preview-04-17,2025-04-17,47.1
Aider Polyglot,grok-3-mini-beta_high,2025-04-09,49.3
Aider Polyglot,gpt-4.1-2025-04-14,2025-04-14,52.4
Aider Polyglot,grok-3-beta,2025-04-09,53.3
Aider Polyglot,DeepSeek-V3-0324,2025-03-24,55.1
Aider Polyglot,DeepSeek-R1-0528,2025-05-28,71.4
Aider Polyglot,gemini-2.5-pro-exp-03-25,2025-03-25,72.9
Aider Polyglot,o4-mini-2025-04-16_high,2025-04-16,72.0
Aider Polyglot,o3-2025-04-16_high,2025-04-16,81.3
Aider Polyglot,gemini-2.0-flash-thinking-exp-01-21,2025-01-21,18.2
Aider Polyglot,gpt-4o-2024-11-20,2024-11-20,18.2
Aider Polyglot,claude-3-5-sonnet-20241022,2024-10-22,51.6
Aider Polyglot,Qwen2.5-Coder-32B-Instruct,2024-11-21,8.0
Aider Polyglot,claude-3-5-haiku-20241022,2024-10-22,28.0
Aider Polyglot,gemini-exp-1206,2024-12-06,38.2
Aider Polyglot,DeepSeek-R1,2025-01-20,56.9
Aider Polyglot,DeepSeek-V3,2024-12-26,48.4
Aider Polyglot,qwen-max-2025-01-25,2025-01-25,21.8
Aider Polyglot,gemini-2.0-pro-exp-02-05,2025-02-05,35.6
Aider Polyglot,QwQ-32B,2025-03-05,20.9
Aider Polyglot,c4ai-command-a-03-2025,2025-03-13,12.0
Aider Polyglot,gpt-4o-mini-2024-07-18,2024-07-18,3.6
Aider Polyglot,gemma-3-27b-it,2025-03-12,4.9
Aider Polyglot,gpt-4.5-preview-2025-02-27,2025-02-27,44.9
Aider Polyglot,claude-3-7-sonnet-20250219,2025-02-24,60.4
Aider Polyglot,gemini-2.0-flash-exp,2024-12-11,22.2
Aider Polyglot,DeepSeek-V2.5,2024-09-06,17.8
Aider Polyglot,Qwen2.5-Coder-32B-Instruct,2024-11-21,16.4
Aider Polyglot,chatgpt-4o-01-29-2025,2025-01-29,27.1
Aider Polyglot,codestral-2501,2025-01-13,11.1
Aider Polyglot,o1-2024-12-17_high,2024-12-17,61.7
Aider Polyglot,o3-mini-2025-01-31_medium,2025-01-31,53.8
Aider Polyglot,yi-lightning,2024-12-02,12.9
Aider Polyglot,gpt-4o-2024-08-06,2024-08-06,23.1
Aider Polyglot,claude-3-7-sonnet-20250219_32K,2025-02-24,64.9
Aider Polyglot,o3-mini-2025-01-31_high,2025-01-31,60.4
Aider Polyglot,o1-mini-2024-09-12_unknown,2024-09-12,32.9
Aider Polyglot,gemini-2.5-pro-preview-03-25,2025-03-31,72.9
Aider Polyglot,Qwen3-235B-A22B-Instruct-2507,2025-07-25,59.6
Aider Polyglot,o3-2025-04-16_unknown,2025-04-16,76.9
Aider Polyglot,grok-4-0709_high,2025-07-09,79.6
Aider Polyglot,moonshotai/kimi-k2-0905,2025-09-05,59.1
Aider Polyglot,openai/gpt-oss-120b_high,2025-08-05,41.8
Aider Polyglot,deepseek-reasoner,2025-12-01,74.2
Aider Polyglot,deepseek-chat,2025-12-01,70.2
Terminal-Bench,gpt-5.5_unknown,2026-04-23,84.7191011236
Terminal-Bench,gpt-5.5_unknown,2026-04-23,83.1460674157
Terminal-Bench,gpt-5.5_unknown,2026-04-23,82.2471910112
Terminal-Bench,gpt-5.5_unknown,2026-04-23,82.0
Terminal-Bench,gpt-5.4-2026-03-05_unknown,2026-03-05,81.8
Terminal-Bench,gemini-3.1-pro-preview,2026-02-19,80.2247191011
Terminal-Bench,claude-opus-4-7_unknown,2026-04-16,80.2247191011
Terminal-Bench,claude-opus-4-6_unknown,2026-02-05,79.80000000000001
Terminal-Bench,gpt-5.3-codex,2026-02-05,78.4269662921
Terminal-Bench,gemini-3.1-pro-preview,2026-02-19,78.4
Terminal-Bench,gemini-3.1-pro-preview,2026-02-19,78.4
Terminal-Bench,gpt-5.3-codex,2026-02-05,77.3033707865
Terminal-Bench,claude-opus-4-6_unknown,2026-02-05,76.404494382
Terminal-Bench,gpt-5.3-codex,2026-02-05,75.8426966292
Terminal-Bench,gpt-5.3-codex,2026-02-05,75.7303370787
Terminal-Bench,claude-opus-4-6_unknown,2026-02-05,75.2808988764
Terminal-Bench,gpt-5.3-codex,2026-02-05,75.0561797753
Terminal-Bench,gemini-3.1-pro-preview,2026-02-19,74.8314606742
Terminal-Bench,claude-opus-4-6_unknown,2026-02-05,74.7191011236
Terminal-Bench,gpt-5.3-codex,2026-02-05,74.606741573
Terminal-Bench,claude-opus-4-6_unknown,2026-02-05,72.0786516854
Terminal-Bench,claude-opus-4-6_unknown,2026-02-05,71.9101123596
Terminal-Bench,gpt-5.3-codex,2026-02-05,71.4606741573
Terminal-Bench,gpt-5.3-codex,2026-02-05,70.3
Terminal-Bench,claude-opus-4-6,2026-02-05,69.89999999999999
Terminal-Bench,claude-opus-4-6_unknown,2026-02-05,69.8876404494
Terminal-Bench,gemini-3-pro-preview,2025-11-18,69.4257178527
Terminal-Bench,gpt-5.3-codex,2026-02-05,69.138576779
Terminal-Bench,claude-opus-4-6_unknown,2026-02-05,66.872659176
Terminal-Bench,claude-opus-4-6_unknown,2026-02-05,66.5168539326
Terminal-Bench,gpt-5.2-codex,2025-12-18,66.5168539326
Terminal-Bench,gpt-5.5_unknown,2026-04-23,66.0674157303
Terminal-Bench,gemini-3-pro-preview,2025-11-18,65.1685393258
Terminal-Bench,gpt-5.2-2025-12-11_unknown,2025-12-11,64.9438202247
Terminal-Bench,gpt-5.2-2025-12-11_medium,2025-12-11,64.9
Terminal-Bench,gpt-5.3-codex,2026-02-05,64.7191011236
Terminal-Bench,gemini-3-flash-preview,2025-12-17,64.2696629213
Terminal-Bench,claude-opus-4-5-20251101_unknown,2025-11-24,63.1460674157
Terminal-Bench,claude-opus-4-5-20251101,2025-11-24,63.1
Terminal-Bench,gpt-5.2-2025-12-11_unknown,2025-12-11,62.9213483146
Terminal-Bench,claude-opus-4-6_unknown,2026-02-05,62.9213483146
Terminal-Bench,gpt-5.2-2025-12-11_medium,2025-12-11,62.9
Terminal-Bench,gemini-3-pro-preview,2025-11-18,62.2471910112
Terminal-Bench,gemini-3-pro-preview,2025-11-18,62.2
Terminal-Bench,gemini-3-pro-preview,2025-11-18,61.79775280900001
Terminal-Bench,gpt-5.1-codex-mini,2025-11-12,61.5730337079
Terminal-Bench,gemini-3.1-pro-preview,2026-02-19,61.4232209738
Terminal-Bench,gemini-3-pro-preview,2025-11-18,61.1235955056
Terminal-Bench,gpt-5.2-2025-12-11_unknown,2025-12-11,60.6741573034
Terminal-Bench,gpt-5.1-codex-max,2025-11-19,60.4494382022
Terminal-Bench,claude-opus-4-5-20251101_unknown,2025-11-24,59.101123595500006
Terminal-Bench,claude-opus-4-5-20251101_128K,2025-11-24,59.099999999999994
Terminal-Bench,claude-opus-4-5-20251101_unknown,2025-11-24,58.426966292100005
Terminal-Bench,claude-opus-4-6_unknown,2026-02-05,57.9775280899
Terminal-Bench,gpt-5.1-codex,2025-11-12,57.8
Terminal-Bench,gpt-5.1-codex,2025-11-12,57.7528089888
Terminal-Bench,claude-opus-4-5-20251101_unknown,2025-11-24,57.7528089888
Terminal-Bench,grok-4-20,2026-02-17,57.3033707865
Terminal-Bench,grok-4-20,2026-02-17,57.3
Terminal-Bench,gemini-3-pro-preview,2025-11-18,56.85393258429999
Terminal-Bench,gemini-3-pro-preview,2025-11-18,55.9550561798
Terminal-Bench,claude-opus-4-5-20251101_unknown,2025-11-24,54.3258426966
Terminal-Bench,gpt-5.2-2025-12-11_unknown,2025-12-11,54.0449438202
Terminal-Bench,gpt-5.1-codex,2025-11-12,53.4831460674
Terminal-Bench,claude-sonnet-4-6_unknown,2026-02-17,53.4269662921
Terminal-Bench,claude-sonnet-4-6,2026-02-17,53.400000000000006
Terminal-Bench,glm-5,2026-02-11,52.415730337099994
Terminal-Bench,claude-opus-4-5-20251101_unknown,2025-11-24,52.134831460700006
Terminal-Bench,claude-opus-4-5-20251101_unknown,2025-11-24,51.9101123596
Terminal-Bench,gemini-3-flash-preview,2025-12-17,51.685393258400005
Terminal-Bench,claude-opus-4-5-20251101_unknown,2025-11-24,51.685393258400005
Terminal-Bench,gemini-3-flash-preview,2025-12-17,51.0
Terminal-Bench,gpt-5-2025-08-07_unknown,2025-08-07,49.606741573
Terminal-Bench,gpt-5-2025-08-07_medium,2025-08-07,49.6
Terminal-Bench,gpt-5.1-2025-11-13_medium,2025-11-13,47.599999999999994
Terminal-Bench,gpt-5.1-2025-11-13_medium,2025-11-13,47.599999999999994
Terminal-Bench,gpt-5.1-2025-11-13_unknown,2025-11-13,47.5842696629
Terminal-Bench,gemini-3-flash-preview,2025-12-17,47.4157303371
Terminal-Bench,claude-sonnet-4-5-20250929_unknown,2025-09-29,46.5168539326
Terminal-Bench,MiniMax-M2.7,2026-03-18,45.056179775299995
Terminal-Bench,gpt-5-codex,2025-09-15,44.3258426966
Terminal-Bench,gpt-5-codex,2025-09-15,44.3
Terminal-Bench,gpt-5-2025-08-07_unknown,2025-08-07,43.8202247191
Terminal-Bench,gpt-5-2025-08-07_medium,2025-08-07,43.8
Terminal-Bench,gpt-5-codex,2025-09-15,43.4269662921
Terminal-Bench,gpt-5-codex,2025-09-15,43.4
Terminal-Bench,kimi-k2.5,2026-01-27,43.202247191
Terminal-Bench,gpt-5.1-codex-mini,2025-11-12,43.1460674157
Terminal-Bench,claude-sonnet-4-5-20250929_unknown,2025-09-29,43.1460674157
Terminal-Bench,gpt-5.1-codex-mini,2025-11-12,43.1
Terminal-Bench,MiniMax-M2.7,2026-03-18,42.9213483146
Terminal-Bench,claude-sonnet-4-5-20250929_unknown,2025-09-29,42.808988764
Terminal-Bench,claude-sonnet-4-5-20250929,2025-09-29,42.8
Terminal-Bench,claude-sonnet-4-5-20250929_unknown,2025-09-29,42.699999999999996
Terminal-Bench,MiniMax-M2.5,2026-02-12,42.696629213499996
Terminal-Bench,claude-sonnet-4-5-20250929_unknown,2025-09-29,42.696629213499996
Terminal-Bench,claude-sonnet-4-5-20250929,2025-09-29,42.6
Terminal-Bench,claude-sonnet-4-5-20250929_unknown,2025-09-29,42.5842696629
Terminal-Bench,claude-sonnet-4-5-20250929_unknown,2025-09-29,42.52808988759999
Terminal-Bench,claude-sonnet-4-5-20250929,2025-09-29,42.5
Terminal-Bench,MiniMax-M2.5,2026-02-12,42.2471910112
Terminal-Bench,gpt-5-codex,2025-09-15,41.3483146067
Terminal-Bench,gpt-5-codex,2025-09-15,41.3
Terminal-Bench,claude-sonnet-4-5-20250929,2025-09-29,40.1
Terminal-Bench,claude-sonnet-4-5-20250929_unknown,2025-09-29,40.0561797753
Terminal-Bench,deepseek/deepseek-v3.2,2025-12-01,39.5505617978
Terminal-Bench,claude-opus-4-1-20250805,2025-08-05,38.0
Terminal-Bench,claude-opus-4-1-20250805_unknown,2025-08-05,37.9775280899
Terminal-Bench,gpt-5.1-codex,2025-11-12,36.9
Terminal-Bench,claude-opus-4-1-20250805,2025-08-05,36.9
Terminal-Bench,claude-opus-4-1-20250805_unknown,2025-08-05,36.8539325843
Terminal-Bench,gpt-5.1-codex,2025-11-12,36.8539325843
Terminal-Bench,MiniMax-M2.1,2025-12-23,36.629213483099996
Terminal-Bench,kimi-k2-thinking,2025-11-06,35.730337078699996
Terminal-Bench,kimi-k2-thinking,2025-11-06,35.699999999999996
Terminal-Bench,claude-haiku-4-5-20251001_unknown,2025-10-15,35.505617977499995
Terminal-Bench,gpt-5-2025-08-07_medium,2025-08-07,35.199999999999996
Terminal-Bench,gpt-5-2025-08-07_unknown,2025-08-07,35.16853932579999
Terminal-Bench,claude-opus-4-1-20250805,2025-08-05,35.099999999999994
Terminal-Bench,claude-opus-4-1-20250805_unknown,2025-08-05,35.0561797753
Terminal-Bench,claude-opus-4-1-20250805_unknown,2025-08-05,34.8314606742
Terminal-Bench,gpt-5-mini-2025-08-07_unknown,2025-08-07,34.8314606742
Terminal-Bench,claude-opus-4-1-20250805,2025-08-05,34.8
Terminal-Bench,gpt-5-mini-2025-08-07_unknown,2025-08-07,34.8
Terminal-Bench,gpt-5-2025-08-07_unknown,2025-08-07,33.9325842697
Terminal-Bench,gpt-5-2025-08-07_medium,2025-08-07,33.900000000000006
Terminal-Bench,glm-4.7,2025-12-22,33.3895131086
Terminal-Bench,glm-4.7,2025-12-22,33.2584269663
Terminal-Bench,gemini-2.5-pro,2025-06-17,32.6404494382
Terminal-Bench,gemini-2.5-pro,2025-06-17,32.6
Terminal-Bench,gpt-5-mini-2025-08-07_medium,2025-08-07,31.900000000000002
Terminal-Bench,gpt-5-mini-2025-08-07_unknown,2025-08-07,31.853932584299997
Terminal-Bench,MiniMax-M2,2025-10-27,30.037453183500002
Terminal-Bench,claude-haiku-4-5-20251001_unknown,2025-10-15,29.8314606742
Terminal-Bench,claude-haiku-4-5-20251001,2025-10-15,29.799999999999997
Terminal-Bench,MiniMax-M2.1,2025-12-23,29.2134831461
Terminal-Bench,gpt-5-mini-2025-08-07_unknown,2025-08-07,29.2134831461
Terminal-Bench,gpt-5-mini-2025-08-07_medium,2025-08-07,29.2
Terminal-Bench,claude-haiku-4-5-20251001_unknown,2025-10-15,28.314606741600002
Terminal-Bench,claude-haiku-4-5-20251001,2025-10-15,28.299999999999997
Terminal-Bench,Kimi-K2-Instruct,2025-07-12,27.800000000000004
Terminal-Bench,Kimi-K2-Instruct,2025-07-12,27.7902621723
Terminal-Bench,claude-haiku-4-5-20251001_unknown,2025-10-15,27.5280898876
Terminal-Bench,claude-haiku-4-5-20251001,2025-10-15,27.500000000000004
Terminal-Bench,grok-4-0709,2025-07-09,27.200000000000003
Terminal-Bench,grok-4-0709,2025-07-09,27.191011236
Terminal-Bench,Qwen3-Coder-480B-A35B-Instruct,2025-07-31,27.191011236
Terminal-Bench,Kimi-K2-Instruct,2025-07-12,26.7415730337
Terminal-Bench,Kimi-K2-Instruct,2025-07-12,26.700000000000003
Terminal-Bench,gemini-2.5-pro,2025-06-17,26.1
Terminal-Bench,gemini-2.5-pro,2025-06-17,26.0674157303
Terminal-Bench,grok-code-fast-1,2025-08-28,25.842696629200002
Terminal-Bench,grok-code-fast-1,2025-08-28,25.8
Terminal-Bench,grok-4-0709,2025-07-09,25.4
Terminal-Bench,Qwen3-Coder-480B-A35B-Instruct,2025-07-31,25.4
Terminal-Bench,grok-4-0709,2025-07-09,25.393258427
Terminal-Bench,Qwen3-Coder-480B-A35B-Instruct,2025-07-31,25.393258427
Terminal-Bench,qwen3.6-35b-a3b,2026-04-14,24.625468164799997
Terminal-Bench,glm-4.6,2025-09-30,24.531835206
Terminal-Bench,glm-4.6,2025-09-30,24.5
Terminal-Bench,gpt-5-mini-2025-08-07_unknown,2025-08-07,24.0449438202
Terminal-Bench,gpt-5-mini-2025-08-07_medium,2025-08-07,24.0
Terminal-Bench,Qwen3-Coder-480B-A35B-Instruct,2025-07-31,23.9
Terminal-Bench,Qwen3-Coder-480B-A35B-Instruct,2025-07-31,23.8951310861
Terminal-Bench,grok-4-0709,2025-07-09,23.1460674157
Terminal-Bench,grok-4-0709,2025-07-09,23.1
Terminal-Bench,gpt-5-mini-2025-08-07_unknown,2025-08-07,22.247191011199998
Terminal-Bench,gpt-5-mini-2025-08-07_medium,2025-08-07,22.2
Terminal-Bench,gpt-5-nano-2025-08-07_unknown,2025-08-07,21.79775280899999
Terminal-Bench,gemini-2.5-pro,2025-06-17,19.6
Terminal-Bench,gemini-2.5-pro,2025-06-17,19.5505617978
Terminal-Bench,gpt-oss-120b,2025-08-05,18.7
Terminal-Bench,gpt-oss-120b_unknown,2025-08-05,18.689138576799998
Terminal-Bench,gemini-2.5-flash-preview-09-2025,2025-09-25,17.1
Terminal-Bench,gemini-2.5-flash,2025-06-17,17.0786516854
Terminal-Bench,gemini-2.5-flash-preview-09-2025,2025-09-25,16.900000000000002
Terminal-Bench,gemini-2.5-flash,2025-06-17,16.8539325843
Terminal-Bench,gemini-2.5-pro,2025-06-17,16.40449438199999
Terminal-Bench,gemini-2.5-flash,2025-06-17,16.40449438199999
Terminal-Bench,gemini-2.5-pro,2025-06-17,16.400000000000002
Terminal-Bench,gemini-2.5-flash-preview-09-2025,2025-09-25,16.400000000000002
Terminal-Bench,gemini-2.5-flash,2025-06-17,15.4494382022
Terminal-Bench,gemini-2.5-flash-preview-09-2025,2025-09-25,15.4
Terminal-Bench,grok-code-fast-1,2025-08-28,14.2
Terminal-Bench,gpt-oss-120b,2025-08-05,14.2
Terminal-Bench,gpt-oss-120b_unknown,2025-08-05,14.157303370800001
Terminal-Bench,grok-code-fast-1,2025-08-28,14.157303370800001
Terminal-Bench,claude-haiku-4-5-20251001_unknown,2025-10-15,13.9325842697
Terminal-Bench,claude-haiku-4-5-20251001,2025-10-15,13.900000000000002
Terminal-Bench,gpt-5-nano-2025-08-07_medium,2025-08-07,11.5
Terminal-Bench,gpt-5-nano-2025-08-07_unknown,2025-08-07,11.460674157299989
Terminal-Bench,gpt-5-nano-2025-08-07_medium,2025-08-07,9.9
Terminal-Bench,gpt-5-nano-2025-08-07_unknown,2025-08-07,9.8876404494
Terminal-Bench,qwen3.5-9B,2026-02-24,9.2134831461
Terminal-Bench,gpt-5-nano-2025-08-07_medium,2025-08-07,7.9
Terminal-Bench,gpt-5-nano-2025-08-07_unknown,2025-08-07,7.865168539300001
Terminal-Bench,gpt-5-nano-2025-08-07_medium,2025-08-07,7.000000000000001
Terminal-Bench,gpt-5-nano-2025-08-07_unknown,2025-08-07,6.966292134799989
Terminal-Bench,gpt-oss-20b,2025-08-05,3.4000000000000004
Terminal-Bench,gpt-oss-20b_unknown,2025-08-05,3.3707865169
Terminal-Bench,gpt-oss-20b,2025-08-05,3.1
Terminal-Bench,gpt-oss-20b_unknown,2025-08-05,3.0711610487
OSWorld,claude-sonnet-4-6,2026-02-17,72.1
OSWorld,claude-sonnet-4-5-20250929,2025-09-29,62.9
OSWorld,claude-sonnet-4-5-20250929,2025-09-29,58.1
OSWorld,claude-sonnet-4-20250514,2025-05-22,43.9
OSWorld,claude-sonnet-4-5-20250929,2025-09-29,42.9
OSWorld,claude-sonnet-4-20250514,2025-05-22,41.4
OSWorld,claude-3-7-sonnet-20250219,2025-02-24,35.8
OSWorld,claude-3-7-sonnet-20250219,2025-02-24,35.6
OSWorld,computer-use-preview-2025-03-11,2025-03-11,31.3
OSWorld,claude-sonnet-4-20250514,2025-05-22,31.2
OSWorld,computer-use-preview-2025-03-11,2025-03-11,30.5
OSWorld,claude-3-7-sonnet-20250219,2025-02-24,27.1
OSWorld,computer-use-preview-2025-03-11,2025-03-11,26.0
OSWorld,o3-2025-04-16_medium,2025-04-16,23.0
OSWorld,o3-2025-04-16_medium,2025-04-16,17.2
OSWorld,o3-2025-04-16_medium,2025-04-16,9.1
OSWorld,Qwen2.5-VL-72B-Instruct,2024-09-19,5.0
OSWorld,Qwen2.5-VL-72B-Instruct,2024-09-19,4.4
OSWorld,claude-opus-4-5-20251101,2025-11-24,66.3
OSWorld,kimi-k2.5,2026-01-27,63.3
Cybench,claude-opus-4-6_unknown,2026-02-05,93.0
Cybench,claude-opus-4-5-20251101_unknown,2025-11-24,82.0
Cybench,claude-sonnet-4-5-20250929_unknown,2025-09-29,60.0
Cybench,claude-sonnet-4-5-20250929,2025-09-29,55.00000000000001
Cybench,grok-4-0709,2025-07-09,43.0
Cybench,claude-opus-4-1-20250805,2025-08-05,42.0
Cybench,grok-4-1,2025-11-17,39.0
Cybench,claude-opus-4-20250514,2025-05-22,38.0
Cybench,claude-sonnet-4-20250514,2025-05-22,35.0
Cybench,grok-4-fast,2025-09-19,30.0
Cybench,o3-mini-2025-01-31_medium,2025-01-31,22.5
Cybench,claude-3-7-sonnet-20250219,2025-02-24,20.0
Cybench,gpt-4.5-preview-2025-02-27,2025-02-27,17.5
Cybench,claude-3-5-sonnet-20240620,2024-06-20,17.5
Cybench,gpt-4o-2024-11-20,2024-11-20,12.5
Cybench,claude-3-opus-20240229,2024-02-29,10.0
Cybench,o1-mini-2024-09-12_medium,2024-09-12,10.0
Cybench,o1-preview-2024-09-12,2024-09-12,10.0
Cybench,Mixtral-8x22B-Instruct-v0.1,2024-04-17,7.5
Cybench,Llama-3.1-405B-Instruct,2024-07-23,7.5
Cybench,gemini-1.5-pro-001-feb24,2024-02-15,7.5
Cybench,Meta-Llama-3-70B-Instruct,2024-04-18,5.0
TheAgentCompany,claude-3-5-sonnet-20241022,2024-10-22,34.4
TheAgentCompany,gemini-2.0-flash-001,2025-02-05,19.0
TheAgentCompany,gpt-4o-2024-11-20,2024-11-20,16.7
TheAgentCompany,Llama-3.1-405B-Instruct,2024-07-23,14.099999999999998
TheAgentCompany,Llama-3.1-70B-Instruct,2024-07-23,12.8
TheAgentCompany,qwen2.5-72b-instruct,2024-09-19,11.799999999999999
TheAgentCompany,gemini-1.5-pro-002,2024-09-24,8.0
TheAgentCompany,Llama-3.1-70B-Instruct,2024-07-23,6.5
TheAgentCompany,amazon.nova-pro-v1:0,2024-12-03,5.7
TheAgentCompany,qwen2-72b-instruct,2024-06-07,4.2
TheAgentCompany,gemini-2.5-flash-preview-09-2025,2025-09-25,51.800000000000004
TheAgentCompany,DeepSeek-V3.2-Exp,2025-09-29,52.400000000000006
TheAgentCompany,claude-sonnet-4-20250514,2025-05-22,43.2
TheAgentCompany,claude-3-7-sonnet-20250219,2025-02-24,40.2
TheAgentCompany,gemini-2.5-pro-preview-05-06,2025-05-06,39.300000000000004
TheAgentCompany,claude-3-7-sonnet-20250219,2025-02-24,36.4