Capability · Global · Apr 7, 2025

4.4% of SWE-bench coding problems could be solved by AI systems in 2023

4.4%

of SWE-bench coding problems could be solved by AI systems in 2023

The AI Index uses SWE-bench as a software-engineering benchmark.

“On SWE-bench, AI systems could solve just 4.4% of coding problems in 2023—a figure that jumped to 71.7% in 2024.”