Why we no longer evaluate SWE-bench Verified

OpenAI has decided to stop evaluating SWE-bench Verified due to its contamination and poor measurement of coding progress. According to their analysis, the tests are flawed and there is training leakage.

Summary written by Kernelia from the original article by OpenAI News. The story and its rights belong to its author.