Collection

Tag: code-evaluation

LiveCodeBench and the Fight Against Benchmark Contamination

LiveCodeBench introduced continuously refreshed programming-contest problems with known publication dates, giving code-model evaluation a way to test on tasks likely to be newer than a model's training cutoff.

↗

SWE-bench and the Shift from Code Completion to Real Repository Repair

SWE-bench changed coding-model evaluation by asking systems to resolve real GitHub issues inside full repositories, making tests, repository context and patch correctness central to measuring progress.

↗