DS-1000 evaluated code generators on one thousand realistic, Stack Overflow-derived data-science problems across seven Python libraries with execution-based and API-aware checks.
Salesforce Research's CodeT5 adapted the T5 encoder-decoder architecture to programming languages with identifier-aware objectives, creating one model family for both code understanding and generation tasks.
Meta AI's InCoder trained a generative model to fill missing regions using both left and right context, helping establish infilling as a core capability for code editing rather than simple end-of-file completion.
Salesforce's CodeRL treated a code language model as an actor and learned critic models from unit-test outcomes, bringing functional execution feedback into both training and inference for program generation.
DeepMind's AlphaCode tested whether transformer models could do more than complete familiar code by generating and filtering solutions to unseen competitive-programming problems.
OpenAI Codex adapted GPT-style language modeling to public source code and introduced HumanEval as a way to measure whether generated functions actually passed tests rather than merely looked plausible.