Theo Mercer

CodeBERT and the Moment Source Code Became a Pretraining Domain

CodeBERT treated programming languages and natural language as a joint pretraining problem, helping establish source code as a first-class domain for Transformer models before generative coding assistants became mainstream.

OpenAI Codex, HumanEval, and the Leap from Language Modeling to Code Generation

OpenAI Codex adapted GPT-style language modeling to public source code and introduced HumanEval as a way to measure whether generated functions actually passed tests rather than merely looked plausible.

GitHub Copilot and the Arrival of Generative AI Inside the Code Editor

GitHub Copilot placed large-model code generation directly in the editor, turning AI assistance from a separate research demo into suggestions delivered at the moment developers wrote software.

AlphaCode and the Attempt to Solve Novel Programming Problems with Language Models

DeepMind's AlphaCode tested whether transformer models could do more than complete familiar code by generating and filtering solutions to unseen competitive-programming problems.

THEO MERCER / 4 SRC

Replit Ghostwriter and the Move from AI Completion to an AI-Native IDE

Replit's 2022 Ghostwriter launch integrated code completion, generation, transformation and explanation into a browser-based development environment, making the IDE itself a delivery surface for generative AI.

Code Llama and the Open-Weight Expansion of Specialized Coding Models

Meta's 2023 Code Llama release showed how a general foundation model could be further trained into code-specialized, Python-specialized and instruction-following variants with infilling support.

SWE-bench and the Shift from Code Completion to Real Repository Repair

SWE-bench changed coding-model evaluation by asking systems to resolve real GitHub issues inside full repositories, making tests, repository context and patch correctness central to measuring progress.

StarCoder and BigCode’s Experiment in Open, Governed Code Models

StarCoder paired a strong open code model with an unusually visible discussion of training data, licensing, opt-out mechanisms and model governance through the BigCode collaboration.

Transformers and the Architecture That Made Code Language Models Possible

The transformer was introduced for machine translation, but its attention-based architecture became the foundation for the large language models later adapted to source code.

Flash Fill and the Pre-LLM History of Programming by Example

Years before large language models generated code from natural-language prompts, programming-by-example systems such as Flash Fill synthesized small programs from demonstrations of the desired result.