Microsoft’s phi-1 showed that a 1.3B-parameter code model trained on curated and synthetic “textbook quality” data could achieve strong results on compact Python coding benchmarks.
DeepCoder combined neural prediction with classical program search, showing an early way for deep learning to guide synthesis from input-output examples without generating complete source code directly.
CodeBERT treated programming languages and natural language as a joint pretraining problem, helping establish source code as a first-class domain for Transformer models before generative coding assistants became mainstream.