THE LIVING ARCHIVE

From punched cards to autonomous collaborators: the ideas, people, languages and systems that taught machines to follow—and increasingly interpret—human intent.

PAST / PRESENT / NEXT

Collection

Tag: web-architecture

Mercator and the Engineering of a Scalable Web Crawler

Mercator documented the crawler as a distributed systems problem: URL scheduling, duplicate detection, politeness, parsing, extensibility, and persistent state all had to work at Web scale.

↗

REST and the Decision to Treat the Web as Resources and Representations

Roy Fielding's REST described the architectural constraints behind the scalable Web and later gave developers a vocabulary for designing network APIs around resources, representations and uniform interfaces.

↗