Articles
Worth reading
A short take on pieces worth your time — architecture, AI, system design, and engineering careers — each one linked back to its original source.
7 articles
In-House LLM Serving at Netflix
Most companies just call a hosted LLM API and move on; Netflix's AI Platform team explains why and how they run the full inference stack themselves instead, inside their existing production environment rather than spinning up a separate ML silo. What's genuinely useful here is the decision-making, not just the architecture — the tradeoffs they weighed on cost, latency, and operational ownership before choosing to build rather than buy. Good grounding if you're evaluating whether your org actually needs to run its own inference infrastructure.
Netflix Tackles Data Deletion at Scale with Centralized Platform Architecture
InfoQ's coverage of a QCon talk on a problem almost nobody designs for up front: how do you actually delete data, correctly and completely, across dozens of heterogeneous storage systems, at a scale where 76.8 billion row deletions across 1,300 datasets is a normal workload? Netflix's centralized deletion platform is a reminder that "delete" is a distributed systems problem with its own consistency and observability requirements, not an afterthought bolted onto each service. Especially relevant if you're anywhere near GDPR or CCPA compliance work and have been treating deletion as someone else's problem.
Netflix's Distributed Counter Abstraction
Netflix's engineers describe the counting service they built on top of their TimeSeries abstraction to handle 75,000 counter requests per second at single-digit-millisecond latency, the kind of "boring" infrastructure problem that turns out to be genuinely hard once you need it fast and consistent-enough at global scale. I like this as a case study in building a narrow, well-defined abstraction on top of an existing platform primitive instead of reinventing storage from scratch for every new use case. Relevant to anyone building shared counting or aggregation infrastructure.
How Discord Stores Trillions of Messages
Bo Ingram's account of Discord's migration off vanilla Cassandra to a Rust-based data service layer is one of the more honest scaling stories out there — no rewrite-everything triumphalism, just a clear-eyed walk through where their original design started to show cracks at trillion-row scale and what they changed. The details on data modeling for a chat workload (message ordering, hot partitions, compaction pressure) are transferable to basically any high-write-volume, append-heavy system. One of the better "how it actually broke and what we did about it" posts I've read.
Timeouts, retries, and backoff with jitter
Marc Brooker's entry in the AWS Builders' Library is the clearest explanation I've read of why naive retry logic makes outages worse, not better: synchronized retries from thousands of clients can turn a brief blip into a self-inflicted thundering herd. The fix, jittered exponential backoff paired with timeout budgets tied to real latency percentiles, is simple to state and easy to get wrong in practice. I reference this every time I review a service's failure-handling code.