The Digest
Short technical reads on how real systems are built. New ones drop on a cadence; everything published is here.
Deep Dive · Issue 21
How an LLM Server Stalls After 200 OK
The server returns 200 OK, then silence. What breaks between the header and the first token.
System Design · Issue 21
Inside ConServe's 84× Latency Cut for Co-Served LLMs
How UCLA and Berkeley built a scheduler that co-serves batch and real-time LLM inference on the same GPUs without collapsing online latency.
Deep Dive · Issue 20
Why More RAM Made LLM Inference Slower
A 364 GB server ran Llama 3 slower than a 48 GB one. The reason is the memory wall.
Paper · Issue 20
How ConServe Preempts LLM Tasks at Layer Granularity
ConServe interrupts LLM batches between transformer layers, cutting preemption latency from hundreds of milliseconds to single digits.
Deep Dive · Issue 20
How Burst Traffic Kills LLM Serving
A sudden flood of long prompts exposes four failure modes stacked across compute, scheduling, memory, and network.
News · Issue 19
How One Redis Config Broke GitHub Actions
A default Redis client setting meets a CI startup race, and the error message tells you exactly what to fix.
System Design · Issue 19
Why AWS Chooses to Fail 5% of Customers Completely
AWS would rather fail a small slice of customers completely than degrade for everyone. Here's the architecture behind that choice.
Deep Dive · Issue 19
How Two CI/CD Breaches Shared One Fatal Flaw
Codecov and CircleCI used different attack vectors, but both exploited the same assumption: every secret in a pipeline is available to every line of code running inside it.
Paper · Issue 18
The 37/31 split: why configuration pushes cause nearly as many incidents as code pushes
Config gets a fraction of the safety tooling that code does, and it shows up in the incident stats.
Deep Dive · Issue 18
The stale-state overwrite: why a one-time freshness check is a time bomb in distributed coordination
A lock you hold is not a lock you own. If your process pauses, the lease expires, and a freshness check can't save you.
News · Issue 18
How a stale-check race in DynamoDB's DNS layer took down AWS US-EAST-1 for 14 hours
A stale DNS plan, a cleanup with no guard, and a retry storm that turned a 3-second wipe into a 14-hour regional collapse.
News · Issue 17
The audit extension that froze Postgres: how a routine table migration triggered a system-catalog lock chain
A routine table migration hung in production because audit triggers fired per-row inside the same transaction, turning a seconds-long operation into a synchronous write storm.
News · Issue 17
Why the same BGP prefix leak recurred six years after the first postmortem
Six years of postmortems and RPKI deployment later, BGP route leaks still propagate the same way because the protocol enforces nothing and the structural fixes are half-deployed.
System Design · Issue 17
The startup that almost died migrating to Kubernetes for non-technical reasons
187 users, $12,400/month on AWS. A $24 VPS saved the company.
Deep Dive · Issue 16
The config update double standard: why infrastructure changes bypass the guardrails code goes through
IaC changes are 10% of commits but take disproportionate review effort, and the LLM guardrails meant to catch them are probabilistic theater.
Deep Dive · Issue 16
The price you see vs. the price you pay: how marketplaces handle search-vs-checkout consistency
Search shows one price, checkout charges another. The gap is built into the architecture, and checkout-time recalculation is the safety net.
Deep Dive · Issue 16
Before You Hash, Define "Duplicate"
Your definition of what counts as a duplicate cascades into every algorithm, data structure, and cost in your dedup pipeline.
Deep Dive · Issue 15
Why the Five Whys makes your postmortems worse
A 1950s Toyota technique is still the default in software postmortems. It drifts toward blame, produces false confidence, and suppresses the operational data postmortems exist to capture.
System Design · Issue 15
Slack's 2022 cache-miss cascade: how one missing key fanned out across every shard
A routine rollout swapped out cache nodes, and a single miss turned into a query to every database shard.
System Design · Issue 15
The leak only production traffic could expose, and the dependency chain that blocked rollback
Mistral's vLLM setup leaked 400 MB/min in a way no profiler could see, and the fix was trapped two layers below the code they owned.
System Design · Issue 14
Why Slack Rebuilt Around Cells After Its Cache Collapsed
Slack's cache cascaded into a metastable meltdown, and a quietly rotting availability zone poisoned every request. Both exposed the same gap: nothing was contained. Here's the drain button they built.
Deep Dive · Issue 14
The Postgres Extension That Quietly Held the Database Hostage
CREATE EXTENSION runs a stranger's SQL as superuser. Here's how a scheduling extension turns that into a covert shell.
News · Issue 14
The Cache Stampede That Ate Slack: Inside the 2022 Cascading Failure
Three healthy systems, each doing exactly its job, took Slack down for a morning. Here's how they collided.
Deep Dive · Issue 13
Why Your Optimization Might Just Be Reshuffling the Cache
Your speedup might be moving the bottleneck around the memory hierarchy instead of removing it.
Paper · Issue 13
The Benchmark Lie: How Rearranging Environment Variables Can Flip Which Compiler Wins
Same code, same compiler, same input, different winner. A 2009 paper showed an unused environment variable can swing a benchmark by almost 300%.
System Design · Issue 13
How GitHub's Telemetry Pipeline Hits 1.6 Million Events a Second
A 700TB/day logging firehose outgrew its hand-tuned Java stack. The fix was serverless, one batch size, and a ceiling set by partition count.
System Design · Issue 12
How Google Scaled Chrome's Bug-Hunting Pipeline to Fix More Flaws in Two Releases Than the Prior 23 Combined
Chrome 149 and 150 fixed 1,072 security bugs in a single month, beating the previous two years of releases put together. Here is the AI pipeline behind the jump.
News · Issue 12
How a 2021 Firmware Bug Let Attackers Predict Coldcard Seeds and Steal $70M in 41 Minutes
One line changed in March 2021 quietly swapped hardware randomness for a predictable software PRNG. Five years later, someone noticed.
Deep Dive · Issue 12
Why OAuth Device-Code Phishing Became 2026's Fastest-Growing Attack, Explained
RFC 8628 assumed the login code always reaches you through a screen you trust. Attackers just email it to you instead, and everything after that is real.
News · Issue 11
Cursor's Vanishing Price Tag: Why an AI Code Editor Pulled Its Own Cost Data
In about a year Cursor changed how it charges three times, and each step made the true cost of a single request harder to see.
Deep Dive · Issue 11
How Go's container/ Package Finally Brings Ordered Maps and Sets to the Standard Library
For 16 years Go shipped no set and no ordered map. Generics and iterators finally made the library types worth building.
System Design · Issue 11
Inside Wafer's Stack for Running Kimi K3 on AMD MI355X
Kimi K3 is so big that memory, not FLOPs, picks the GPU. That quietly reopens the AMD-vs-Nvidia serving fight.
News · Issue 10
Why AI's Top Labs Have Gone Quiet: The Research Publishing Slowdown
The labs that open-sourced the transformer now sit on their best work. Here's what changed, when, and what it costs.
System Design · Issue 10
How GitHub Engineered Stacked Pull Requests for Millions of Repositories
Splitting a big change into a chain of small PRs used to mean rebasing branches by hand. Here is how GitHub made the stack a first-class object.
Deep Dive · Issue 10
Inside Keychron's Open-Source Gaming Mouse Firmware
Keychron wants to do to gaming mice what QMK did to keyboards. So far the repo is a promise, not a codebase.
News · Issue 9
The DeepSeek Compute-Gap Memo: What the Leaked Transcript Reveals About China's AI Ambitions
DeepSeek's founder told investors exactly how far behind China's AI sits, in chips and in years. Then the transcript got out.
News · Issue 9
Anthropic Draws a Line: Inside Its 'Position on Open-Weights Models'
Every major AI lab signed a letter defending open weights. Anthropic didn't, then published a post explaining exactly where it draws the line.
Deep Dive · Issue 9
Golang's Swiss Tables: How Go Quietly Replaced Its Map Internals
Go 1.24 swapped out the guts of every map without touching a line of your code. Here is the hash table that did it, and the tricks Go had to add to make it fit.
News · Issue 8
OpenAI Publishes Codex's Security Model: Inside the Sandbox After Agent Scrutiny
How OpenAI actually boxes in its own coding agent, down to the config file, and the bug reports that pushed the details into the open.
News · Issue 8
The GrapheneOS Panic-Wipe Case: When Anti-Forensics Meets Border Search Law
A phone wiped itself at an Atlanta airport. Now the traveler is facing federal charges, and the twist is that a CBP officer typed the code.
Deep Dive · Issue 8
Inside Zig's Incremental Compilation Engine
Change one line, get a new binary in milliseconds. Here is the machinery that makes Zig rebuild only what actually moved.
Deep Dive · Issue 7
PGSimCity: What Building a City-Builder Taught Us About Postgres Internals
Postgres will tell you which page it evicted, never how the clock hand got there. So someone built a 3D city to show the how.
News · Issue 7
Inside Anthropic's $1.5B Book-Piracy Settlement
Anthropic won the argument that training on books is fair use, then paid a record sum anyway. Here's why.
System Design · Issue 7
Inside Fireworks AI's Stack for Serving Kimi K3 at SOTA Cost and Latency
A 2.8-trillion-parameter open model only beats the closed APIs if you can serve it fast and cheap. Here is how Fireworks makes that case.
Deep Dive · Issue 6
Speculation Is All You Need: How Speculative Decoding Wins the Latency Budget
Decode is sequential and memory-bound, so the GPU sits half-idle. Speculative decoding fills that slack with guessed tokens, for multiplicative speedups.
System Design · Issue 6
How Supercell Made Brawl Stars Load in 5 Seconds
A 35-second load on old Android phones was quietly gating when people played. Here is how the Titan team tore apart the loading pipeline.
News · Issue 6
Inside Hugging Face's Autonomous-Agent Breach
An AI model built by OpenAI found a zero-day, escaped its sandbox, and broke into Hugging Face. No human was driving.
System Design · Issue 5
How 2 Engineers Built Supercell's Cross-Game Social Backend
Chat, presence, and friends for five billion-dollar games, run by two people on a single abstraction.
News · Issue 5
Inside the Wipe: How a Single Attacker Took Down Romania's Land Registry Database
Leaked passwords and unpatched bugs let one hacker freeze Romania's entire property market for a week. Here is how the break-in actually worked.
System Design · Issue 5
How to Price Serverless GPUs
One ratio decides whether a reserved GPU contract or pay-as-you-go is cheaper. Here is the math Modal uses.
News · Issue 4
The LG Monitor Update Fiasco: When Windows Update Becomes a Silent Software Installer
Plug in an LG monitor and Windows Update quietly installs an app that shows you McAfee ads. Here's the pipeline that makes it possible.
System Design · Issue 4
How Supercell Handles Real-Time Persisted Events with ScyllaDB
Two engineers built the real-time social backbone for hundreds of millions of players. Here is the one abstraction that made it possible.
Deep Dive · Issue 4
Achieve State-of-the-Art Inference Latencies with Speculative Decoding
GPUs sit idle while an LLM types one token at a time. Speculative decoding fills that slack by guessing ahead and checking in bulk.
News · Issue 3
Inside the Stripe-Advent Bid for PayPal
A $53 billion cash offer, zero official confirmation, and a financing trick that lets Stripe stay private. Here's what's actually going on.
News · Issue 2
Inside Apple's Trade-Secrets Lawsuit Against OpenAI
A 41-page complaint, an authentication bug, and hardware parts smuggled into job interviews. What Apple actually alleges.
News · Issue 2
Inside AWS's $1.7 Billion Billing Bug
For a day in July 2026, AWS told a hobbyist his $5 account owed $1.7 billion. Here is what actually broke.
Deep Dive · Issue 2
How Homomorphic Encryption Now Runs CIFAR-10 Inference in 200ms
Encrypted image classification went from days to seconds. A new vendor demo claims 200ms. Here is the engineering that actually got us there.
System Design · Issue 1
How Discord Stores Trillions of Messages
By 2022, Discord's message store ran on 177 Cassandra nodes and paged its on-call team most nights. Here is how a new C++ engine plus a Rust tier in front of it got them to 72 calm nodes.
Paper · Issue 1
Why vLLM Uses PagedAttention
Serving an LLM wastes most of its memory on empty reservations. The fix was hiding in a 50-year-old operating-systems trick.
News · Issue 1
OpenAI's Confidential IPO Filing
OpenAI confirmed a confidential S-1 that reveals almost no numbers, but the timing alone signals how badly the AI labs now need public-market money.