Resources
Reference library
The course curates a focused set of videos, books, research papers, and engineering blogs. These are the anchors for the weekly modules and the inference-performance weeks in particular.
Coming in without the fundamentals?
There is an optional 36-week prerequisite lane: CS50x, Nand to Tetris, MIT 6.042J, Statistical Learning, Git, search and docs fluency, programming paradigms, and Software Engineering at Google, in the order to take them, with a week-by-week budget. See the on-ramp →
YouTube playlists
Video reference
Books & reading list
| Book | Pages | Topic |
|---|---|---|
| Designing ML Systems | 118–132 | Labeling, natural labels, weak supervision |
| Designing ML Systems | 149–156 | Data augmentation, synthetic-to-real gaps |
| Designing ML Systems | 224–247 | Deployment, batch vs online, compression |
| Designing ML Systems | 267–290 | Production failures, distribution shift, monitoring |
| Practical MLOps | 181–204 | Logging, observability, model monitoring, drift |
| Acing System Design | 203–226 | Rate limiting, token bucket, sliding windows |
| ML Interviews | 176–189 | Live coding, thinking aloud, edge cases |
| Designing ML Systems | 174–180 | Data leakage, contamination detection |
| Designing ML Systems | 182–195 | Evaluation, experiment tracking, versioning |
| Designing ML Systems | 203–217 | Baselines, offline evaluation, slices, CIs |
| Designing ML Systems | 316–325 | A/B testing, shadow deployment, canary, bandits |
| ML Interviews | 163–170 | Metrics - Recall@K, MRR, NDCG trade-offs |
| Acing System Design | 277–296 | Search architecture, candidate generation, freshness |
| ML Interviews | 117–122 | Recommender systems refresher |
| Acing System Design | 56–79 | Requirements, trade-offs, API, data model, observability |
| Acing System Design | 86–107 | Scalability, latency, privacy, tenant isolation |
| Designing ML Systems | 52–76 | Business objectives, constraints, iterative development |
| Software Architecture | 317–339 | Orchestration vs choreography, workflow state ownership |
| Software Architecture | 383–397 | Contracts, schema evolution, backward compatibility |
| Software Architecture | 417–433 | Trade-off analysis, MECE reasoning, bottom line over evidence |
| System Design | 191–195 | Interview method checklist - read night before |
| Practical MLOps | 181–204 | Four-layer monitoring framework |
| ML Interviews | 222–239 | Monitoring, MLOps, system design questions |
Research papers
| Paper | Venue | Topic |
|---|---|---|
| The Optimal Choice of Hypothesis Is the Weakest, Not the Shortest | arXiv 2301.12987v4 (2024) | Induction, weakness, description length, and generalization |
| PagedAttention / vLLM | SOSP 2023 | KV cache paging |
| Orca | OSDI 2022 | Continuous batching |
| SGLang | NeurIPS 2024 | RadixAttention, prefix reuse |
| SARATHI / Sarathi-Serve | OSDI 2024 | Chunked prefill |
| DistServe | OSDI 2024 | Prefill/decode disaggregation |
| Splitwise | ISCA 2024 | Phase splitting across GPU types |
| Mooncake | FAST 2025 | KV-cache-centric serving |
| StreamingLLM | ICLR 2024 | Attention sinks, long context |
| EAGLE | arXiv | Speculative sampling |
| FlashAttention | NeurIPS 2022 | IO-aware attention |
| FlashAttention-2 | ICLR 2024 | Faster attention kernels |
| FlashAttention-3 | - | Hopper-optimized attention |
| Transformers are RNNs (Linear Attention) | ICML 2020 | Linear attention, ELU+1 feature map |
| Mamba | arXiv 2023 | Selective state spaces, linear-time sequence modeling |
| Transformers are SSMs (Mamba-2) | arXiv 2024 | Structured state space duality |
| DeltaNet | NeurIPS 2024 | Delta-rule linear attention |
| Gated DeltaNet | ICLR 2025 | Gated delta rule over Mamba-2 |
| Ring Attention | ICLR 2024 | Long-context distributed attention |
| Infini-attention | NeurIPS 2024 | Infinite context |
| KIVI | ICML 2024 | KV cache quantization |
| H2O | NeurIPS 2023 | Heavy hitter KV eviction |
| SnapKV | NeurIPS 2024 | KV cache compression |
| MInference | NeurIPS 2024 | Dynamic sparse attention |
| Medusa | ICML 2024 | Multi-head speculative decoding |
| SpecInfer | SIGMOD 2024 | Speculative inference |
| Lookahead Decoding | ACL 2024 | Faster decoding |
| TensorRT-LLM | NVIDIA | Production inference optimization |
| DeepSpeed-Inference | Microsoft | High-performance inference |
| FasterTransformer | NVIDIA | Transformer inference |
| Megatron-LM | SC 2019 | Large-scale transformer training |
| DeepSpeed | KDD 2020 | ZeRO optimizer |
| ZeRO | SC 2020 | Optimizer state partitioning |
| ZeRO-Infinity | SC 2021 | Extreme-scale training |
| FSDP | PyTorch | Fully Sharded Data Parallel |
| Switch Transformer | JMLR | Mixture of Experts |
| GShard | ICLR 2021 | Sparse expert scaling |
| DeepSeekMoE | arXiv | Modern MoE architecture |
| COMET | arXiv 2502.19811 | Fine-grained computation-communication overlap for MoE |
| LoRA | ICLR 2022 | Parameter-efficient fine-tuning |
| QLoRA | NeurIPS 2023 | 4-bit fine-tuning |
| DoRA | ICML 2024 | Weight decomposition tuning |
| RLHF | NeurIPS 2022 | Human feedback alignment |
| Direct Preference Optimization (DPO) | NeurIPS 2023 | Preference optimization |
| ORPO | ICLR 2024 | Odds-ratio preference optimization |
| ColBERT | SIGIR 2020 | Late interaction retrieval |
| ColBERTv2 | NAACL 2022 | Efficient retrieval |
| HyDE | ACL 2023 | Hypothetical document retrieval |
| RAPTOR | arXiv 2024 | Hierarchical RAG |
| GraphRAG | Microsoft | Graph retrieval |
| Self-RAG | ICLR 2024 | Self-reflective retrieval |
| CRAG | NeurIPS 2024 | Corrective retrieval augmentation |
Agent & Graph-RAG papers
The anchors for the GenAI agents track: ReAct, tool-use, DSPy, MCP, corrective RAG, and Graph-RAG.
| Paper | Source | Focus Area |
|---|---|---|
| ReAct: Synergizing Reasoning and Acting in Language Models | arXiv 2210.03629 | Foundation of ReAct agents |
| Toolformer: Language Models Can Teach Themselves to Use Tools | arXiv 2302.04761 | Self-supervised tool-use agents |
| DSPy: Compiling Declarative Language Model Calls | arXiv 2310.03714 | Programming LMs, self-improving pipelines |
| LangGraph: Multi-Agent Workflows | LangChain Docs | Graph-based orchestration, cycles, human-in-the-loop |
| Corrective RAG (CRAG) | arXiv 2401.06999 | Self-correcting retrieval loops |
| Model Context Protocol (MCP) Specification | Model Context | Standardized tool/data access for LLMs |
| Generative Agents: Interactive Simulacra of Human Behavior | arXiv 2304.03442 | Simulating multi-agent societies |
| Reflexion: Language Agents with Verbal Reinforcement Learning | arXiv 2303.11366 | Self-verbalization and strategy updates |
| AutoGen: Enabling Next-Gen Multi-Agent Applications | Microsoft Research | Collaborative protocol design |
| Kùzu: An Embedded Property Graph Database | Kùzu | Architecture and performance of Kùzu |
| Building Effective Agents | Anthropic 2024 | Workflows vs agents, harness design patterns |
| Best Practices for Claude Code | Anthropic (engineering) | Coding-agent harness: approval gates, session state, context budget, sandboxing |
Blog & article URLs