Learning track · 32 concepts
AI Systems
Practical AI engineering: LLM apps, RAG, chunking, tool calling, agents, evals, and model/transformer foundations.
What mastery looks like
This track contains 32 connected concepts rather than an unordered reading list. Mastery means you can move from vocabulary to mechanisms, predict how the system behaves under pressure, and support a design decision with code, measurements, or a failure-recovery exercise. For AI Systems, use the track description as the boundary: learn enough detail to reason clearly about practical ai engineering: llm apps, rag, chunking, tool calling, agents, evals, and model/transformer foundations.
A useful explanation names the state involved, the operation that changes it, the resource or safety constraint, and the observable signal that tells you whether the mechanism works. Avoid stopping at product names. Compare at least two approaches, state what each optimizes, and identify what breaks first as scale, concurrency, latency, or uncertainty increases.
Suggested study sequence
Start with ML Math Foundations, Tokenization (LLM) to establish the basic vocabulary. Continue through the core concepts by alternating explanation with an executable drill. Treat Context Packing, Agent Loops, Self-Attention, Multi-Head Attention as integration work: they should combine earlier mechanisms rather than introduce disconnected facts.
At the end of each session, record one decision you can now make, one failure mode you can now predict, and one unanswered question. Revisit that question through the linked primary sources, then prove the answer in the Playground or a real repository. The track is complete when you can transfer the reasoning to an unfamiliar system, not when every page has been opened.
Roadmaps
Concepts in this track
core
RAG
Retrieval-Augmented Generation: ground an LLM answer in retrieved context.
core
Chunking
Splitting documents into retrievable units that preserve meaning.
advanced
Context Packing
Ordering and budgeting retrieved context within the model's window.
core
Structured Outputs
Forcing LLM output into a validated JSON schema.
core
Tool Calling
Letting an LLM invoke functions/APIs via structured calls.
advanced
Agent Loops
The plan → act → observe loop, with memory and stopping conditions.
core
Model Routing
Sending each request to the cheapest model that can handle it.
core
Prompt & Version Logging
Treating prompts as versioned artifacts with logged inputs/outputs.
core
LLM Evals
Measuring LLM output quality with datasets, graders, and LLM-as-judge.
intro
ML Math Foundations
Vectors, matrices, dot products, matmul, shapes.
core
Gradient Descent
Loss surfaces, learning rate, SGD steps.
core
Backpropagation
Chain rule, autograd, forward/backward passes.
core
Softmax & Cross-Entropy
Logits to probabilities, negative log-likelihood loss.
core
AdamW Optimizer
Moments, weight decay, gradient clipping.
intro
Tokenization (LLM)
Byte-level, char-level, BPE, vocab design.
core
Language Modeling
Next-token prediction, context windows, perplexity.
core
Sampling & Decoding
Temperature, top-k, greedy decoding.
core
Embeddings (Transformer)
Token & position embeddings, tied weights.
advanced
Self-Attention
Q/K/V, scaled dot-product, causal masking.
advanced
Multi-Head Attention
Parallel heads, head_dim split, output projection.
advanced
Transformer Block
Pre-LayerNorm, residuals, MLP, GELU.
advanced
Training & Debugging
Init, NaNs, overfit tests, gradient checks.
core
Checkpointing
Weights + optimizer state, resume, dataset manifests.
advanced
LoRA & PEFT
Frozen base, low-rank adapters, rank/alpha.
advanced
RL Alignment (GRPO & Policy Gradient)
Policy gradient, advantages, GRPO/CISPO, on- vs off-policy RLHF.
core
Training Data Engineering
Cleaning, dedup, JSONL tasks, memorization tests.
advanced
Browser ML Runtime
Web Workers, WASM, OPFS, TypedArrays.
advanced
WebGPU Compute
WGSL, compute kernels, matmul, CPU parity.
core
Model Evaluation
Held-out loss, baselines, hallucination, leakage.
core
Model Pre-training
Data mixtures, next-token objectives, scaling laws, distributed training, checkpoints, and training stability.
core
Model Quantization
Post-training and quantization-aware methods, integer and low-bit formats, calibration, kernels, and quality trade-offs.
core
Open-Weight Models
Model cards, licenses, weights, tokenizers, chat templates, adapters, provenance, and reproducible packaging.