Supplement: Topics the Book Named but Never Taught

This supplement fills the gaps found in the main book. Each topic gives a plain explanation, a small example, and a snippet where one fits.

Where each topic belongs. Read S1–S4 after Day 12 (CNN), S5 after Day 6, S6 after Day 11, and S7 after the Transformer / GenAI part.

S1. Transfer Learning (Day 12)

Idea: do not train from random weights. Start from a network already trained on a huge dataset (for images, ImageNet with 1.2M images) and adapt it to your smaller task. Early layers already detect edges and textures that almost every image task needs.

Two ways to do it

• Feature extraction: freeze all pretrained layers, replace only the final classifier, train just that. Fast, works with little data.
• Fine-tuning: unfreeze some or all layers and train them with a much smaller learning rate (e.g. 1e-5 to 1e-4) so you do not destroy what was learned.

When to use which: tiny dataset and similar domain → feature extraction. More data or a different domain → fine-tune the last blocks, then more if it helps.

import torch.nn as nn
from torchvision import models
m = models.resnet18(weights=models.ResNet18_Weights.DEFAULT)   # pretrained
for p in m.parameters(): p.requires_grad = False               # freeze everything
m.fc = nn.Linear(m.fc.in_features, 10)
# new head (trainable)
opt = torch.optim.Adam(m.fc.parameters(), lr=1e-3)
# optimize ONLY the head
# Inputs must match the pretrained model: 3 channels, ImageNet mean/std
# normalization.
# MNIST is 1-channel: repeat it to 3 channels or use x.repeat(1,3,1,1).

Common mistakes: forgetting to normalize with the pretrained model's own mean/std; passing the frozen parameters to the optimizer; using a high learning rate when fine-tuning; leaving batch-norm layers in train mode when the batch is tiny.

S2. Data Augmentation

Idea: create extra training examples by applying small random changes that do not change the label. A slightly rotated 7 is still a 7. The model sees more variety, so it memorizes less. It is an anti-overfitting tool like dropout, but works on the data.

from torchvision import transforms
train_tf = transforms.Compose([
    transforms.RandomRotation(10),                       # ±10 degrees

    transforms.RandomAffine(0, translate=(0.1, 0.1)),    # shift up to 10%
    transforms.ToTensor(),
    transforms.Normalize((0.1307,), (0.3081,)),
])
val_tf = transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.1307,),
    (0.3081,))])
# Rule: augment TRAIN only. Validation and test stay untouched.

• Only label-preserving changes. Flipping a 6 upside down makes it look like a 9, so do not flip digits. Flipping a photo of a cat left-right is fine.
• Text has its own versions: synonym swaps, back-translation. Tabular data usually needs other tools (more data, regularization).
• Expect the training loss to be a bit higher and validation to improve. That is the point.

S3. Batch Normalization

Idea: inside a network, each layer's inputs keep shifting as earlier layers change. BatchNorm re-centers and re-scales each feature over the current mini-batch, then lets the network learn its own scale (γ) and shift (β). Training becomes more stable and tolerates higher learning rates.

Formula for one feature over a batch: x̂ = (x − mean) / √(var + ε), then y = γ·x̂ + β.

x  = np.array([[1., 100.], [2., 200.], [3., 300.]])
# 3 samples, 2 features, very different scales
xn = (x - x.mean(0)) / np.sqrt(x.var(0) + 1e-5)

xn = [[-1.22, -1.22], [0.0, 0.0], [1.22, 1.22]] # mean 0, std 1 for every feature

• Train vs eval: in training it uses the batch's own mean/variance; in eval it uses running averages saved during training. This is another reason for model.eval().
• PyTorch: nn.BatchNorm1d(features) after Linear, nn.BatchNorm2d(channels) after Conv2d. Typical order: Conv → BatchNorm → ReLU.
• BatchNorm vs LayerNorm: BatchNorm averages across the batch (needs a reasonable batch size, awkward for variable-length text). LayerNorm averages across the features of one sample, so it works with any batch size. That is why Transformers use LayerNorm.

S4. Test, Checkpoint and Inference (Day 11 workflow)

The Day 11 opener lists Train → Validate → Test → Checkpoint → Inference. The body covered train and validate. The rest:

# 1. TEST once, at the very end, with the best checkpoint (never tune on it)
model.load_state_dict(torch.load("best.pt")); model.eval()
test_loss, test_acc = evaluate(model, test_loader, criterion)

# 2. CHECKPOINT the best model during training (by validation loss)
if va_loss < best: best = va_loss; torch.save(model.state_dict(), "best.pt")
# 3. INFERENCE on one new example
model.eval()
with torch.no_grad():
    x = tf(img).unsqueeze(0)                  # add batch dim: (1, 1, 28, 28)
    probs = torch.softmax(model(x), dim=1)    # logits -> probabilities
    pred = probs.argmax(1).item()

Checklist for inference: same preprocessing as training (same normalization), model.eval(), torch.no_grad(), add the batch dimension, apply softmax only if you want probabilities (the raw logits already give the argmax).

S5. Classical ML Gaps (Days 5–6)

Precision–Recall curve and PR-AUC

Sweep the decision threshold from 1 down to 0 and plot precision against recall at each threshold. PR-AUC (average precision) summarizes the curve. With rare positives (fraud, disease) it is more honest than ROC-AUC, because ROC-AUC counts the huge number of easy negatives. A random model scores PR-AUC ≈ the positive rate, not 0.5.

from sklearn.metrics import precision_recall_curve, average_precision_score, \
    roc_auc_score
prec, rec, thr = precision_recall_curve(y_test, proba)
print(roc_auc_score(y_test, proba), average_precision_score(y_test, proba),
    y_test.mean())

ROC-AUC 0.841 PR-AUC 0.348 base rate 0.036 (4,000 samples, 3% positives)

ROC-AUC looks good at 0.84, but PR-AUC of 0.35 against a 3.6% baseline shows the real difficulty of finding the rare class.

Entropy (why trees split the way they do)

Entropy measures impurity: H = −Σ pᵢ·log₂(pᵢ). A 50/50 node is maximally mixed; a pure node has zero. A tree picks the split that lowers entropy (or Gini) the most. The drop is called information gain.

H([0.5, 0.5]) = 1.000 H([0.9, 0.1]) = 0.469 H([1.0, 0.0]) = 0.000

ElasticNet

Loss = MSE + α·[ ρ·Σ|w| + (1−ρ)/2 · Σw² ]. It mixes Lasso (ρ, the L1 share, called l1_ratio in sklearn) and Ridge. Use it when features are correlated: pure Lasso tends to keep one of a correlated group arbitrarily, while ElasticNet keeps the group together and still zeroes useless features.

from sklearn.linear_model import ElasticNet
m = ElasticNet(alpha=0.1, l1_ratio=0.5).fit(X, y)     # scale features first!

True weights: 3 and 2, eight useless features
coef_ = [2.88, 1.88, 0, 0, 0, -0.15, 0, -0.03, -0.06, 0]   # noise shrunk toward 0

XGBoost, LightGBM, CatBoost

All three are fast, tuned implementations of gradient boosting (Day 6): trees added one at a time, each fitting the errors of the ones before. Typical starting point for tabular data.

Key knobs: nestimators, learningrate (lower = need more trees), max_depth (3–6 is typical), subsample, and early stopping on a validation set.

Curse of dimensionality

As the number of features d grows, data becomes sparse and distances lose meaning: the nearest and farthest neighbors end up almost equally far. This hurts k-NN, k-Means and anything distance-based, and lets models overfit with few samples. Remedies: fewer features, feature selection, PCA, regularization, more data.

# 500 random points, ratio of nearest to farthest distance from one point
d=2: 0.01    d=10: 0.30    d=100: 0.73    d=1000: 0.90
# closer to 1 = "everything is equally far"

S6. Deep Learning Gaps (Days 9–11)

Vanishing gradients

Backprop multiplies one derivative per layer. Sigmoid's derivative is at most 0.25, so after 10 layers the gradient is at most 0.25¹⁰ ≈ 1e-6. Early layers barely learn. Fixes: ReLU-family activations, good initialization

(He/Xavier), normalization layers, residual connections. The mirror problem, exploding gradients, is handled with gradient clipping (torch.nn.utils.clipgradnorm_).

Momentum, SGD, Adam and AdamW

• Momentum: keep a running velocity v = β·v + gradient, then w −= lr·v (β ≈ 0.9). It smooths zig-zags and speeds up along consistent directions.
• Adam: momentum plus a per-weight adaptive step size (divides by a running average of squared gradients). Forgiving default: lr = 1e-3.
• Weight decay: shrink weights each step (the L2 idea). In plain Adam the L2 penalty gets mixed into the adaptive scaling. AdamW applies the decay separately, which works better. Use torch.optim.AdamW(params, lr=1e-3, weight_decay=0.01).

Activation functions

RNN, LSTM, GRU (why Transformers replaced them)

An RNN reads a sequence one step at a time and carries a hidden state: h = tanh(W·[h , x ]). Training t t−1 t multiplies through every step, so gradients shrink (0.9⁵⁰ ≈ 0.005) or blow up (1.1⁵⁰ ≈ 117) over long sequences. LSTM adds a cell state with three gates (forget, input, output) that decide what to keep, write and reveal. GRU is a lighter version with two gates. Both remember longer, but still process steps one after another, so they train slowly and cannot look at all tokens at once. Attention (Day 13) fixes both problems.

S7. GenAI Gaps

Subword tokenization (BPE)

Whole-word vocabularies cannot cover new words; character vocabularies make sequences too long. Byte-Pair Encoding starts from characters and repeatedly merges the most frequent adjacent pair into a new token. Frequent pieces become tokens, rare words split into known pieces.

corpus: low×5  lower×2  newest×6  widest×3      (each word ends with a </w> marker)
merge 1: (e, s) → es      merge 2: (es, t) → est      merge 3: (est, </w>) → est</w>
merge 4: (l, o) → lo      merge 5: (lo, w) → low
result: low | low e r | n e w est</w> | w i d est</w>
# "est" is now one shared token

KV cache

When generating token t, attention needs the keys (K) and values (V) of all earlier tokens. They never change, so store them and compute only the new token's Q, K, V each step. Without the cache, generating 100 tokens reprocesses 1+2+…+100 = 5,050 token positions; with it, 100. The cost is memory: the cache grows with sequence length × layers × heads, which is why long contexts are memory-bound.

Fine-tuning and LoRA / PEFT

Full fine-tuning updates every weight and needs a full copy of the model plus optimizer state. LoRA freezes the pretrained weight W and learns a low-rank update: W' = W + B·A, where A is (r × d) and B is (d × r) with a small rank r (4–64). Only A and B train. B starts at zero so the model begins identical to the original.

one 4096×4096 layer: full = 16,777,216 weights LoRA r=8: 2·4096·8 = 65,536 (0.39%)

Adapters are small files that can be swapped per task. QLoRA also stores the frozen base in 4-bit precision to fit larger models on one GPU. When to fine-tune vs RAG: use RAG to give the model facts that change; fine-tune to change style, format or skills.

RLHF and DPO (how chat models are aligned)

After pretraining and supervised fine-tuning on example answers, models are tuned toward human preferences. RLHF: humans rank answers → train a reward model on the rankings → optimize the LLM (often with PPO) to score high, with a penalty for drifting too far from the reference model. DPO skips the reward model and RL loop: for each pair (chosen, rejected) it directly raises the chosen answer's likelihood relative to the rejected one, measured against a frozen reference.

loss = −log σ( β · [ (logπ(chosen) − logπ_ref(chosen)) − (logπ(rejected) −
    logπ_ref(rejected)) ] )
example, β=0.1: model prefers chosen more than reference does → loss 0.598;
    prefers rejected → 0.798

Better retrieval for RAG: hybrid search and re-ranking

• Hybrid search: combine keyword search (BM25, exact terms and IDs) with vector search (meaning). Merge the two ranked lists with Reciprocal Rank Fusion: score = Σ 1/(60 + rank).
• Re-ranking: retrieve a wide set (say 50) cheaply, then have a cross-encoder model read query and passage together and re-order the top results (keep 5). Slower but more accurate than embeddings alone.

rrf([[a, b, c], [c, a, d]]) → [a, c, b, d] # a and c appear in both lists, so they rise

Evaluating RAG: RAGAS

RAGAS is an open-source toolkit that scores a RAG system without hand-labelling everything. Common metrics: faithfulness (is the answer supported by the retrieved text?), answer relevancy (does it address the question?), context precision (are the retrieved chunks relevant and well-ranked?), context recall (was the needed information retrieved?). Several use an LLM as judge, so treat scores as signals and keep a small human-checked test set.

Prompt engineering, orchestration and deployment tools

from fastapi import FastAPI
from pydantic import BaseModel
app = FastAPI()
class Req(BaseModel): text: str
@app.post("/predict")
def predict(r: Req): return {"label": model_predict(r.text)}
# run: uvicorn main:app

S8. Day 4 - Gradient Descent (Line-by-Line)

The Day 4 code trains w*x + b to fit a line. Here is what each line does:

np.random.seed(42)
X = 2 * np.random.rand(100, 1)
y = 4 + 3*X + np.random.randn(100, 1)

True values: w=3, b=4. Gradient descent will try to recover them.

w = 0.0; b = 0.0; lr = 0.1; epochs = 200; n = len(X); losses = []

Start with guesses. lr (learning rate) controls step size.

for epoch in range(epochs):
    y_pred = w*X + b
    error = y_pred - y
    loss = np.mean(error**2)
    losses.append(loss)
    dw = (2/n) * np.sum(error*X)
    db = (2/n) * np.sum(error)
    w = w - lr*dw
    b = b - lr*db

Forward pass, compute MSE loss, compute gradients, update w and b opposite to gradient direction.

S9. Two-Vector Attention (Worked Example)

Two words: cat=[1,0,1], dog=[0,1,1]. Query=Key=Value=embeddings. d_k=3.

scores = Q @ K.T / sqrt(d_k)
Q @ K.T = [[1*1+0*0+1*1, 1*0+0*1+1*1], [0*1+1*0+1*1, 0*0+1*1+1*1]] = [[2,1],[1,2]]
scores ~ [[1.15,0.58],[0.58,1.15]]

attn = softmax(scores)
softmax([1.15,0.58]) ~ [0.64, 0.36]
output = attn @ V

Result: each position gets a weighted mix of all embeddings. Cat attends 64% to itself, 36% to dog.

S10. Tiny Attention in PyTorch

import torch, torch.nn as nn
class TinyAttention(nn.Module):
    def __init__(self, d_model=64, num_heads=2):
        super().__init__()
        self.d_model = d_model
        self.num_heads = num_heads
        self.d_k = d_model // num_heads
        self.W_q = nn.Linear(d_model, d_model)
        self.W_k = nn.Linear(d_model, d_model)
        self.W_v = nn.Linear(d_model, d_model)
        self.W_o = nn.Linear(d_model, d_model)
def forward(self, x):
    B, T, C = x.shape
    Q = self.W_q(x).view(B, T, self.num_heads, self.d_k).transpose(1, 2)
    K = self.W_k(x).view(B, T, self.num_heads, self.d_k).transpose(1, 2)
    V = self.W_v(x).view(B, T, self.num_heads, self.d_k).transpose(1, 2)
    scores = Q @ K.transpose(-2, -1) / (self.d_k ** 0.5)
    attn = torch.softmax(scores, dim=-1)
    out = attn @ V
    out = out.transpose(1, 2).contiguous().view(B, T, C)
    return self.W_o(out)

This is a complete multi-head attention module. Use it as a layer in a Transformer.

S11. Prompt Engineering

Structure: [Role] [Task] [Context] [Format] [Example or constraint]

You are an expert science writer.
Summarize this paper in 3-4 sentences for high school students.
Use everyday language and one analogy. Avoid jargon.
Example output: 'This shows X works like [analogy]. The key finding is Y. It
    matters because Z.'

Key tips: be specific (not just "summarize"), show examples, ask for format (JSON, list, markdown).