Project 3 — The Capstone (Day 14)

Goal. Prove you understand a model by rebuilding it from a blank file, with no tutorial open. The capstone is not about a high score. It is about being able to explain every line and to show that someone else can run it.

The chapter (Day 14, p. 287) explains the framework and the checklists, Part IV (p. 364) has the hints and README fill-ins, and Part IV (p. 379–380) covers GitHub submission and the self-rubric. This walkthrough adds what those pages do not spell out: a build order, a folder layout, runnable reference code and tests that say exactly when you are done.

*Every step below has four parts Do (what to do), Why (why it matters), Expect (what you should see) and If it fails* (what to try).

Step 1 — Pick ONE model

Option Model Level Choose it if…

A Logistic regression from scratch Easy you want rock-solid (NumPy) fundamentals

B 2-layer neural network from scratch Medium you want to show you (NumPy) understand backprop

C MNIST CNN in PyTorch Medium you want practical deep learning (recommended)

D Self-attention block in NumPy Hard you are aiming at GenAI

Do Choose one option and write it on the first line of your README.

Why A finished A beats an unfinished D. If you are short on time, pick C or A.

Expect One decision, made in under five minutes.

If it fails Unsure? Pick the easiest one you can finish.

Step 2 — Set the rules for yourself

1. Close all notes, videos and old notebooks. 2. Start a new empty file.
2. If you are stuck for more than 10 minutes, look up one function in the official docs. Never look up the whole solution. 4. Keep a “looked-up log”: a plain list of every function you searched. These are your weak spots, and they go into the README. 5. Start a timer. Aim for a trained model in about 2 hours.

Why The rules turn the project into a test of what you really know.

Expect A short log by the end. An empty log is suspicious, not impressive.

If it fails If you copied a whole solution, delete it and restart that block from the skeleton.

Step 3 — Write the skeleton before any real code

Type these comments into the blank file. Each comment becomes a block you fill in.

STARTER CODE

# 1. imports, seed, device
# 2. data: load, split train / val / test
# 3. model: define the architecture
# 4. loss and optimizer
# 5. training loop (+ validation each epoch)
# 6. evaluate on the test set once
# 7. plots: loss curves, sample predictions
# 8. save the model

Why A skeleton keeps you from writing the training loop before the data is loaded.

Expect Eight comment lines and nothing else.

Step 4 — Time-box the work

Time What you finish

0:00–0:15 Skeleton + imports + seed

0:15–0:35 Data loaded and one batch printed (check the shapes)

0:35–1:00 Model defined, and a random-input shape check passes

1:00–1:30 Training loop runs and the loss goes down

1:30–2:00 Test evaluation + plots

After Clean the code, write the README, record the Feynman test

Step 5 — Answer from memory (prompts, not code)

For Option C, answer each question before you type. If you cannot, that is your weak spot: log it and look up that one thing.

• Data: What is the shape of one batch? Which transform scales the pixels? Why is the test loader not shuffled?
• Model: What goes into the first Conv2d ? How many numbers come out of the last pooling layer, and what number goes into the first Linear ?
• Loss: Does CrossEntropyLoss want probabilities or raw scores? What shape and type are the labels?
• Loop: What are the four lines inside the training loop, and in what order?
• Eval: Which two things must you switch on or off ( model.eval() and torch.no_grad() ) and why?

Step 6 — If you chose A, B or D: reference code to check

yourself against

Write your own first. Use these only to compare the heart of your version. Each one is a complete program that runs on its own with NumPy and scikit-learn, and each ends with checks that print what a correct run looks like.

Option A — logistic regression (the gradient is the key line)

REFERENCE IMPLEMENTATION · option_a.py Checked: ran on stand-in data

import numpy as np
from sklearn.datasets import load_breast_cancer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

np.random.seed(42)
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, stratify=y, random_state=42)
scaler = StandardScaler().fit(X_train)            # learn the scaling on TRAIN rows only
X_train, X_test = scaler.transform(X_train), scaler.transform(X_test)

def sigmoid(z):
    return 1 / (1 + np.exp(-z))

def loss_fn(w, b, X, y):
    p = sigmoid(X @ w + b)
    return -np.mean(y * np.log(p + 1e-9) + (1 - y) * np.log(1 - p + 1e-9))

(continued)

def gradients(w, b, X, y):
    p = sigmoid(X @ w + b)                        # predictions between 0 and 1
    dw = X.T @ (p - y) / len(y)        # how the loss changes with each weight
    db = (p - y).mean()
    return dw, db

# Gradient check: compare with (L(w+e) - L(w-e)) / 2e for one weight.
w = np.random.randn(X_train.shape[1]) * 0.1
e = 1e-5
w_hi, w_lo = w.copy(), w.copy()
w_hi[0] += e
w_lo[0] -= e
numeric = (loss_fn(w_hi, 0.0, X_train, y_train)
           - loss_fn(w_lo, 0.0, X_train, y_train)) / (2 * e)
analytic = gradients(w, 0.0, X_train, y_train)[0][0]
print("gradient check difference:", abs(numeric - analytic))   # tiny (< 1e-6)

w, b, lr = np.zeros(X_train.shape[1]), 0.0, 0.1
for epoch in range(1000):
    dw, db = gradients(w, b, X_train, y_train)
    w -= lr * dw
    b -= lr * db
    if epoch % 200 == 0:
        print(epoch, round(loss_fn(w, b, X_train, y_train), 4))

mine = ((sigmoid(X_test @ w + b) > 0.5) == y_test).mean()
sk = LogisticRegression(max_iter=1000).fit(X_train, y_train).score(X_test, y_test)
print("my test accuracy:", round(mine, 3), "| scikit-learn:", round(sk, 3))

Expect: a gradient-check difference below 1e-6, a loss that falls from about 0.5, and a test accuracy within a few points of scikit-learn’s (above 95% on this dataset). *If it fails:* a large gradient difference means the formula for dw is wrong; a loss that grows means the learning rate is too high.

Option B — 2-layer network on make_moons (y has shape (n, 1))

REFERENCE IMPLEMENTATION · option_b.py Checked: ran on stand-in data

import numpy as np
from sklearn.datasets import make_moons
from sklearn.model_selection import train_test_split

np.random.seed(42)
X, y = make_moons(n_samples=1000, noise=0.2, random_state=42)
y = y.reshape(-1, 1)                                  # shape (n, 1)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y.ravel())

hidden, lr = 16, 0.5

(continued)

W1 = np.random.randn(2, hidden) * 0.5
b1 = np.zeros((1, hidden))
W2 = np.random.randn(hidden, 1) * 0.5
b2 = np.zeros((1, 1))

def forward(X):
    z1 = X @ W1 + b1
    a1 = np.maximum(0, z1)                            # ReLU
    z2 = a1 @ W2 + b2
    p = 1 / (1 + np.exp(-z2))                         # sigmoid
    return z1, a1, p

for epoch in range(2000):
    z1, a1, p = forward(X_train)
    loss = -np.mean(y_train * np.log(p + 1e-9) + (1 - y_train) * np.log(1 - p + 1e-9))
    n = len(y_train)                        # backward: chain rule, layer by layer
    dz2 = (p - y_train) / n
    dW2 = a1.T @ dz2
    db2 = dz2.sum(0, keepdims=True)
    da1 = dz2 @ W2.T
    dz1 = da1 * (z1 > 0)
    dW1 = X_train.T @ dz1
    db1 = dz1.sum(0, keepdims=True)
    W1 -= lr * dW1
    b1 -= lr * db1
    W2 -= lr * dW2
    b2 -= lr * db2
    if epoch % 400 == 0:
        print(epoch, round(loss, 4))

acc = ((forward(X_test)[2] > 0.5) == y_test).mean()
print("test accuracy:", round(acc, 3))

Expect: the loss falls steadily and the test accuracy ends high (above 90% here). *If it fails:* shapes. Print W1.shape , a1.shape , dz2.shape and check each matrix product. db must keep its shape: use keepdims=True .

Option D — self-attention for one sequence x of shape (T, d)

REFERENCE IMPLEMENTATION · option_d.py Checked: ran on stand-in data

import numpy as np

np.random.seed(42)

def softmax(s):
    e = np.exp(s - s.max(axis=-1, keepdims=True))     # subtract the max for stability
    return e / e.sum(axis=-1, keepdims=True)

(continued)

def self_attention(x, Wq, Wk, Wv, causal=False):
    Q, K, V = x @ Wq, x @ Wk, x @ Wv
    scores = Q @ K.T / np.sqrt(K.shape[-1])
    if causal:                                        # hide future positions
        scores = np.where(np.triu(np.ones_like(scores), 1) == 1, -1e9, scores)
    weights = softmax(scores)
    return weights @ V, weights

T, d = 5, 8
x = np.random.randn(T, d)
Wq, Wk, Wv = (np.random.randn(d, d) * 0.3 for _ in range(3))

out, w = self_attention(x, Wq, Wk, Wv)
print("output shape:", out.shape, "| weights shape:", w.shape)
assert out.shape == (T, d) and w.shape == (T, T)
assert np.allclose(w.sum(axis=-1), 1.0)               # every row sums to 1

out_c, w_c = self_attention(x, Wq, Wk, Wv, causal=True)
assert np.allclose(w_c.sum(axis=-1), 1.0)
assert np.allclose(np.triu(w_c, 1), 0.0)              # nothing above the diagonal
assert np.allclose(w_c[0], [1, 0, 0, 0, 0])           # first position only sees itself
print("all attention checks passed")

Expect: every row of the weights sums to 1, and with causal=True everything above the diagonal is 0. If it fails: you probably forgot to subtract the row maximum in the softmax or applied it along the wrong axis.

Option C is the MNIST CNN from Project 2: rebuild it from the blank file and compare with Step 7 of that project.

Step 7 — Write the explanation (6 parts, 3–6 sentences each)

Use these sentence starters so you never face a blank page:

1. Problem: “This model teaches a computer to __ because .” 2. **Input and output:** “The input has shape and means . The output has shape and the answer is the .” 3. **How it works:** “Data first goes through , which changes the shape to . Then .” 4. **Loss:** “I use because . A lower value means .” 5. **Training:** “The optimizer is with learning rate . Each step does: .” 6. **Evaluation:** “The test accuracy was . It failed on . Next I would try __.”

Step 8 — Organise the project

Use this layout. One tidy notebook is also acceptable, but the folder, requirements.txt and README are not optional.

FILE

capstone/
├── README.md            # see Step 10 for the structure
├── requirements.txt     # pip freeze output, or the packages you import
├── .gitignore           # data/, *.pt, env/, __pycache__/, .env
├── data.py              # load + split (Option C: the MNIST loaders)
├── model.py             # the architecture only
├── train.py             # training loop; prints loss per epoch; saves outputs/
├── evaluate.py          # test metrics, confusion matrix, error analysis
├── tests/test_checks.py # the automatic checks from the table below
└── outputs/             # plots and saved weights (not committed)

• Functions instead of copied blocks; clear names ( train_loader , not dl1 ).
• Seeds set, no hard-coded personal paths, unused imports and debug prints removed.
• Restart and Run All works (or python train.py from a fresh terminal).
• Datasets: built-in or downloaded by the code (scikit-learn datasets, MNIST). Say which in the README.

Run commands: create the environment (Step 0.2), then run these from the capstone/ folder.

COMMANDS

pip install -r requirements.txt
python train.py          # trains and saves outputs/
python evaluate.py       # test set, once
python -m pytest -q      # automatic checks (pip install pytest)

Step 9 — The Feynman test

Record yourself on your phone explaining the model for 5 minutes to someone with no ML background. Every place you hesitate is a place to re-read the book. Write those places in the README under “What I still find hard”.

Step 10 — README structure

FILE · README.md

# <Model name>: rebuilt from scratch
## Problem and data       - what, why, source, shapes, split sizes, class balance
## Model and loss         - architecture, shapes, loss function and why
## How to run             - the exact commands from Step 8
## Results                - baseline vs yours, metric, confusion matrix or curves
## Error analysis         - 5+ mistakes, with a guess about why each happened
## Checks                 - which acceptance checks pass (copy the printed lines)
## Looked-up log         - every function you had to look up
## What I still find hard - from the Feynman test
## Six-part explanation   - from Step 7

Minimum submission and optional improvements

Minimum viable submission (must have) Optional improvements (pick any)

One model that trains from a blank file, a A second model or a hyperparameter baseline to compare with, test metric reported comparison; a small ablation (for example once, all acceptance checks below passing, the without dropout); a blog or LinkedIn post README from Step 10, the six-part explanation, (template on p. 367); filter or attention-weight the looked-up log. visualisation; a recorded Feynman video.

Acceptance criteria (measurable)

You are done when every row passes. Where a check is a number, the number is the pass line.

Area How to test Pass line

Data quality Print shapes, class balance and All printed in the README; no np.isfinite(X).all() . Confirm the NaN or infinity. scaler or any other learned step was fitted on training rows only.

Baseline Compare with a trivial baseline Yours beats the majority-class (most-frequent class, or scikit-learn’s own baseline. A: within 3 model for A). percentage points of scikit-learn.

Correctness check A: gradient check. B: loss at the last A: difference < 1e-6. B: loss epoch vs the first. C: random-input shape falls by at least 50%. C: output check. D: attention-row and causal-mask is (B, 10). D: rows sum to 1 checks. within 1e-6 and the upper triangle is 0.

Evaluation Evaluate on the test set once, after every One test number; metric and choice is made. confusion matrix or curves in the README.

Area How to test Pass line

Error analysis List at least 5 wrong predictions (input, 5 examples + 3 sentences on true, predicted). For D, show one why. attention matrix instead.

Reproducibility In a fresh environment run pip install Runs with no edits; the final -r requirements.txt && python metric matches to 3 decimals train.py . Run it twice with the same on CPU. seed.

Documentation Read the README as a stranger. Every heading from Step 10 is filled; the log and the explanation are in your own words.

You are done when…

n A model you wrote from a blank file trains and gets a sensible score. n Every row of the acceptance table passes. n The six-part explanation is written in your own words. n The looked-up log is in the README. n Someone else could run it from the README alone.