Day 1 · Part 2: Math Essentials

Part 2 — Math Essentials

Good news: you need intuition and a few formulas, not proofs. Every idea below is followed by a tiny number example you can check by hand.

1. Arithmetic, percentages, averages

Definition A percentage is a part of a whole written out of 100. The mean (average) is the sum of the values divided by how many there are.

How it works, step by step

1. Percentage: divide the part by the whole, then multiply by 100. 45 out of 50 → 45 ÷ 50 = 0.90 → 90%.
2. Mean: add all the values together.
3. Count the values.
4. Divide the sum by the count. [2, 4, 6, 8] → 20 ÷ 4 = 5.

Percentage = part ÷ whole × 100. Average (mean) = sum ÷ count.

Example: a model gets 45 of 50 predictions right, so accuracy = 45 ÷ 50 = 0.90 = 90%. The mean of [2, 4, 6, 8] = 20 ÷ 4 = 5.

Why it matters in ML: accuracy, precision and recall are all percentages, and “average loss” is just the mean of many error values.

Worked Problem · Accuracy And Average

Problem. A model classifies 45 of 50 images correctly. Separately, four test scores are 70, 85, 90 and 95. Find the accuracy and the mean score.

1. Accuracy = correct ÷ total = 45 ÷ 50 = 0.90.
2. Percentage = 0.90 × 100 = 90%.
3. Add the scores: 70 + 85 + 90 + 95 = 340.
4. Divide by how many there are: 340 ÷ 4 = 85.

Answer. Accuracy 90%, mean score 85.

Your Turn · Percent In Both Directions

Problem. A model gets 72 of 90 right. What is its accuracy? A second model has accuracy 0.85 on 200 samples. How many did it get right?

Try it on paper or in a notebook first. Then check the solution.

Solution — Step By Step

1. First model: 72 ÷ 90 = 0.8, which is 80%.
2. Second model: accuracy = correct ÷ total, so correct = accuracy × total.
3. 0.85 × 200 = 170.

Answer. 80%, and 170 correct predictions.

2. Basic algebra and functions

Definition A function is a rule that turns an input number into an output number. In y = w·x + b, x is the input, y is the output, w (weight or slope) says how steeply y changes, and b (bias or intercept) is the value of y when x = 0.

How it works, step by step

1. Write the rule with the numbers filled in: y = 3·x + 10.
2. Pick an input, for example x = 5.
3. Substitute it: y = 3 × 5 + 10.
4. Multiply first, then add: 15 + 10 = 25.
5. Changing w changes how fast y grows. Changing b slides the whole line up or down.

A function is a machine: put a number in, get a number out. The most important one in ML is the straight line:

y = w·x + b (w = slope / weight, b = intercept / bias)

Example: house price = 3 × size + 10 (in $1000s). A 5-unit house gives 3×5 + 10 = 25. Change w and the line gets steeper; change b and it shifts up or down.

Why it matters in ML: a neuron computes exactly this, w·x + b, and “training” means finding good values of w and b.

Worked Problem · Predict With A Line

Problem. House price = 3 × size + 10. Find the price for sizes 5 and 8. Then change the intercept to 20 and find both prices again.

1. Here w = 3 and b = 10.
2. Size 5: 3 × 5 + 10 = 15 + 10 = 25.
3. Size 8: 3 × 8 + 10 = 24 + 10 = 34.
4. Change b to 20. Size 5: 15 + 20 = 35. Size 8: 24 + 20 = 44.

Answer. 25 and 34 with b = 10; 35 and 44 with b = 20. Changing b shifted every price up by 10.

Your Turn · A Line That Goes Down

Problem. For y = −2·x + 5, find y at x = 0, 1 and 4. Then find the x where y = 0.

Try it on paper or in a notebook first. Then check the solution.

Solution — Step By Step

1. x = 0: −2×0 + 5 = 5.
2. x = 1: −2×1 + 5 = 3.
3. x = 4: −2×4 + 5 = −3.
4. Set y = 0: 0 = −2x + 5. Move terms: 2x = 5, so x = 2.5.

Answer. y = 5, 3, −3, and the line crosses zero at x = 2.5. A negative w means y falls as x grows.

3. Vectors and matrices

Definition A scalar is one number. A vector is a list of numbers. A matrix is a grid of numbers with rows and columns. The shape of a matrix is written (rows, columns).

How it works, step by step

1. Count the rows (going down) and the columns (going across). A grid with 2 rows and 3 columns has shape (2, 3).
2. Read the shape as meaning: rows = examples, columns = features.
3. Find one value with [row][column], counting from 0. In [[1,2,3],[4,5,6]], [1][2] is 6.
4. The transpose swaps rows and columns: shape (2, 3) becomes (3, 2).

Scalar = one number (5). Vector = a list of numbers ([1, 2, 3], shape 3). Matrix = a grid (shape rows × columns).

A = [[1, 2, 3],
     [4, 5, 6]]     # a 2x3 matrix: 2 rows, 3 columns
# A[0][2] = 3 (row 0, column 2)

Convention you will see everywhere: rows = examples, columns = features. A dataset of 100 people with 4 measurements each is a 100 × 4 matrix.

Why it matters in ML: images, text and tables all become arrays of numbers. Shapes tell you what the array means.

Worked Problem · Read A Small Dataset

Problem. Three students have two features each (hours studied, practice score): [[60, 7], [75, 8], [90, 9]]. What is the shape? What are row 1, column 0, and the value at row 2, column 1?

1. Count the rows: 3 students. Count the columns: 2 features. Shape = 3 × 2.
2. Rows are examples, so the shape means “3 examples with 2 features each”.
3. Python counts from 0. Row 1 is the second row: [75, 8].
4. Column 0 is the first entry of every row: [60, 75, 90].
5. Row 2, column 1: the third row, second value = 9.

Answer. Shape 3 × 2; row 1 = [75, 8]; column 0 = [60, 75, 90]; value = 9.

Your Turn · Shapes Of A Bigger Dataset

Problem. A dataset has 100 people with 4 measurements each. What is its shape? What is the shape if you keep only the first measurement? What is the shape after transposing the full dataset (swapping rows and columns)?

Try it on paper or in a notebook first. Then check the solution.

Solution — Step By Step

1. Rows = people = 100, columns = measurements = 4, so the shape is 100 × 4.
2. Keeping one measurement leaves 100 rows and 1 column: 100 × 1.
3. Transposing swaps the two numbers: 4 × 100.

Answer. 100 × 4, 100 × 1, and 4 × 100.

4. Dot product and matrix multiplication

Definition The dot product of two vectors multiplies matching entries and adds the results to give one number. Matrix multiplication takes the dot product of every row of the first matrix with every column of the second.

How it works, step by step

1. Dot product: line up the two vectors entry by entry.
2. Multiply each pair.
3. Add all the products. [1, 2, 3] · [4, 5, 6] = 4 + 10 + 18 = 32.
4. Matrix product: first check the shapes. (a × b)·(b × c) works because the inner numbers match, and the result is (a × c).
5. For each row of the left matrix and each column of the right matrix, take a dot product and write it in that row and column of the result.

Dot product: multiply matching entries, then add. [1, 2, 3] · [4, 5, 6] = 1×4 + 2×5 + 3×6 = 4 + 10 + 18 = 32. A large value means the two vectors point the same way.

Matrix multiplication = a dot product of each row of A with each column of B. Example, A (2×3) times B (3×2):

A = [[1, 2, 3],     B = [[1, 0],
     [4, 5, 6]]          [0, 1],
                         [1, 1]]
row 1 of A . col 1 of B = 1*1 + 2*0 + 3*1 = 4
row 1 of A . col 2 of B = 1*0 + 2*1 + 3*1 = 5
row 2 of A . col 1 of B = 4*1 + 5*0 + 6*1 = 10
row 2 of A . col 2 of B = 4*0 + 5*1 + 6*1 = 11
result = [[ 4, 5],
          [10, 11]]                      shape 2x2

Shape rule: (a × b) · (b × c) → (a × c). The two inner numbers must match; they disappear. (2×3)·(3×2) → 2×2 ✓. (2×3)·(2×3) → ✗ error.

Why it matters in ML: a whole neural-network layer is one matrix multiplication, and attention (Day 13) is built from dot products. Most PyTorch shape errors are this rule being broken.

Worked Problem · Multiply Two 2×2 Matrices

Problem. Multiply A = [[2, 1], [0, 3]] by B = [[1, 4], [2, 0]]. Check the answer with NumPy.

1. Shapes first: (2×2)·(2×2). The inner numbers match, so the result is 2×2.
2. Row 1 · column 1: 2×1 + 1×2 = 4.
3. Row 1 · column 2: 2×4 + 1×0 = 8.
4. Row 2 · column 1: 0×1 + 3×2 = 6.
5. Row 2 · column 2: 0×4 + 3×0 = 0.
6. Check with NumPy: @ is the matrix-multiplication operator.

import numpy as np
A = np.array([[2, 1], [0, 3]])
B = np.array([[1, 4], [2, 0]])
print(A @ B)

[[4 8]
 [6 0]]

Answer. The result is [[4, 8], [6, 0]], matching the hand calculation.

Your Turn · Is This Multiplication Allowed?

Problem. (a) Find the dot product of [1, −1, 2] and [3, 3, 1]. (b) Is a (4×3) matrix times a (3×5) matrix allowed? What is the result shape? (c) Is (4×3)·(4×3) allowed? How can you fix it?

Try it on paper or in a notebook first. Then check the solution.

Solution — Step By Step

1. (a) Multiply matching entries: 1×3 = 3, −1×3 = −3, 2×1 = 2. Add: 3 − 3 + 2 = 2.
2. (b) Write the shapes: (4× 3)·(3 ×5). The inner numbers match (3 and 3), so it is allowed. The outer numbers give the result: 4×5.
3. (c) (4× 3)·(4 ×3). The inner numbers are 3 and 4, so it is not allowed.
4. Fix: transpose the second matrix. (4×3) becomes (3×4), and (4× 3)·(3 ×4) → 4×4.

Answer. (a) 2; (b) allowed, 4×5; (c) not allowed, but transposing the second matrix gives 4×4.

5. Mean and variance

Definition The mean is the center of the data. The variance is the average squared distance from the mean, so it measures spread. The standard deviation is the square root of the variance, in the same units as the data.

How it works, step by step

1. Find the mean: sum ÷ count.
2. Subtract the mean from every value to get the gaps.
3. Square each gap, so negatives do not cancel out.
4. Average the squared gaps: this is the variance.
5. Take the square root: this is the standard deviation.

Mean = the center. Variance = the average squared distance from the mean (how spread out). Std dev = √variance, in the original units.

data = [2, 4, 4, 4, 5, 5, 7, 9]
mean = (2+4+4+4+5+5+7+9) / 8 = 40 / 8 = 5
squared gaps from mean: 9, 1, 1, 1, 0, 0, 4, 16         -> sum = 32
variance = 32 / 8 = 4
std dev = sqrt(4) = 2

Why it matters in ML: feature scaling (x - mean) / std, normalization layers, and understanding whether a metric is stable all use these two numbers.

Worked Problem · Variance By Hand

Problem. Find the mean, variance and standard deviation of [3, 5, 7, 9].

1. Mean = (3 + 5 + 7 + 9) ÷ 4 = 24 ÷ 4 = 6.
2. Gaps from the mean: −3, −1, 1, 3.
3. Square each gap: 9, 1, 1, 9. Their sum is 20.
4. Variance = 20 ÷ 4 = 5.
5. Standard deviation = √5 ≈ 2.236. Check it with NumPy.

import numpy as np
data = [3, 5, 7, 9]
print(np.mean(data))
print(np.var(data))
print(round(np.std(data), 3))

6.0
5.0
2.236

Answer. Mean 6, variance 5, standard deviation ≈ 2.236.

Your Turn · Zero Spread, And A Quick Check

Problem. What are the variance and standard deviation of [10, 10, 10]? Then find them for [1, 2, 3, 4, 5].

Try it on paper or in a notebook first. Then check the solution.

Solution — Step By Step

1. [10, 10, 10]: mean = 10. Every gap is 0, so every squared gap is 0. Variance = 0 and std = 0.
2. [1, 2, 3, 4, 5]: mean = 15 ÷ 5 = 3.
3. Squared gaps: 4, 1, 0, 1, 4. Sum = 10. Variance = 10 ÷ 5 = 2.
4. Std = √2 ≈ 1.414.

Answer. First list: variance 0, std 0 (all values identical). Second list: variance 2, std ≈ 1.414.

6. Basic probability

*Definition Probability is a number from 0 (impossible) to 1 (certain) that says how likely an event is. An outcome is one possible result, and an event is a set of outcomes you care about. Two events are independent* when one does not change the other.

How it works, step by step

1. List all equally likely outcomes. A die has 6.
2. Count the outcomes that match your event. “Even” has 3 (2, 4, 6).
3. Probability = matching ÷ total = 3 ÷ 6 = 0.5.
4. “Not the event” = 1 − probability.
5. Both of two independent events: multiply their probabilities. Two heads = 0.5 × 0.5 = 0.25.

A probability is a number from 0 (impossible) to 1 (certain). All outcomes together add to 1.

Fair die: P(4) = 1/6. P(even) = 3/6 = 0.5. Two independent coin flips both heads: 0.5 × 0.5 = 0.25.

Preview — softmax turns any scores into probabilities. Scores [2, 1, 0]: e² = 7.39, e¹ = 2.72, e⁰ = 1.00, total 11.11, so probabilities ≈ [0.665, 0.245, 0.090] (they sum to 1).

Why it matters in ML: classifiers output probabilities, and an LLM picks the next word by producing a probability for every word it knows.

Worked Problem · Dice, Coins And Softmax

Problem. (a) What is the probability of rolling more than 4 on a fair die? (b) What is the probability of at least one head in two coin flips? (c) Turn the scores [3, 1, 0] into probabilities with softmax.

1. (a) The outcomes above 4 are 5 and 6, so 2 out of 6: 2 ÷ 6 ≈ 0.333.
2. (b) “At least one head” is easier as 1 minus “no heads”. No heads means two tails: 0.5 × 0.5 = 0.25. So 1 − 0.25 = 0.75.
3. (c) Softmax step 1: raise e to each score. Step 2: add them up. Step 3: divide each by the total.
4. Let NumPy do the arithmetic and print each step.

import numpy as np
scores = np.array([3.0, 1.0, 0.0])
e = np.exp(scores)
probs = e / e.sum()
print(np.round(e, 2))
print(np.round(probs, 3))
print(probs.sum())

[20.09  2.72  1.  ]
[0.844 0.114 0.042]
1.0

Answer. (a) ≈ 0.333; (b) 0.75; (c) [0.844, 0.114, 0.042], which sum to 1. The biggest score gets the biggest probability.

Your Turn · Drawing From A Bag

Problem. A bag has 3 red and 2 blue marbles. You draw one without looking. Find P(red), P(blue), and P(not red). What do P(red) and P(blue) add up to?

Try it on paper or in a notebook first. Then check the solution.

Solution — Step By Step

1. Total marbles = 3 + 2 = 5.
2. P(red) = 3 ÷ 5 = 0.6.
3. P(blue) = 2 ÷ 5 = 0.4.
4. “Not red” means blue, so P(not red) = 0.4. Also 1 − 0.6 = 0.4.
5. P(red) + P(blue) = 0.6 + 0.4 = 1.

Answer. 0.6, 0.4, 0.4. The two outcomes add to 1, as all outcomes together always do.

7. Derivative and gradient intuition

Definition A derivative is the slope of a function at one point: how fast the output changes when the input is nudged. A gradient is the list of slopes, one for each input. Gradient descent repeatedly steps against the slope to reach a low point. The learning rate is the size of each step.

How it works, step by step

1. Write the function and find its slope formula with the three rules (x² → 2x, a·x → a, constant → 0).
2. Put your current value into the slope formula to get the slope there.
3. Step against the slope: new value = old value − learning rate × slope.
4. Repeat. The slope gets smaller as you approach the bottom, so the steps shrink.
5. Stop when the slope is almost 0 or after a fixed number of steps.

A derivative is the slope at one point: “if I nudge the input a tiny bit, how much does the output change?” Positive slope = output rises; negative = it falls; zero = flat (a bottom or top).

Three rules are enough for now: d/dx of x² is 2x; d/dx of a constant is 0; d/dx of a·x is a.

A gradient is the list of slopes, one per input. For f(x, y) = x² + y² at the point (3, 4), the gradient is [2·3, 2·4] = [6, 8]. It points uphill, so we walk the opposite way to go down.

Gradient descent in 5 lines. Minimize f(w) = w². The slope is 2w. Start at w = 3, take small steps of size 0.1 against the slope:

w = 3.0
for step in range(4):
    print(round(w, 4))
    w = w - 0.1 * (2 * w)     # new w = old w - learning_rate * slope
# prints: 3.0, 2.4, 1.92, 1.536   -> w is sliding down toward 0 (the minimum)

Why it matters in ML: training a neural network is exactly this loop, just with millions of weights instead of one. That is all Day 4 and Days 9–10 are about.

You do NOT need yet: the chain rule, integrals, eigenvalues, or formal probability theory. Backpropagation on Day 10 is explained step by step when you get there.

Worked Problem · Find The Slope, Then Walk Downhill

Problem. For f(w) = w² + 4w, find the slope at w = 1. Then take three gradient-descent steps from w = 1 with step size 0.1.

1. Use the three rules: d/dw of w² is 2w, and d/dw of 4w is 4. So the slope is 2w + 4.
2. At w = 1 the slope is 2×1 + 4 = 6. It is positive, so f rises as w grows. To go down, move w to the left (smaller).
3. Step 1: w = 1 − 0.1 × 6 = 0.4.
4. Step 2: slope = 2×0.4 + 4 = 4.8, so w = 0.4 − 0.48 = −0.08.
5. Step 3: slope = 2×(−0.08) + 4 = 3.84, so w = −0.08 − 0.384 = −0.464.
6. Check with code. The minimum is where the slope is 0: 2w + 4 = 0, so w = −2.

w = 1.0
for step in range(3):
    slope = 2 * w + 4
    w = w - 0.1 * slope
    print(round(w, 4))

0.4
-0.08
-0.464

Answer. Slope at w = 1 is 6. The steps give 0.4, −0.08, −0.464, sliding toward the minimum at w = −2.

Your Turn · A Gradient With Two Inputs

Problem. For f(x, y) = x² + 3y, find the gradient at the point (2, 5). Take one step of size 0.1 downhill. Did f go down?

Try it on paper or in a notebook first. Then check the solution.

Solution — Step By Step

1. Slope in x: d/dx of x² is 2x = 2×2 = 4. Slope in y: d/dy of 3y is 3.
2. Gradient = [4, 3]. It points uphill.
3. Go downhill: new point = old point − 0.1 × gradient = (2 − 0.4, 5 − 0.3) = (1.6, 4.7).
4. Compare f. Before: 2² + 3×5 = 4 + 15 = 19. After: 1.6² + 3×4.7 = 2.56 + 14.1 = 16.66.

Answer. Gradient [4, 3]; new point (1.6, 4.7); f fell from 19 to 16.66.