Day 8 · Mini Project 1: Telco Customer Churn

Suggested pace: 1–1.5 hrs learn · 2–3 hrs code · rest to debug and document

What you should know first ☐ Day 3: Pandas cleaning and encoding ☐ Days 5–6: cross-validation and metrics ☐ Day 7: why scaling matters If one of these feels shaky, spend 15 minutes revisiting it before starting. That is normal and saves time later.

Goal of today Build your first complete machine-learning project end to end on a real dataset: load, clean, split, preprocess, train, cross-validate, evaluate once on the test set, and write a README.

You will be able to:

• Build a preprocessing pipeline that avoids data leakage
• Compare models with stratified cross-validation
• Evaluate on the test set once
• Explain which features matter
• Write a README for your portfolio

PART F — ★ MINI PROJECT 1: TELCO CUSTOMER CHURN

Problem: A telecom company wants to know which customers will leave (churn) so it can offer them deals. Binary classification: Churn = Yes/No.

Dataset: Kaggle “Telco Customer Churn” (WA_Fn-UseC_-Telco-Customer-Churn.csv , 7043 rows, ~21 columns, ~27% churn).

Pipeline:

Data → Preprocessing → Model → Cross-Validation → Metrics → Feature Importance

Step 1 — Load and explore

import pandas as pd, numpy as np
df = pd.read_csv("telco.csv")
df.shape ; df.info() ; df["Churn"].value_counts(normalize = True)

Step 2 — Clean (a known gotcha!)

TotalCharges is read as text because some values are blank spaces.

df["TotalCharges"] = pd.to_numeric(df["TotalCharges"], errors = "coerce")
# blanks → NaN
# blanks stay NaN for now: Step 4 fills them with the TRAIN median (no leakage)
df = df.drop(columns = ["customerID"])
# unique id, useless
df["Churn"] = df["Churn"].map({"Yes": 1, "No": 0})

Step 3 — Split features/target

from sklearn.model_selection import train_test_split
X = df.drop(columns = ["Churn"]) ; y = df["Churn"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, stratify
    = y, random_state = 42)
num_cols = ["tenure", "MonthlyCharges", "TotalCharges"]
cat_cols = [c for c in X.columns if c not in num_cols]
# includes SeniorCitizen (0/1) - fine either way

Step 4 — Preprocessing with a Pipeline (prevents leakage)

from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.pipeline import Pipeline, make_pipeline
from sklearn.preprocessing import StandardScaler, OneHotEncoder
pre = ColumnTransformer([
    ("num", make_pipeline(SimpleImputer(strategy="median"), StandardScaler()),
        num_cols),
    ("cat", OneHotEncoder(handle_unknown = "ignore"), cat_cols),
    # one-hot categories
])

• ColumnTransformer = apply different processing to different columns
• Pipeline = chain steps so fit learns everything from train only

Step 5 — Models

from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
models = {
  "LogReg": LogisticRegression(max_iter = 1000, class_weight = "balanced"),
  "RandomForest": RandomForestClassifier(n_estimators = 300, class_weight =
      "balanced", random_state = 42),
  "GradBoost": GradientBoostingClassifier(random_state = 42),
}
pipes = {n: Pipeline([("pre", pre), ("clf", m)]) for n, m in models.items()}

Step 6 — Cross-validation (5-fold, stratified)

from sklearn.model_selection import StratifiedKFold, cross_validate
cv = StratifiedKFold(n_splits = 5, shuffle = True, random_state = 42)
for name, p in pipes.items():
    r = cross_validate(p, X_train, y_train, cv = cv,
   scoring = ["accuracy","precision","recall","f1","roc_auc"])
    print(name, {k: round(v.mean(),3) for k, v in r.items() if
        k.startswith("test_")})

Step 7 — Final evaluation on the TEST set (once!)

from sklearn.metrics import classification_report, confusion_matrix, roc_auc_score
best = pipes["LogReg"].fit(X_train, y_train)      # choose based on CV results
pred = best.predict(X_test) ; prob = best.predict_proba(X_test)[:, 1]
print(classification_report(y_test, pred)) ; print(confusion_matrix(y_test, pred))
print("ROC-AUC", roc_auc_score(y_test, prob))

Realistic results: ROC-AUC ≈ 0.84; recall on churn ≈ 0.75 with balanced weights.

Step 8 — Feature importance

import matplotlib.pyplot as plt
rf = pipes["RandomForest"].fit(X_train, y_train)
names = rf.named_steps["pre"].get_feature_names_out()
imp = pd.Series(rf.named_steps["clf"].feature_importances_, index =
    names).sort_values(ascending = False)
imp.head(10).sort_values().plot.barh() ; plt.title("Top 10 features") ; plt.show()

For Logistic Regression, look at coef_ (sign = direction of effect). Typical insights: short tenure, month-to-month contract, fiber optic internet, high *monthly charges*, no tech support → higher

churn.

Step 9 — README.md (this is what recruiters read)

# Telco Customer Churn Prediction
## Problem  (1-2 lines, business value)
## Dataset  (source, size, class balance)
## Approach (pipeline diagram: preprocessing → models → CV)
## Results  (table of models: precision/recall/F1/AUC)
## Key Insights (top churn drivers + business recommendation)
## How to run (pip install -r requirements.txt; jupyter notebook)

Common Mistakes

Practice Questions

1. Why must features be scaled for k-NN?
2. What happens with k=1 in k-NN? With k = total samples?
3. Describe the k-Means loop in your own words.
4. PCA with explained_variance_ratio_ = [0.7, 0.2, 0.1] — how many components keep ≥ 90%? → 2
5. In one sentence: what is an embedding?
6. Why is cosine similarity preferred for embeddings?
7. Why use a Pipeline in the churn project?

Quick Summary

• Similar = close; distance needs scaled features
• k-NN = vote of nearest neighbors; k-Means = grouping by centroids; PCA = compress features
• Embeddings = meaning as vectors (the base of RAG and vector DBs)
• Full project flow: clean → pipeline → CV → test → importance → README

Next: Days 9–10 — Neural Networks From Scratch

Worked Problem · Fix A Numeric Column Stored As Text

Problem. In a churn dataset, TotalCharges arrives as text: ["29.85", "1889.5", " ", "108.15"], where a blank means unknown. Convert it to numbers and fill the blank with the median.

1. Blank strings cannot become numbers. pd.to_numeric(..., errors="coerce") turns them into NaN instead of crashing.
2. The numbers that exist are 29.85, 1889.5 and 108.15. Their median is the middle one: 108.15.
3. Fill the NaN with that median.
4. Doing this inside a pipeline (fit on the training rows only) avoids data leakage.

import pandas as pd
s = pd.Series(["29.85", "1889.5", " ", "108.15"])
n = pd.to_numeric(s, errors="coerce")
print(n.tolist())
print(n.fillna(n.median()).tolist())

[29.85, 1889.5, nan, 108.15]
[29.85, 1889.5, 108.15, 108.15]

Answer. After converting: [29.85, 1889.5, nan, 108.15]. After filling: *[29.85, 1889.5, 108.15, 108.15]*.

Your Turn · Why Fit The Scaler On Training Data Only

Problem. Training values are [1, 2, 3] and one test value is 10. Standardize the test value using (a) a scaler fit on the training data and (b) a scaler fit on all four values. Why is (b) a mistake?

Try it on paper or in a notebook first. Then check the solution.

Solution — Step By Step

1. (a) Training mean = 2 and std ≈ 0.816. The test value becomes (10 − 2) ÷ 0.816 ≈ 9.80.
2. (b) All four values: mean = 4 and std ≈ 3.54. The test value becomes (10 − 4) ÷ 3.54 ≈ 1.70.
3. In (b) the test value changed the scaler, so information from the test set leaked into preprocessing. Your test score would look better than reality.
4. Rule: fit on training data, then only transform validation and test data.

import numpy as np
tr = np.array([1, 2, 3.]); te = 10.
print(round((te - tr.mean()) / tr.std(), 2))
al = np.append(tr, te)
print(round((te - al.mean()) / al.std(), 2))

9.8
1.7

Answer. (a) 9.8, (b) 1.7. (b) leaks test information, so always fit preprocessing on the training set only.

Worked Problem · Stratified Folds

Problem. A churn column has 5 churners and 15 non-churners. Use 5-fold stratified cross-validation. How many churners and non-churners are in each test fold?

1. Stratified folds keep the class ratio. The ratio is 5 : 15 = 1 : 3.
2. Each fold has 20 ÷ 5 = 4 test rows, so 1 churner and 3 non-churners.
3. Without stratification a fold could contain no churner at all and give a meaningless score.

import numpy as np
from sklearn.model_selection import StratifiedKFold
y = np.array([1] * 5 + [0] * 15)
for tr, te in StratifiedKFold(5, shuffle=True, random_state=0).split(np.zeros((20,
    1)), y):
    print(np.bincount(y[te]).tolist())

[3, 1]
[3, 1]
[3, 1]
[3, 1]
[3, 1]

Answer. Every fold prints [3, 1]: 3 non-churners and 1 churner.

⚡ Quick Recap — Days 7–8

(Familiarity level — know the idea, not every detail)

5.1 Distance & Similarity

5.2 k-NN (k-Nearest Neighbors)

“You are like your closest neighbors.” To predict a new point: find the k nearest training points and vote (classification) or average (regression).

• No real training — stores the data (“lazy learner”).
• Small k → overfit/noisy; large k → underfit/smooth. Try odd k , 9, 11).
• Must scale features. Slow on big data.

5.3 k-Means Clustering (Unsupervised)

Group unlabeled data into k clusters.

Steps: ① Pick k random centroids (cluster centers) ② Assign each point to nearest centroid ③ Move each centroid to the mean of its points ④ Repeat 2–3 until stable

• Choose k with Elbow method (plot inertia vs k) or Silhouette score.

⊠ Assumes round clusters; sensitive to scale and initialization (n_init=10).

5.4 PCA (Principal Component Analysis)

Dimensionality reduction: compress many features into fewer while keeping most variance (information).

• New axes = principal components, ordered by variance explained.
• Steps: standardize → find directions of max variance → project data.
• PCA(n_components=0.95) keeps 95% variance.
• Uses: visualization (2D/3D), speeding up models, removing noise.
• ✗ Components are less interpretable.

5.5 Embeddings & Vector Representations ★

Embedding = a list of numbers (vector) that captures the meaning of something (word, sentence, image, user).

• Similar meaning → vectors are close together.
• "king" − "man" + "woman" ≈ "queen"
• Typical sizes: 384, 768, 1536 dimensions.
• Compare with cosine similarity (that’s your Days 2–3 assignment!).

# semantic search idea
query_vec = embed("How do I reset my password?")
scores = cosine_sim_matrix(query_vec, doc_vecs)
best = scores.argmax()

Used in: RAG, vector databases, recommendation, clustering, duplicate detection.

Mini Project 1 — Telco Churn Pipeline

Data → Preprocessing → Model → Cross-Validation → Metrics → Feature Importance

1. Load & explore: df["Churn"].value_counts()
2. Preprocess: TotalCharges to numeric, fill missing, drop customerID
3. Encode: binary → 0/1; multi-category → one-hot
4. Scale numeric columns (for Logistic Regression/k-NN, not trees)
5. Model: Logistic Regression baseline → Random Forest / Gradient Boosting
6. Use Pipeline + ColumnTransformer (prevents leakage)

7. 5-fold stratified CV

1. Metrics: Recall, Precision, F1, ROC-AUC (churn is imbalanced)
2. Feature importance: model.feature_importances_ → bar chart
3. README: problem, data, approach, results, insights, how to run

from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
pre = ColumnTransformer([
    ("num", StandardScaler(), num_cols),
    ("cat", OneHotEncoder(handle_unknown = "ignore"), cat_cols),
])
pipe = Pipeline([("pre", pre), ("clf", RandomForestClassifier(class_weight =
    "balanced"))])

README template: Problem → Dataset → Approach → Results table → Key insights (e.g. month-tomonth contracts churn most) → Run instructions.

Beginner checkpoint — Days 7–8 ■ Explain the day’s main idea without using the textbook wording. ■ Write one tiny example from memory. ■ Identify one common failure mode. ■ Explain how today’s topic connects to the next day. ■ Write one question you still cannot answer.

Move on when you can… ☐ build a preprocessing pipeline that is fit on the training data only ☐ compare models with stratified cross-validation ☐ evaluate on the test set once ☐ explain which features matter ☐ write a README with honest results If you cannot tick most of these, repeat the worked problems for this day. Taking an extra day is fine; it is not falling behind.