Project 1 — Telco Customer Churn (Day 8)

Goal. A phone company wants to know which customers are about to leave (“churn”) so it can offer them a deal first. You will build a model that answers: will this customer churn, yes or no?

What you will produce. A notebook, a table comparing models against a baseline, a bar chart of the features that matter, and a README with honest results.

The plan in one line. Data → clean → split → preprocessing pipeline → models → cross-validation → one final test → feature importance → README.

Why churn is imbalanced, what precision and recall mean and how cross-validation works are taught in Day 8 (p. 173), Day 6 (p. 134) and Day 5 (p. 114). Here you only run them.

Step 1 — Get the data

1. Make a free Kaggle account and search “Telco Customer Churn”. It is IBM’s sample churn dataset. Read the license on the dataset page and credit the source in your README. 2. Download the CSV. Kaggle may give you a ZIP file: unzip it first. The file inside is called WAFn-UseC-Telco-Customer-Churn.csv . 3. Move it into data/ and rename it telco.csv , so the code and the README use one name:

COMMANDS

# Mac / Linux
mv ~/Downloads/WA_Fn-UseC_-Telco-Customer-Churn.csv data/telco.csv
# Windows PowerShell
Move-Item $HOME\Downloads\WA_Fn-UseC_-Telco-Customer-Churn.csv data\telco.csv

Your notebook lives in notebooks/ , so the code below reads ../data/telco.csv . (Day 8 reads a bare telco.csv because it assumes the file sits next to the notebook. Pick one layout and keep it.) Never use two names for one file.

Step 2 — Load and look

Why: never model data you have not looked at.

COMPLETE RUNNABLE EXAMPLE Checked: ran on stand-in data

import pandas as pd
import numpy as np

df = pd.read_csv("../data/telco.csv")
print(df.shape)
df.info()
print(df["Churn"].value_counts(normalize=True))
df.head()

You should see: about 7,043 rows and 21 columns. Churn is roughly 73% “No” and 27% “Yes”.

What this tells you: the classes are imbalanced. A lazy model that always says “No” is already about 73% accurate, so accuracy alone is a bad score here. You will use recall, precision, F1 and ROC-AUC as well.

If it breaks: FileNotFoundError means the path is wrong. Run import os; print(os.getcwd()) and fix the relative path. KeyError: 'Churn' means the file is not the Telco CSV.

Step 3 — Spot the known trap

Look at the info() output. TotalCharges is listed as text ( object ), although it should be a number. A few rows hold a blank space instead of a value.

COMPLETE RUNNABLE EXAMPLE Checked: ran on stand-in data

blank = df["TotalCharges"].str.strip() == ""
print(blank.sum(), "blank TotalCharges rows")
print(df.loc[blank, "tenure"].unique())   # brand-new customers, so tenure is 0

You should see: about 11 blank rows, and [0] for their tenure. These are brand-new customers with no charges yet. If your counts differ, you have a different version of the file; the idea is unchanged.

Step 4 — Clean

COMPLETE RUNNABLE EXAMPLE Checked: ran on stand-in data

# blanks become NaN
df["TotalCharges"] = pd.to_numeric(df["TotalCharges"], errors="coerce")
df = df.drop(columns=["customerID"])                 # a unique ID helps no model
df["Churn"] = df["Churn"].map({"Yes": 1, "No": 0})   # target as 0/1
print("missing values:", df.isna().sum().sum(), "| Churn values:", df["Churn"].unique())
print("duplicate rows:", df.duplicated().sum())

You should see: a total of about 11 missing values, and [1 0] or [0 1] for Churn. duplicate rows may be above zero: after dropping the ID, two different customers can look identical. Note the number in your README.

Why we do not fill the blanks yet If you fill them with the median of the whole dataset, information from the test rows leaks into training. The pipeline in Step 6 fills them using the training rows only.

Step 5 — Split into features and target, then train and test

COMPLETE RUNNABLE EXAMPLE Checked: ran on stand-in data

from sklearn.model_selection import train_test_split

X = df.drop(columns=["Churn"])
y = df["Churn"]
assert "Churn" not in X.columns            # the answer must never be an input

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, stratify=y, random_state=42)
print(X_train.shape, X_test.shape, y_train.mean(), y_test.mean())

num_cols = ["tenure", "MonthlyCharges", "TotalCharges"]
cat_cols = [c for c in X.columns if c not in num_cols]

Key words. X = the inputs, y = the answer we want to predict. stratify=y keeps the same churn rate (about 0.27) in both parts. The test set is now locked away until Step 9. Cross-validation on the training rows (Step 8) plays the role of the validation set.

You should see: about 5,634 train rows, 1,409 test rows, and both churn rates near 0.27. The assert line stops the run if the answer column ever sneaks into the inputs (target leakage).

Step 6 — Build the preprocessing pipeline

Models only understand numbers, so text columns must be encoded and number columns on very different scales should be scaled.

COMPLETE RUNNABLE EXAMPLE Checked: ran on stand-in data

from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.pipeline import Pipeline, make_pipeline
from sklearn.preprocessing import StandardScaler, OneHotEncoder

pre = ColumnTransformer([
    ("num", make_pipeline(SimpleImputer(strategy="median"), StandardScaler()),
     num_cols),
    ("cat", OneHotEncoder(handle_unknown="ignore"), cat_cols),
])

Read it like a sentence. For the number columns: fill blanks with the median, then scale. For the category columns: turn each category into its own 0/1 column (one-hot).

ColumnTransformer applies a different recipe to different columns. Nothing is learned yet: the medians, scales and category lists are learned later, from training rows only, when the pipeline is fitted.

Step 7 — Set a baseline, then build three models

Why: a model only means something if it beats a baseline. The baseline is a dummy that always predicts the most common answer. It goes through exactly the same cross-validation as the real models, so the comparison is fair.

COMPLETE RUNNABLE EXAMPLE Checked: ran on stand-in data

from sklearn.dummy import DummyClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier

models = {
    "Baseline": DummyClassifier(strategy="most_frequent"),
    "LogReg": LogisticRegression(max_iter=1000, class_weight="balanced"),
    "RandomForest": RandomForestClassifier(n_estimators=300, class_weight="balanced",
                                           random_state=42),
    "GradBoost": GradientBoostingClassifier(random_state=42),
}
pipes = {name: Pipeline([("pre", pre), ("clf", m)]) for name, m in models.items()}

class_weight="balanced" tells the model that missing a churner costs more than missing a loyal customer, which helps with imbalance. Gradient boosting has no such switch here, which is part of what you are comparing.

Step 8 — Compare them with cross-validation

Cross-validation splits the training data into 5 parts, trains on 4, checks on the 5th, and repeats so every part gets a turn. It gives a steadier score than one split and never touches the test set. Because the preprocessing sits inside each pipeline, it is re-learned on every fold's training part.

COMPLETE RUNNABLE EXAMPLE Checked: ran on stand-in data

from sklearn.metrics import make_scorer, precision_score
from sklearn.model_selection import StratifiedKFold, cross_validate

cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
scoring = {
    "accuracy": "accuracy",
    "precision": make_scorer(precision_score, zero_division=0),
    "recall": "recall",
    "f1": "f1",
    "roc_auc": "roc_auc",
}
rows = []

(continued)

for name, p in pipes.items():
    r = cross_validate(p, X_train, y_train, cv=cv, scoring=scoring)
    row = {"model": name}
    row.update({m: round(r["test_" + m].mean(), 3) for m in scoring})
    rows.append(row)
results = pd.DataFrame(rows).set_index("model")
print(results)

You should see: a small table with one row per model. The baseline has recall 0.0 and ROC-AUC 0.5: it never finds a churner. On the real data, expect ROC-AUC somewhere around 0.80–0.85 for the three real models (a typical range, not a promise), with the two “balanced” models showing much higher recall than gradient boosting. Copy your table into your README.

How to choose: the business goal is to catch churners, so look at recall and ROC-AUC together, not accuracy. Pick the model with the best trade-off and write your own reason in the README. Simple and fast counts for something when scores are close.

If it breaks: ValueError: could not convert string to float means a text column skipped encoding; ConvergenceWarning from LogisticRegression is safe to ignore here, or raise max_iter .

Step 9 — Test set: touch it once

COMPLETE RUNNABLE EXAMPLE Checked: ran on stand-in data

from sklearn.metrics import classification_report, confusion_matrix, roc_auc_score

# replace with the model YOUR Step 8 table supports
chosen = "LogReg"
best = pipes[chosen].fit(X_train, y_train)
pred = best.predict(X_test)
prob = best.predict_proba(X_test)[:, 1]
print(classification_report(y_test, pred))
print(confusion_matrix(y_test, pred))
print("ROC-AUC", roc_auc_score(y_test, prob))

Rule: after you see this number you do not go back and tune. If you do, the test set has become a training set and your score is no longer honest. Set chosen from your Step 8 table, never from the test result.

If you want a different threshold (for example, catch more churners at the cost of more false alarms): choose it from cross-validated predictions on the training rows with crossvalpredict , never from the test set. Day 6 (p. 147) shows the idea.

Step 10 — Read the confusion matrix

The matrix prints as [[TN, FP], [FN, TP]] :

Predicted stay Predicted churn

Actually stayed TN: correct FP: wasted discount

Actually churned FN: lost customer we missed TP: customer we saved

Write two sentences in your README: how many churners you caught (recall), and how many of your alarms were real (precision).

Step 11 — Find what drives churn

COMPLETE RUNNABLE EXAMPLE Checked: ran on stand-in data

import matplotlib.pyplot as plt

names = [n.split("__", 1)[1] for n in best.named_steps["pre"].get_feature_names_out()]
# Random Forest / GradBoost: use feature_importances_ instead
coefs = best.named_steps["clf"].coef_[0]
imp = pd.Series(coefs, index=names).sort_values()
top = pd.concat([imp.head(8), imp.tail(8)])
top.plot(kind="barh", figsize=(7, 6), title="Top features: left = stay, right = churn")
plt.tight_layout()
plt.savefig("../outputs/telco_features.png")
plt.show()

What to look for: positive bars push toward churn, negative bars toward staying. On the real data you will probably see month-to-month contracts and short tenure on the churn side, and long contracts on the stay side. Report what your chart shows. These are associations in the data, not proof of cause.

If you chose Random Forest or gradient boosting, use best.namedsteps["clf"].featureimportances in place of coef[0] . Those values are all positive: importance only, no direction.

Step 12 — Write the README and push

Use this template (each heading gets 2–5 sentences). General GitHub steps are in Part IV (p. 379); do not repeat them here.

FILE · README.md

# Telco Churn Prediction
## Problem     - what are we predicting and why it matters
## Data        - source and licence, rows, columns, churn rate, the TotalCharges fix
## Approach    - pipeline, baseline + 3 models, 5-fold stratified CV, test used once
## Results     - the CV table + final test numbers (paste real values)
## Insights    - 3 findings from the feature chart
## Limits      - what this model cannot tell you
## How to run  - pip install -r requirements.txt, then open the notebook

    git add . git commit -m "Telco churn project" git push

You are done when…

n The notebook runs top to bottom with Restart and Run All and no red errors. n Preprocessing lives in a Pipeline and learns from training data only. n You compared the baseline and at least 3 models by cross-validation and justified your pick. n The test set was used exactly once, after the choice was made. n The README has a results table with real numbers, 3 insights and the dataset source.

If you get stuck

Problem Fix

FileNotFoundError Check the file name ( telco.csv ) and run os.getcwd() to see where you are.

ValueError: could not convert A text column skipped encoding. Check that every string to float non-number column is in cat_cols .

Input contains NaN The imputer step is missing from the numeric pipeline, or TotalCharges was not converted in Step 4.

Recall looks suspiciously perfect You may have leaked the answer. Make sure Churn is not in X .

CV scores look great but test is Something was fitted on all the data before splitting. Move it much worse into the pipeline.

Your numbers differ from the book Small differences are normal (versions, seeds). Large ones: re-check Steps 4–5.