Assignment & Project Guide: GenAI Project Ladder

The book does not ask you to build all ten projects at once. Pick a stage only after its prerequisite concepts feel comfortable. The ladder climbs from data cleaning to vector search, then RAG, tools, agents, MCP and a production-style API. Part II has no Level 2 section: each stage has one clear finish line (the book's “Done when…”).

The ten stages at a glance

Stage 1 - Data cleaning notebook

Difficulty: Intermediate | Book: Part II, p. 316

Goal in plain words Turn a messy CSV into a clean dataset and document every change you made.

Words you need EDA = exploring the data with summaries and plots before modeling.

Before you start

• Days 2–3 NumPy/Pandas basics; the Titanic cleaning example is your reference.

Input / data Use titanic.csv and the Days 2–3 cleaning recipe.

Level 1 - Must do

1. Inspect shape, info, missing values and basic distributions.
2. Apply the book's missing-value and categorical-encoding pattern.
3. Save a clean CSV with documentation of each dropped or changed field.
4. Write a short EDA summary.

You are done when… □ Messy CSV is converted to a clean, documented dataset with no unexpected missing values □ Each transformation has a reason; README explains how to reproduce the cleaning □ Output: cleaned CSV, README, 3 documented EDA observations

If you get stuck - mistakes to avoid

• Dropping a column without documenting why
• Changing the book's encoding recipe
• Saving a file with hidden NaNs

Hints

• Hint 1: Start with df.info() and df.isna().sum().
• Hint 2: Use the same Titanic encodings as Days 2–3.
• Hint 3: Save with index=False and document the output filename.

README sections Shared project template plus a “Cleaning decisions” section.

Submit Folder Project-Ladder/Stage-01-Data-Cleaning/ with README.md and the runnable files. Keep large datasets, secrets and generated junk out of GitHub. Reference: VERIFY LOCALLY on titanic.csv; reuse the Days 2–3 code.

Stage 2 - Regression / classification

Difficulty: Intermediate | Book: Part II, p. 316

Goal in plain words Build a simple baseline model, then an improved one, and prove on validation data that the improvement is real.

Words you need Baseline = the simplest sensible model you must beat.

Before you start

• Day 4/3 model fitting and evaluation; validation-set reasoning.

Input / data One of the book's existing generated regression or classification setups; document the dataset/source.

Level 1 - Must do

1. Build a baseline model.
2. Build one improved model from concepts taught in the book.
3. Compare them on a validation set.
4. Explain why the metric and split are appropriate.

You are done when… □ Baseline and improved models are both present; the comparison uses validation data, not the final test set □ README states what changed and why; results are reproducible □ Output: baseline vs improved table, validation result, README explanation

If you get stuck - mistakes to avoid

• Choosing a more complex model without validation
• Leaking scaling/encoding
• Reporting only a training score

Hints

• Hint 1: Baseline first.
• Hint 2: Use the exact Day 4/3 model APIs already shown.
• Hint 3: Do not tune on the test set.

README sections Problem / Dataset / Models / Validation / Results / Reflection.

Submit Folder Project-Ladder/Stage-02-Regression-Classification/ with README.md and the runnable files. Keep large datasets, secrets and generated junk out of GitHub. Reference: VERIFY LOCALLY; reuse the Day 4 or Day 5 pattern.

Stage 3 - Imbalanced classification

Difficulty: Intermediate | Book: Part II, p. 316

Goal in plain words Judge a model on rare events and justify your metric and cut-off with the business cost in mind.

Words you need Threshold = the probability cut-off above which you say “fraud”.

Before you start

• Day 6 precision/recall/F1/ROC-AUC and the class-imbalance workflow.

Input / data creditcard.csv, target Class.

Level 1 - Must do 1. Build the stratified train/test split.

1. Use the metric set from Day 6.
2. Experiment with the decision threshold if you use the book's threshold-tuning pattern.
3. Justify the metric and threshold choice in plain language.

You are done when… □ You can explain why accuracy is not enough; precision, recall and F1 are reported □ Threshold choice is connected to the business cost described in the book; README has the reasoning □ Output: metric table, confusion matrix, threshold observation if used

If you get stuck - mistakes to avoid

• SMOTE before splitting
• Calling 99% accuracy “good” without context
• Mixing threshold choice with the test set

Hints

• Hint 1: Recall = find the positives; precision = how many flagged positives are real.
• Hint 2: Use probabilities for ROC-AUC.
• Hint 3: Keep any resampling strictly on the training side.

README sections Add a “Metric choice” section and a confusion-matrix explanation.

Submit Folder Project-Ladder/Stage-03-Imbalanced-Classification/ with README.md and the runnable files. Keep large datasets, secrets and generated junk out of GitHub. Reference: VERIFY LOCALLY on creditcard.csv.

Stage 4 - Semantic similarity

Difficulty: Intermediate | Book: Part II, p. 316

Goal in plain words Show that embeddings can match meanings, not just keywords: “I forgot my password” should find “Change account login credentials”.

Words you need Embedding shape = (number of sentences, vector length), e.g. (3, 384) for MiniLM.

Before you start

• Days 2–3 cosine similarity; Days 7–8 embeddings and the tiny semantic-search example.

Input / data The exact Days 7–8 sentences and the query “I forgot my password”.

Level 1 - Must do

1. Embed the three documents.
2. Embed the query.
3. Compute cosine similarity.
4. Rank the documents and inspect the scores.
5. State why semantic similarity can find “change account login credentials” without exact keyword overlap.

You are done when… □ Similar sentences rank above unrelated ones in your tests □ Cosine calculation matches your Days 2–3 implementation; README explains embedding shape and similarity □ Output: 3 embedding vectors, query scores, ranked result table

If you get stuck - mistakes to avoid

• Using word IDs as semantic embeddings
• Comparing raw dot products when the book uses cosine
• Treating random vectors as meaningful

Hints

• Hint 1: Normalize rows before the dot product.
• Hint 2: The top score is the retrieval result.
• Hint 3: Use the exact all-MiniLM-L6-v2 example from the book.

README sections “Input sentences”, “Embedding shape”, “Similarity results”.

Submit Folder Project-Ladder/Stage-04-Semantic-Similarity/ with README.md and the runnable files. Keep large datasets, secrets and generated junk out of GitHub.

Reference: VERIFY LOCALLY: the sentence-transformers model is an external download.

Stage 5 - Vector search

Difficulty: Intermediate | Book: Part II, p. 316

Goal in plain words Search 1,000+ text items by meaning and return the top 5 matches for a query.

Words you need Vector database = storage built for fast nearest-neighbor search at scale (FAISS, pgvector).

Before you start

• Days 7–8 k-NN/embeddings; the Part II vector-database and top-k search idea.

Input / data A documented collection of 1,000+ text items and an embedding matrix. For a small collection, NumPy matrix multiplication is enough.

Level 1 - Must do

1. Create embeddings for the 1,000+ items.
2. Implement query embedding + cosine similarity.
3. Return the top 5 items.
4. Inspect several queries and judge whether the results are sensible.
5. Document the data structure and retrieval flow.

You are done when… □ Top-5 search over 1,000+ items returns sensible results; results are ranked by similarity □ Query and corpus vectors have the same dimension; README states the search approach □ Output: top-5 lists for several queries, flow explanation, timing note if measured

If you get stuck - mistakes to avoid

• Mixing embedding dimensions
• Ranking by a different metric without documenting it
• Checking only one query

Hints

• Hint 1: For a small corpus A @ B.T is enough.
• Hint 2: Keep the vector dimension consistent.
• Hint 3: A vector database is about faster nearest-neighbor search at scale.

README sections “Corpus”, “Embedding model”, “Search method”, “Top-5 examples”, “Limitations”.

Submit Folder Project-Ladder/Stage-05-Vector-Search/ with README.md and the runnable files. Keep large datasets, secrets and generated junk out of GitHub. Reference: VERIFY LOCALLY; NumPy matrix search is the toy pattern, FAISS/pgvector the scale-up concept.

Stage 6 - PDF RAG assistant

Difficulty: Advanced | Book: Part II, p. 316

Goal in plain words Let an assistant answer questions from a PDF using only the passages it retrieved, cite where each answer came from, and admit when it does not know.

Words you need RAG = retrieve relevant chunks, then let the LLM answer from them. Chunk = a small piece of the PDF. Grounding = answering only from evidence.

Before you start

• Part II RAG bridge: chunking, embeddings, retrieval, grounding, citations, and the 20-question evaluation idea.

Input / data One small PDF and a fixed set of 20 questions.

Level 1 - Must do

1. First implement plain keyword retrieval so the problem is visible.
2. Replace retrieval with embeddings on the same questions.
3. Return the retrieved chunks with source IDs.
4. Build the answer prompt using only retrieved context.
5. Require citations in the answer and allow “I don't know” when evidence is insufficient.
6. Compare keyword vs embedding retrieval on the same 20 questions.

You are done when… □ Retrieved context is actually used; answers cite sources; the system says “I don't know” when needed □ The same 20 questions are used for the retrieval comparison; README explains one failure case and one fix □ Output: 20-question eval set, retrieved-chunk evidence, answers with source IDs, comparison table

If you get stuck - mistakes to avoid

• Putting the whole PDF into one embedding
• Letting retrieved text act like trusted instructions
• Answering from model memory when evidence is absent
• No source IDs in the retrieved context

Hints

• Hint 1: Inspect retrieved chunks before changing the model.
• Hint 2: Chunking is the first retrieval-quality decision.
• Hint 3: Keep source IDs with every chunk so citations survive retrieval.

README sections Problem / PDF / Chunking / Retrieval / Prompt / Results / Citations / “I don't know” cases / Run.

Submit Folder Project-Ladder/Stage-06-PDF-RAG/ with README.md and the runnable files. Keep large datasets, secrets and generated junk out of GitHub. Reference: VERIFY LOCALLY; follow Part II pp. 304–308 and the ladder rule.

Stage 7 - Tool-using assistant

Difficulty: Advanced | Book: Part II, p. 316

Goal in plain words Let the model ask for a tool call, but make your program validate and run it safely.

Words you need Tool calling flow: user → LLM → structured tool call → validation → tool → result → LLM → answer. Schema = the allowed arguments of a tool.

Before you start

• The book's tool-calling flow.

Input / data The book's get_weather example plus one additional simple Python tool using the same schema pattern.

Level 1 - Must do

1. Describe both tools with a name, purpose and argument schema.
2. Validate every argument before execution.
3. Run the tool and send the result back into the message flow.
4. Make the assistant choose between the two tools.
5. Reject bad arguments safely with a controlled error.

You are done when… □ Two tools work; bad arguments are rejected safely; tool calls are logged or visible □ README documents the validation rules □ Output: two working tools, accepted/rejected examples, execution trace, schema table

If you get stuck - mistakes to avoid

• Calling arbitrary functions by model-provided names
• Skipping argument validation
• No controlled response for malformed tool calls

Hints

• Hint 1: Start with one normal Python function.
• Hint 2: Use JSON-like arguments and validate types/ranges before running.
• Hint 3: The model does not run the tool itself; your program does.

README sections Tool schemas / Validation / Example calls / Error handling / Run.

Submit Folder Project-Ladder/Stage-07-Tool-Assistant/ with README.md and the runnable files. Keep large datasets, secrets and generated junk out of GitHub. Reference: VERIFY LOCALLY; use the book's mock_llm / get_weather control flow.

Stage 8 - Stateful agent

Difficulty: Advanced | Book: Part II, p. 316

Goal in plain words Build a loop where the model picks actions step by step, remembers what happened, and can never run forever.

Words you need Agent loop: action → execution → observation → repeat. State = what has happened so far. Step limit = a hard cap on loop turns.

Before you start

• The book's agent loop; LangGraph is a convenience for stateful workflows, after you understand the plain loop.

Input / data A small multi-step task that genuinely needs more than one tool action.

Level 1 - Must do

1. Add multi-step state.
2. Allow the model to choose actions in a loop.
3. Store what has already happened.
4. Enforce a finite step limit.
5. Log each step and the tool call/result.
6. Stop when the goal is complete or the limit is reached.

You are done when… □ A multi-step task completes with a step limit; execution logs show the sequence □ The agent cannot loop forever; README explains state and stopping conditions □ Output: step-by-step log, final result, a step-limit demonstration

If you get stuck - mistakes to avoid

• No step limit
• State overwritten between turns
• Tools called outside the controlled loop

Hints

• Hint 1: Keep the state explicit in a list/dict before reaching for a framework.
• Hint 2: Use the book's hard-cap pattern; five steps is the runnable example.
• Hint 3: LangGraph is a framework path after the core loop is understood.

README sections Goal / State / Loop / Step log / Stop conditions / Results.

Submit Folder Project-Ladder/Stage-08-Stateful-Agent/ with README.md and the runnable files. Keep large datasets, secrets and generated junk out of GitHub. Reference: VERIFY LOCALLY; a plain Python loop is enough to prove the concept.

Stage 9 - MCP integration

Difficulty: Advanced | Book: Part II, p. 316

Goal in plain words Expose a tool you already built through MCP so any compatible client can use it.

Words you need MCP = Model Context Protocol, a standard connection between AI apps and tools/data. Host/client uses tools; server offers tools, resources and prompts.

Before you start

• MCP as a standard way to connect AI applications to tools and data.

Input / data One tool you already built in Stage 7 or 8, exposed through an MCP server and used by an MCP-compatible client.

Level 1 - Must do

1. Identify the host/client, server and capability.
2. Expose the chosen tool as an MCP server capability.
3. Use an MCP-compatible client to invoke it.
4. Document what MCP standardizes compared with custom tool wiring.
5. Keep ordinary tool-calling semantics understandable in the README.

You are done when… □ Your tool runs as an MCP server used by a client; the flow is reproducible from the README □ README identifies host/client/server roles and documents the exposed capability □ Output: MCP server capability, client execution evidence, architecture/flow diagram

If you get stuck - mistakes to avoid

• Starting MCP before ordinary tool calling
• No client/server role separation
• Hiding the real tool capability behind a generic demo

Hints

• Hint 1: Reuse a known tool from Stage 7.
• Hint 2: Think “standard connection”, not a new tool behavior.
• Hint 3: Document tools/resources/prompts only to the depth the book teaches.

README sections MCP architecture / Capability / Client invocation / What MCP adds.

Submit Folder Project-Ladder/Stage-09-MCP/ with README.md and the runnable files. Keep large datasets, secrets and generated junk out of GitHub. Reference:

VERIFY LOCALLY; use only the MCP concepts introduced on book pp. 311–313.

Stage 10 - Production GenAI API

Difficulty: Advanced | Book: Part II, p. 316

Goal in plain words Package a working GenAI workflow as an API, measure how good, how fast and how costly it is, and make it reproducible.

Words you need FastAPI = a Python web framework. Docker = packaging that runs the same everywhere. Eval set = fixed questions you re-run after every change.

Before you start

• Book roadmap: FastAPI, Docker, evaluation, cost/latency; an eval set of 20-50 questions with simple supported/unsupported scoring.

Input / data The working GenAI project from a prior ladder stage, packaged as an API.

Level 1 - Must do

1. Expose the working GenAI workflow through FastAPI.
2. Package the service with Docker so it runs reproducibly.
3. Create a 20-50 question evaluation set.
4. Record answer quality, plus support-by-context when RAG is involved.
5. Record cost and latency numbers.
6. Document how to run the API and the evaluation.

You are done when… □ Deployed or runnable endpoint exists; Docker configuration included □ 20-50 question eval set exists and results are repeatable □ Cost/latency numbers are recorded; README explains the run/deploy/evaluate flow

If you get stuck - mistakes to avoid

• Shipping without an evaluation set
• No cost/latency measurement
• Hard-coded secrets in the image or repository
• Changing the system without re-running the eval set

Hints

• Hint 1: Keep the API thin: call the already-working workflow.
• Hint 2: Start with a small eval set before changing prompts/retrieval.
• Hint 3: Measure the same questions after each significant change.

README sections API / Docker / Evaluation / Cost & latency / Run / Deployment notes.

Submit Folder Project-Ladder/Stage-10-Production-API/ with README.md and the runnable files. Keep large datasets, secrets and generated junk out of GitHub. Reference: VERIFY LOCALLY; FastAPI and Docker are the book's roadmap tools, the deployment provider is left open.