All lessons

Course

11th Grade AI: Generative AI and Agent Engineering

Build LLM apps with prompts, APIs, retrieval, and tools, then test them for safety.

Work through the lessons in order. By the end, you can count tokens, compare texts by embedding similarity, and explain sampling and context windows. You can write structured prompts and compare prompt versions with test cases. You can review AI-written code and write tests it must pass. You can get JSON that matches a schema, build a chatbot with a cost limit, and build an app that answers from documents and cites them. You can build an agent with limited tools and a stop rule, defend an app against prompt injection, build an automated test suite, and ship a grounded research assistant.

How to use these lessons

  • Start at the lesson marked Do this. Go on when you can do the line under the title by yourself.
  • Play the Games first. Then take the Quizzes. Eight of ten correct passes. The Worksheets are on paper. The answers are at the bottom.
  • Each lesson builds on the one before it. Keep a test set for every app and run it after each change.
  • The coding lessons use Python. Use only the AI services your teacher approves, keep API keys out of your code, set a spending limit, and never send personal data to a model.

Each lesson has games, quizzes, and worksheets.

0 of 36

LLM foundations

  1. 1Tokens and vocabularyCount tokens and explain why they matter.
    • Tokenize text with a real tokenizer.
    • Compare token counts across languages.
    • Link tokens to cost and context limits.
  2. 2EmbeddingsCompare texts by embedding similarity.
    • Get embeddings from a model.
    • Compute cosine similarity.
    • Rank sentences by meaning, not by shared words.
  3. 3How LLMs generateExplain sampling, temperature, and context windows.
    • Explain next-token prediction.
    • Change temperature and compare outputs.
    • Show what happens when text goes past the context window.
  4. 4Pretraining and tuningTell pretraining, fine-tuning, and instruction tuning apart.
    • Say what data each step uses.
    • Explain feedback from people.
    • Say why a model has a knowledge cutoff.

Prompt engineering

  1. 5Structured promptsWrite a prompt with role, task, rules, and format.
    • Separate instructions from input data.
    • State the output format.
    • Compare a structured and an unstructured prompt.
  2. 6Few-shot examplesImprove output with examples.
    • Add two or three examples.
    • Pick examples that cover edge cases.
    • Measure the change on a test set.
  3. 7ConstraintsLimit length, tone, and format.
    • Set length and reading-level limits.
    • Tell the model what to do when it does not know.
    • Check outputs against each constraint.
  4. 8Prompt testsWrite test cases and compare prompt versions.
    • Write twenty test inputs with expected results.
    • Score two prompt versions.
    • Keep the version that wins.

AI-assisted coding

  1. 9Generate codeGet working code from a clear spec.
    • Write a spec with inputs, outputs, and examples.
    • Generate code from it.
    • Run it on the examples.
  2. 10Review AI codeFind bugs and risks in AI-written code.
    • Check edge cases and error handling.
    • Spot made-up functions and libraries.
    • Spot unsafe code, like secrets in the source.
  3. 11Tests firstWrite tests that AI code must pass.
    • Write tests with pytest.
    • Have AI write code to pass them.
    • Add a test for each bug you find.
  4. 12Debug with AIFix a bug with AI and explain the fix.
    • Make a small example that shows the bug.
    • Ask AI for causes, not just a fix.
    • Explain why the fix works.

LLM APIs

  1. 13First API callSend a chat request and read the reply.
    • Load the API key from the environment.
    • Send a request with an SDK.
    • Read the reply and the token usage.
  2. 14System and user messagesUse system and user messages.
    • Put rules in the system message.
    • Put user input in user messages.
    • Keep the history of a conversation.
  3. 15Structured outputsGet JSON that matches a schema.
    • Define a JSON schema.
    • Request output that follows it.
    • Validate the output and retry on failure.
  4. 16ChatbotBuild a chatbot with history and a cost limit.
    • Keep the conversation history.
    • Trim old messages to fit the context.
    • Stop when the cost limit is reached.

Retrieval-augmented generation

  1. 17Why retrievalExplain why grounding reduces made-up answers.
    • Ask a model about a document it never saw.
    • Add the document and ask again.
    • Draw the retrieve-then-generate pipeline.
  2. 18ChunkingSplit documents into chunks.
    • Split by size with overlap.
    • Split by headings.
    • Keep the source and page with each chunk.
  3. 19Vector searchFind the chunks closest to a question.
    • Embed and store chunks.
    • Retrieve the top matches for a question.
    • Measure how often the right chunk is found.
  4. 20Document Q&ABuild an app that answers from documents and cites them.
    • Put retrieved chunks in the prompt.
    • Require a citation for each claim.
    • Check that citations point to real passages.

AI tools and agents

  1. 21Tool callingLet a model call a function.
    • Describe a tool with a schema.
    • Run the tool the model asks for.
    • Send the result back to the model.
  2. 22PlanningBreak a task into steps a model can run.
    • Ask the model for a plan first.
    • Run the plan one step at a time.
    • Check each step before going on.
  3. 23State and memoryKeep track of what an agent has done.
    • Store each step and its result.
    • Summarize long histories.
    • Decide what the agent should forget.
  4. 24Controlled agentBuild an agent with limited tools and a stop rule.
    • Allow only safe, read-only tools.
    • Cap the number of steps.
    • Ask a person before any action that changes something.

Multimodal generative AI

  1. 25Image generationExplain how diffusion models make images.
    • Explain adding and removing noise.
    • Explain how text guides the image.
    • Discuss consent and copyright for generated images.
  2. 26Vision and languageAsk a model questions about an image.
    • Send an image with a question.
    • Get a structured description.
    • Find a detail the model gets wrong.
  3. 27SpeechTurn speech to text and text to speech.
    • Transcribe a short recording.
    • Measure the word error rate.
    • Generate speech and label it as AI-made.
  4. 28Multimodal prototypeBuild a prototype that uses two kinds of input.
    • Pick two inputs, like voice and image.
    • Connect them in one app.
    • Test it with five users.

Reliability and security

  1. 29HallucinationsDetect answers that the sources do not support.
    • Check each claim against the sources.
    • Use a second model call to flag unsupported claims.
    • Measure how often answers are fully supported.
  2. 30Prompt injectionDefend an app against prompt injection.
    • Write attacks hidden in user input and documents.
    • Separate trusted instructions from untrusted text.
    • Limit what tools an attack could reach.
  3. 31EvaluationScore outputs with rubrics and test sets.
    • Write a scoring rubric.
    • Score by hand and with a model.
    • Check how often they agree.
  4. 32Test suiteBuild an automated test suite for an LLM app.
    • Cover accuracy, format, safety, and cost.
    • Run it with one command.
    • Fail the build when scores drop.

AI product studio

  1. 33Product specWrite the users, features, and success measures.
    • Pick a collection of approved research documents.
    • Write the users and their questions.
    • Set targets for accuracy, citations, and cost.
  2. 34Build the pipelineBuild retrieval, answers, and citations.
    • Index the documents.
    • Answer with citations.
    • Run the test suite.
  3. 35Say I don't knowMake the app say when the evidence is missing.
    • Detect when retrieval finds nothing useful.
    • Answer “The sources do not say” instead of guessing.
    • Test it with questions the documents cannot answer.
  4. 36AI research assistantShip and present a grounded research assistant.
    • Deploy the app for your class.
    • Present its scores for accuracy, grounding, security, and cost.
    • Explain its limits.