Foundations for Beginners: LLMs, Agents, Tools, and Traces
Learn what a language model, an agent, a tool, and a trace are, and why you decide what correct means before you build.
- Explain in one sentence what a language model does and why its output varies between runs.
- Describe an agent as a model in a loop that can call tools.
- Read a tool call and a tool result and say what happened.
- Explain what a trace is and why we keep one for every conversation.
- State the spec-driven mindset: decide what correct means before you build.
Why this matters
Dev builds a chatbot for Pip's Plant Shop and ships it on Friday. The first chats look great. On Monday, Pip writes: "It told a customer their monstera would arrive on Tuesday. Where did it get that?" There is no record of the conversation, of what the model saw, or of which order it was about. Dev types the same question into the bot and gets a different answer. Nothing was recorded, and the model does not repeat itself, so the mistake cannot be found or reproduced. This lesson gives you the vocabulary that keeps this from happening to you.
1. What a language model actually does
A large language model (LLM) is a program that reads text and predicts what should come next, one small piece at a time. Each piece is a token, roughly a short word or part of a longer one. "the" is one token; "Monstera" is probably two.
The model does not look up an answer. For every possible next token it computes a likelihood, picks one, and repeats until it decides to stop.
Surprisingly, it does not always pick the most likely token. It samples, like rolling weighted dice. Likely tokens win often, unlikely ones win rarely, but the roll is random. This is nondeterminism: the same input can produce different output.
2. A prompt is instructions plus context
A prompt is everything you send to the model in one request. It has two parts. The instructions say what to do and how to behave; when they come from you, the builder, and stay the same for every conversation, they are the system prompt. The context is what the model needs right now: the customer's message, the order record you pasted in, the earlier turns of this conversation.
The same instructions give different answers when the context changes. Paste in order #1042 (shipped) and "Where is my order?" gets "It is on its way with Bloom Post." Paste in order #1077 (processing) and the same question gets "It has not shipped yet."
3. From chatbot to agent: the loop
A chatbot replies with text. An agent can also act. An agent is a model in a loop that can call tools. A tool is a function your code offers to the model, such as "look up an order."
The agent loop fits on a napkin:
messages = [system prompt, customer message]
loop:
reply = model(messages)
if reply is a tool call:
result = run the tool with the arguments the model asked for
append the tool call and the result to messages
else:
done: show the reply to the customer
A tool call is the model's request: run this tool with these arguments. A tool result is what your code sends back. The model writes the request. Your code decides whether to run it.
4. Tools are functions with a contract
To the model, a tool is a description. To your code, it is a function. The tool schema connects the two: the name, what the tool does, and the shape of the input it accepts, written in JSON.
A schema says how to call the tool, not what comes back or what can go wrong. For that we write a tool contract in plain words:
| Part | lookup_order |
|---|---|
| Input | order_id, a string |
| Output | status, items, ordered_at, shipped_at |
| Errors | not_found when no order has that id |
The error row matters most. When Jordan mistypes #1077 as #1707, the tool returns { "error": "not_found" }. A good agent asks Jordan to check the number. A bad agent invents a shipping date.
5. Traces: the flight recorder
A trace is the complete record of one conversation: every message, model call, tool call, and result, in order, with timestamps. It is your agent's flight recorder. When something goes wrong, you open the trace instead of guessing.
Each step inside a trace is a span, with a name, start and end times, an input, and an output. Spans nest: a turn contains model calls, and a model call can contain the tool calls it asked for. L2 covers the full data model.
Now replay Pip's Monday question. With a trace, Dev finds the tool call and sees whether "Tuesday" came from a tool result or from nowhere. Without one, Dev is guessing.
6. Why normal testing is not enough
Testing a normal function is easy: call it, assert on the result.
assert add(2, 2) == 4
assert reply == "Your order ships Tuesday."
The first line works forever. The second fails on perfectly good answers.
Two problems show here. Variety: correct answers come in many wordings. Open-ended inputs: customers ask about orders, refunds, repotting, spider mites, and things you never imagined, so you cannot write one assertion per question.
A third problem hides underneath. An answer can look right and be wrong. "Order #1707 ships Tuesday with Bloom Post" is fluent and confident. If the tool returned not_found, it is also invented: a hallucination, a made-up fact that reads well. The reply alone cannot tell you which replies are true. The trace can.
So you define what correct means in checkable terms: did it call the right tool, did every fact come from a tool result, did it avoid promising a refund. Then you check those across many varied inputs and count how often each fails. That is the rest of this course.
7. The spec-driven mindset
Spec-driven development means you write down what correct means before you build, and everything else is derived from that document, the spec. It says what the agent is for, what it may do, what it must never do, and what a good answer looks like.
The loop has four moves: write down what correct means; build the agent so its behavior is recorded; measure how often it is correct, using real traces; improve whatever fails most, then measure again.
The course has three verbs.
| Verb | Question it answers | Lessons |
|---|---|---|
| Analyze | What is the agent doing? Where does it fail? | L0 to L4 |
| Measure | How often does each failure happen? Can we detect it automatically? | L5 to L7 |
| Improve | Which change fixes it cheapest, and did it really help? | L8 and L9 |
You are in Analyze now.
8. Meet Pip's Plant Shop and Sprout
Pip's Plant Shop is a small online store that sells houseplants and pots. It ships to three zones: Local, Domestic, and International. Returns are accepted within 30 days for plants that arrived damaged. Refunds go to the original payment method.
Sprout is the support agent you will build, evaluate, and improve. Customers chat with Sprout about orders, shipping, refunds, and plant care.
The cast
| Name | Who they are |
|---|---|
| Pip | The owner. Asks the hard questions. |
| Maya | Support agent. Approves refunds and takes over when Sprout escalates. |
| Dev | The engineer building Sprout. That is you. |
| Alex | Customer, order #1042. A monstera that arrived with a broken pot. |
| Jordan | Customer, order #1077. Asks about repotting. |
| Sam | Customer, order #2001. Tries a prompt injection (L7). |
Sprout's tools and their risk tiers
A risk tier says how much damage a wrong call can do. T0 is read-only. T1 is a reversible write: a mistaken cancel_order can be undone by reordering. T2 is irreversible or involves money: a mistaken issue_refund cannot be taken back, so it always requires human approval.
| Tool | What it does | Risk tier |
|---|---|---|
lookup_order(order_id) | Returns status, items, dates for one order | T0 read-only |
get_shipping_status(order_id) | Returns carrier and tracking events | T0 read-only |
search_care_guide(query) | Returns plant-care articles | T0 read-only |
cancel_order(order_id) | Cancels an unshipped order | T1 reversible write |
issue_refund(order_id, amount, reason) | Sends money back | T2 irreversible, needs human approval |
escalate_to_human(summary) | Hands the conversation to Maya | T0, always allowed |
Summary
- A language model predicts the next token and samples it, so the same input can give different outputs.
- A prompt is instructions plus context, and the model remembers nothing unless you resend it.
- An agent is a model in a loop that can call tools. The model asks; your code runs the tool.
- A trace records every step of a conversation as spans, so you can explain and reproduce any behavior.
- Exact-match assertions fail on good answers, so write down what correct means first, then measure it across many traces.
Key terms in this lesson
Check your understanding
Pick an answer to see why it is right or wrong. These are graded in code, instantly.
Apply it
Write your answer in your own words. Eve grades it against a rubric and gives you feedback, the same way an LLM judge works in Lesson 5.
In your own words, what is an agent? Use Sprout as your example.
Jordan writes to Sprout: "Can I return my fern?" Write the first three steps of the trace for this turn. For each step give the span name, its input, and its output. Use Sprout's real tool names. Assume Jordan's order is #1077 and was delivered five days ago.
Instructor notes(for whoever is teaching this lesson)
Talking points
- This lesson is vocabulary plus one habit. The vocabulary: token, prompt, context, agent loop, tool call, tool result, trace, span. The habit: decide what correct means before you build.
- Sampling is the first idea to land. Same input, different output, and nothing is broken. Everything later (why assertions fail, why we measure rates instead of checking one run) follows from it.
- Say "the model asks, your code runs" three times in different words. It is the most useful mental model in the course and the one beginners get wrong.
- Present the trace as a flight recorder, not as logging. The four-row table in section 5 is the shape they will see in Langfuse in L2.
Live demo idea
Send "Suggest a name for a fern" twice at the default temperature and read both answers aloud. Then set temperature to 0 and send it twice more. They will probably match; say out loud that "probably" is the point. Next, send "Where is order #1042?" to a raw API call with the lookup_order schema attached and show the tool_use block in the response. Point at it: nothing has happened yet. Then show the line of your code that would run the function.
Common misconceptions
- "The model runs the tool." No. The model emits a request; your harness runs the function and returns the result.
- "The model remembers the conversation." No. Your code resends the whole history every turn.
- "Temperature 0 makes the model deterministic." Mostly, but not guaranteed. Treat every answer as one sample.
- "If the reply reads well, it is right." A fluent invented date is the canonical failure in this course. Only the trace can tell fluent from true.
Timing
Eight sections at about four minutes each, 32 minutes, plus three minutes to introduce the quiz. Sections 1 and 3 run long; take the time from sections 2 and 4. Section 8 is a two-minute walk through the tables.
If you only have 20 minutes
Teach sections 1, 3, 5, and 7 in that order: what a model does, the loop, the trace, the mindset. Show the four-row trace table and the verbs table. Assign sections 2, 4, 6, and 8 as reading before L1. Run the two-prompts demo if you have three minutes.