AI Engineer Agentic Track: The Complete Agent & MCP

Day 2 - Agentic AI Risks, Guardrails, Evals and Traps to Avoid

Beginner 15 minutes
Rupinder Singh
6 views
The two traps that catch almost everyone building their first AI agent — leading with the solution before the problem, and anthropomorphizing an agent architecture — plus the real mitigations for unpredictable path, output and cost: monitoring and guardrails.

Week 1 · Day 2 · Lecture 11. Before we spend the next six weeks marveling at what agentic AI can do, it's worth being honest about why it's hard to work with — and about two traps that catch almost everyone the first time they try to build an agent for a real business.

The core risk: agentic AI is unpredictable

The same flexibility that makes agents useful — an LLM deciding what to do next rather than following a fixed script — is also their biggest liability. Three things become unpredictable at once:

risks of agent frameworks

  • Unpredictable path — the route the agent takes through the system can differ run to run.
  • Unpredictable output — results aren't deterministic; you can get a different answer each time.
  • Unpredictable costs — the number and size of LLM calls can vary, so spend varies with it.

Mitigation 1: monitoring

Know what your agent is actually doing, both in development and in production.

  • Observability — traces showing exactly what happened on each LLM call.
  • Evals — measured performance checks. The most important is an eval tied to your commercial goal (leads generated, revenue driven) rather than a purely technical metric. A second useful kind is an LLM-as-judge eval — using an LLM to grade the output — which is just the evaluator-optimizer pattern from the previous lecture applied to production monitoring.

Mitigation 2: guardrails

OpenAI defines guardrails as code that "ensures your agents behave safely, consistently, and within your intended boundaries." In practice, guardrails are just tests wrapped around your agent's inputs and outputs — checks you add because an LLM's output is never guaranteed to be something you should act on without verifying it first. You'll build guardrails throughout this course.

Trap #1: leading with the solution instead of the problem

first agent trap

This is a trap almost every technologist building agentic AI has fallen into: a business sponsor calls and says "I need an agent for my business," or more specifically, "I want a Strategy Agent to give me advice for my org." The hype around agentic AI has made people solution-first — they've already decided they want "an agent," sometimes with budget already earmarked for it, before anyone has named the problem it's meant to solve.

The fix is always the same question: "OK — and what business problem is that solving?" Only once you have a real problem can you tell afterward whether the agent actually solved it.

A concrete example: a client asked for a "culture agent" to advise on company culture. Pressed on what problem it would solve, the real issue turned out to be a morale problem, which turned out to be measurable as employee attrition. Once the problem was named as "reduce attrition," an AI agent became one possible option among several — alongside things as basic as running an employee survey — rather than the only conceivable answer.

There's a deeper point buried in this trap. LLMs are trained to generate plausible, believable content — that's the objective function, full stop. A culture agent will cheerfully produce believable-sounding advice about improving culture. Being believable does not make it correct. The only way to know whether an agent's output is actually right is to measure it against a real business outcome and iterate until the metric moves. Without a measurable problem, you can't tell a good agent from a persuasive one.

Trap #2: anthropomorphizing your architecture

second agent trap

The second trap: reaching for a whiteboard and sketching an "agent architecture" with human-sounding job titles before you've built anything. For a stock-trading system, that might look like a Trading Manager Agent coordinating a Market Research Agent, a Trader Agent, and a Risk Manager Agent — an org chart dressed up as a system design.

This is a mistake because it treats LLMs as if they were people with roles, when they are token-generation engines predicting plausible continuations of an input. The org-chart shape feels intuitive because it mirrors how humans divide labor — but that intuition is not evidence that it performs better.

The right approach:

  1. Start simple. One LLM, one prompt, one objective.
  2. Measure that single LLM call against your real business outcome (response rate, leads generated, whatever the deliverable is meant to achieve).
  3. Only then iterate — split into more LLM calls or more agents because it measurably improves performance, never because a multi-agent org chart "sounds right."

It's entirely possible that dividing work along human-team-shaped lines does turn out to perform best — but you should arrive there deliberately, through measured experiments, not by starting there because it mirrors how a human team would be organized.

The three takeaways

  • Build an agent system because it solves a business problem in a way you can measure — not because "agent" is the word of the moment.
  • Choose your architecture because it improves a business metric, not because it looks like a sensible human org chart.
  • LLMs generate believable output, not necessarily accurate output. Aligning next-token prediction with a real business outcome — and adding the guardrails to keep it there — is the AI engineer's actual job. Since you are building the system rather than merely using one, "the model hallucinated" is not an excuse available to you; you're expected to have coded for it.

One more thing: what is an "agentic engineer"?

agentic engineer terminology

The term "agentic engineer" is genuinely ambiguous — it can mean either someone who uses agents to engineer (e.g. a developer who relies heavily on coding agents like Claude Code), or someone who engineers agents (builds agentic systems). Both meanings are in circulation. The recommendation is simply to be explicit about which you mean: after an AI-coding-focused course you'd describe yourself as "an engineer skilled at using coding agents"; after this course, you'll be able to say "an AI engineer skilled at building agents."

Resource links

Next

That wraps Day 2 — theory only, no labs. Day 3 gets hands-on again, with lots of LLM calls.

Frequently Asked Questions (7)

If agentic AI is so unpredictable, why use it at all?

The unpredictability is the flip side of the flexibility that makes agents useful for problems that can't be hard-coded. The lecture's answer isn't to avoid agents, but to mitigate the unpredictability with monitoring (observability + evals) and guardrails rather than pretending it doesn't exist.

What's the difference between an "eval" and a "guardrail"?

An eval measures how well your agent is performing, usually against a business outcome — it tells you if things are working. A guardrail is a check on an input or output that stops a bad result from happening in the first place. You typically need both: evals to know if you're succeeding, guardrails to keep you inside safe bounds while you find out.

Is "LLM as a judge" the same thing as the evaluator-optimizer pattern from the previous lecture?

Yes — using an LLM to grade another LLM's output is exactly the evaluator-optimizer workflow pattern, just applied as a production monitoring technique rather than as a one-off design pattern.

Isn't naming your agents (like "Trading Manager Agent") just a helpful way to organize a system?

Naming things can be useful documentation, but the trap is deciding the org-chart-shaped architecture *before* you've measured anything. The lecture's point isn't that human-team-shaped architectures are always wrong — sometimes they do perform best — but that you should arrive there by starting simple and measuring, not by assuming it upfront because it mirrors how people would divide the work.

As an AI engineer, can I ever blame a hallucination for something going wrong?

Per this lecture, no — not in the way an end user might. An LLM only ever generates plausible tokens; interpreting those tokens correctly, aligning them with a business goal, and catching bad ones with guardrails and evals is the AI engineer's job. If a hallucination reaches production, that's a gap in your guardrails, not the model's fault.

What does the "culture agent" example actually teach?

It shows how solution-first thinking hides the real problem. The client asked for a "culture agent"; only after pushing did the real, measurable problem emerge (employee attrition tied to morale) — and once named, an AI agent turned out to be just one of several possible solutions, not the only one.

Which meaning of "agentic engineer" does this course use?

This course makes you "an AI engineer skilled at building agents" — the engineers-agents meaning. The other common meaning, "someone who uses agents to engineer" (e.g. with coding agents like Claude Code), is what a course like the instructor's AI Coder course teaches instead.