Production AI agents course Production AI agents: evals, guardrails and deploy
Take a support agent that refunds money and opens tickets all the way to production: an eval suite that gates every change, a trace for every turn, guardrails in code, cost and rate limits, retries that cannot pay twice, human approval for large refunds, and an HTTP API with a dashboard.
One-time payment · about US$17 · international cards accepted
7 video lessons (3 h) · 7 coding rounds · capstone · certificate
Enrol nowLevelIntermediate
You needPython; an agent loop with tool use (our first course or your own)
Modules7, in order
Ends withCertificate
Taught by Team Homingo
What you’ll build: an agent ready for real money
01Typed, validated tools whose writes are idempotent: a retried refund pays once
02A trajectory eval suite that checks outcomes, tool calls and arguments, with a safety gate in CI
03A trace for every turn: nested spans, tokens, cost and errors
04Guardrails in code: refund policy before any tool runs, an untrusted-data fence, redaction on the way out
05Retries with full jitter and Retry-After, rate limits and a spend budget per turn
06Human approval bound to the exact payload, executed once, expiring on time
07The agent as an authenticated HTTP API with a live dashboard, deployed from a public repo
What you get
- 7 modules of lessons
- From tools that take real actions to deploy as an api, with a dashboard, in order.
- A project in every module
- Each module ends with something that runs.
- Capstone project
- Your own project, in a public GitHub repo with a live demo.
- Certificate of completion
- From Homingo, with your name, when you finish.
- Help
- By email at official@homingo.co.in
Syllabus
7 modules, each ending in code that runs.
Every module ends with a coding round: one more working part of the project, written by you and checked before you move on.
#ModuleYou buildFrom Homingo’s own system
0Tools that take real actionsThe tool layer of a support agent that refunds money: strict schemas, input validated in code, idempotency keys from what a write means, passed to the provider so a retry cannot pay twice, and failures returned as results
- 0.1Tools that take real actions: strict, validated, idempotent22:00
- 0.2Check: tools that act05:00
- 0.3Coding round: validate and call_tool15:00
1Evals for agents: test the trajectoryA trajectory eval gate: score_case checks outcome, tool order, forbidden tools, arguments, step budget and leaks with a reason for every failure, and score_suite fails the run unless the pass rate and every safety case hold
2Tracing and observabilityA tracer for every agent turn: nested spans with ids, monotonic timing, error status that re-raises and OpenTelemetry GenAI attribute names, plus a trace summary with tokens, cost priced safely for unknown models, errors and the slowest step
3Guardrails in codeThe guard layer of the support agent: a refund policy decided in code before any money moves (allow, needs approval or deny, with every reason), and output redaction that masks emails and real card numbers while order totals stay readable
4Cost, rate limits and retriesThe limits layer of the support agent: retries only for errors that can pass later, full-jitter backoff that honours Retry-After, and a token bucket in front of model calls and per API key. The per-turn budget checked before every model call is given, and runs on your code
5Human approval for risky actionsAn approval queue for large refunds: the agent requests, a person approves the exact payload by hash, the policy is checked again at execution time, the action runs once through the idempotent tool path, stale approvals expire, and every decision is audited. The SQLite copy that survives a restart, the executor and the API routes are givenHomingo's own agents draft work into an approval queue (Homingo Ops); nothing sends without a yes
6Deploy as an API, with a dashboardThe dashboard numbers (resolution rate, approval rate, nearest-rank p50 and p95 latency, cost per resolved turn, flagged inputs, top tool errors, the last eval gate) and your own deployment: a public repo, a live URL, a real-model eval table and a demo. Already wired for you to read and run: the HTTP API with customer-app and admin keys, constant-time key checks, a per-key rate limit, a catch-all error handler, the Dockerfile and the CI workflowHomingo Hack Days asks every team for a public repo and a live demo
Certificate
Finish the work, get the certificate.
Complete every module and the capstone project, and you receive a certificate of completion from Homingo with your name and the course title.
Your instructor
Team Homingo
Wrote every module and coding round of this course, and answers your questions by email.
FAQ
Questions before you enrol
- A capstone project in your own public GitHub repo with a live demo, and a certificate of completion from Homingo: work you can show and run, not just notes.