← Events
talk
Agentic AI Observability — Codebasics AI & Data Fest 2025
T-Hub, Hyderabad
A talk on what it takes to monitor AI agents once they are actually running: what to trace, what to attribute cost to, and why non-determinism makes conventional observability insufficient.
Agent demos are easy to build and almost impossible to debug once they are serving real traffic. The same run never happens twice, cost is spread across a dozen model calls, and the failure you need to explain happened three steps before the error surfaced.
This session, delivered at Codebasics AI & Data Fest 2025 at T-Hub, covered the practical side of agent observability:
- What to trace — the full trajectory, not just the final answer. Which tool was chosen, what it returned, and where the loop decided to continue.
- Cost attribution — tying spend to a feature and a user rather than to a monthly provider invoice that tells you nothing actionable.
- Replay and diffing — running a fixed scenario set against every change so a regression is visible before a user finds it.
- Tooling — where AgentOps and OpenTelemetry-based tracing fit, and what each still leaves you to build yourself.
The recurring theme: most agent failures are loop failures, and you cannot fix a loop you cannot see.