Grafana Labs Adds LLM Monitoring and Agent Tracing to Its Open-Source Observability Stack

An agent completes a task and the logs show success, but three hours later a customer flags a wrong output. You pull up your dashboards and latency looks fine, error rate is zero, with nothing to investigate because nothing in your current stack registered it as a failure.

This is the gap Grafana Labs is trying to close. The LLM observability platform market is growing from $1.97 billion in 2025 to $2.69 billion in 2026 at a CAGR of 36.3%, a growth rate that reflects one core problem: enterprises are deploying LLM-powered systems faster than they can instrument them. Latency dashboards and error rates still matter, but they tell you almost nothing about whether an agent reasoned correctly, which tool call ate your API budget, or when a model version started producing worse outputs.

At GrafanaCON 2026 in Barcelona on April 21, Grafana Labs announced a set of new AI-focused capabilities: AI Observability in Grafana Cloud, a significant expansion of Grafana Assistant into more environments, the Grafana Cloud CLI (GCX), and o11y-bench, a new open-source benchmark for evaluating AI agents running observability workflows. Together, these releases attempt to close what the company is calling the “AI blind spot” in modern observability stacks.

Why traditional monitoring breaks down for AI

Standard application performance monitoring was built for deterministic systems. A given input produces a predictable output, and deviations are bugs. AI agents behave very differently compared with traditional software, and existing monitoring tools struggle to provide much insight into agents, making them difficult to debug. The same prompt can trigger different tool sequences, different costs, and different response quality depending on context. Errors look like correct behavior. Latency spikes happen inside multi-step reasoning chains that traditional traces cannot follow.

Grafana Labs’ 2026 Observability Survey found near-universal interest in AI’s value, alongside real caution about autonomy: 15% of respondents expressed skepticism about AI taking autonomous actions without stronger safeguards. That skepticism has a practical dimension. Without visibility into what an agent decided and why, production incidents become extremely difficult to diagnose or prevent. 

AI observability in Grafana cloud

Grafana AI Observability is built on OpenTelemetry and gives teams running LLM agents in production a single place to monitor agent activity, trace conversations, track costs, and evaluate quality. It provides thin SDKs for Go, Python, TypeScript, Java, and .NET, with built-in framework integrations for LangChain, LangGraph, OpenAI Agents, and Vercel AI SDK.

The OpenTelemetry foundation is significant. Rather than building a proprietary data collection layer, Grafana routes AI telemetry through the same standards-based pipeline already handling infrastructure metrics, application traces, and logs. Teams that already instrument with OpenTelemetry get AI observability without rearchitecting their stack.

The platform covers the full AI stack: performance tracking for LLM response times and availability; cost management and real-time spend tracking; token analytics for consumption patterns; quality assessment, including automated hallucination detection and factual accuracy verification; and resource optimization for GPU memory and power consumption. 

The cost attribution capability deserves specific attention. Teams can identify which agent step or tool call accounts for most of their spending and reroute simple queries to cheaper models. That kind of per-step cost visibility turns AI spend from a blunt line item into something engineers can actually optimize.

On the quality side, the integration adds evaluators directly on top of traces. These evaluators allow teams to create alerts for hallucinations, verify factual accuracy, and score content quality, while also monitoring for toxicity, bias, and other safety issues that can be used to gate deployments and alert operators when model quality drifts. This makes quality a continuous operational signal rather than an offline evaluation exercise. 

The feature is now in public preview and treats agent conversations and sessions as first-class telemetry signals, just as Grafana already handles metrics, logs, and traces. 

Full-stack coverage: Beyond the LLM

Most coverage of Grafana’s AI observability release focuses on LLM tracing. The broader scope is worth highlighting. The integration covers GenAI applications with LLM metrics, costs, and quality evaluations; vector database performance and operations; MCP (Model Context Protocol) implementations; and GPU performance across AI workloads, with six pre-built dashboards out of the box. 

This matters because production AI systems rarely consist of a single LLM call. A typical agentic workflow retrieves context from a vector database, passes it through a model, calls external tools via MCP servers, and executes on GPU infrastructure. Visibility limited to the LLM layer leaves the rest of that chain unmonitored. Grafana’s approach instruments each layer and surfaces it in unified dashboards.

GCX: Observability inside the developer workflow

GCX is designed to bring Grafana Cloud into developer workflows, so teams can access the full Grafana Cloud surface area agentically, including provisioning, configuration, and telemetry querying, and invoke Grafana Assistant capabilities from the development environment. The goal is reducing handoffs between code, alerts, and dashboards so investigation and remediation stay closer to where changes ship.

The new Grafana Cloud CLI integrates observability directly into AI-native development environments, enabling agents to query telemetry, correlate code changes, and recommend fixes. GCX is available now in public preview. 

o11y-bench: Measuring what agents do

Grafana Labs open-sourced o11y-bench, a benchmark for evaluating AI agents on observability workflows. Built on Harbor and designed to run against a real Grafana stack, it measures how agents perform on practical tasks: querying metrics, logs, and traces; investigating incidents; and making targeted dashboard changes.

The distinction from static benchmarks matters. Unlike static benchmarks, o11y-bench tests what agents actually do in the system, not just what they say they did. Grafana has seeded a public leaderboard with frontier models and invited the community to test custom agent configurations, model combinations, and harness setups.

Why Grafana is betting on open over proprietary  

According to Grafana Labs’ 2026 Observability Survey, more than 77% of organizations rely on open source or open standards for observability, yet more than 38% of teams still cite complexity as their top challenge. Grafana’s response to that tension is to keep the instrumentation layer open while providing managed tooling for teams that want faster time-to-value. 

The AI observability stack sits on OpenTelemetry standards, which means teams that start on Grafana Cloud can migrate telemetry to any compatible backend without re-instrumenting their codebase. For teams already running self-managed Grafana, a one-click setup connects their instance to Grafana Cloud to access AI Assistant capabilities without switching platforms.

What this signals for the industry

Grafana built AI Observability initially to monitor its own Grafana Assistant internally. Once customers started asking for similar tooling, the team turned it into a full product. That origin story matters: the platform was validated against a real production AI system before it was sold to anyone else.

Mat Ryer, Senior Director of AI at Grafana Labs, stated that AI breaks in ways traditional observability was not designed for, and that visibility into correctness, consistency, and shifting agentic behaviors over time is now required alongside latency and error monitoring.

The broader implication is a category shift. Observability is no longer just infrastructure health and application performance. It now includes output correctness, cost per decision, model version behavior, and the reasoning paths that autonomous agents take between input and action. Grafana is betting that the team already trusted with infrastructure telemetry becomes the natural owner of that broader signal set.

For engineering teams deploying LLM workloads in production today, the practical question is whether to monitor AI systems with a separate specialist tool or fold that visibility into their existing observability stack. Grafana’s announcements make a direct argument for the latter.