Skip to main content
Monocle provides a dedicated AI Agents dashboard for monitoring your application’s AI activity. Track LLM calls, tool usage, token consumption, estimated costs, and full conversation flows.

​
What Monocle tracks

The dashboard has four tabs:
  • Overview: total calls, token usage, estimated costs, and error rates at a glance
  • Models: per-model breakdown with latency, token consumption, and cost
  • Tools: tool call stats with success/error rates and duration
  • Scorers: evaluation and scorer results with score distributions and explanations
  • Conversations: multi-turn conversation flows with drill-down into individual runs

​
Token tracking

Monocle captures detailed token usage per LLM call, including provider-specific breakdowns when available:
  • Input and output tokens
  • Cached input tokens (cache hits)
  • Cache write tokens
  • Reasoning tokens (for models like o1 and o3)
These metrics are used to estimate costs based on published model pricing.

​
Conversation tracking

You can group multiple LLM calls into a single conversation by setting a conversation ID. This enables the Conversations tab, where you can follow the full flow of an agentic loop: which models were called, which tools were invoked, and how many tokens were consumed at each step.

​
Trace viewer integration

When viewing a trace that contains AI spans, an AI tab appears in the span details sidebar. It shows token breakdowns, input/output previews, and model information for that specific call.

​
Span attributes

All AI instrumentations must emit these attributes for the AI dashboard to work correctly.

​
Standard GenAI semconv attributes

These follow the official OpenTelemetry GenAI Semantic Conventions:

​
Token usage attributes

​
Tool call attributes

​
Evaluation result attributes

AI evaluations and scorer results should be emitted as span events named gen_ai.evaluation.result. The event name is a Monocle convention; the attributes below reuse the OpenTelemetry GenAI namespace. The core attributes follow the official OpenTelemetry GenAI attribute registry: Monocle also recognizes these gen_ai.evaluation.* extensions for richer scorer dashboards:

​
Monocle attributes

These custom attributes power specific dashboard features:

​
Supported SDKs

Vercel AI SDK

Auto-instruments generateText, streamText, generateObject, streamObject, embed, embedMany, and rerank

​
Getting started

See the instrumentation page above for setup instructions, or the AdonisJS guide if you are using AdonisJS.