Agent Observability: How to Track What AI Agents Do Inside Your Product

By Matheus Reis, Co-founder at Kn8 · Published April 14, 2026 · Updated April 21, 2026 · 9 min read

Tags: AI Agents · WebMCP · Observability · Security · SaaS Analytics

The observability gap in agent-ready products: When you add WebMCP tools to your application, a new category of actor starts using your product. AI agents — your users' ChatGPT, Claude, Gemini, or custom browser assistants — can call your tools, trigger workflows, and modify data. Without purpose-built observability, you cannot see what they are doing, cannot debug when something goes wrong, and cannot give your security team the audit trail they require. This article covers how to close that gap.

Why Agent Observability Is Different

Standard product analytics tracks what humans do. Session tracking, event logging, funnel analysis — these assume an individual user takes deliberate actions in a browser with intent and context.

Agent-driven interactions break that assumption in two ways:

Attribution is ambiguous. When an agent calls createInvoice on behalf of a user, who created the invoice? The user, technically — but the agent executed it. If something goes wrong, you need to know which actor did what. Without attribution, your logs show the action happened but not that it was agent-initiated.

Scale and pattern are different. A human might create 3 invoices in a session. An agent might create 50 in under a minute, systematically processing a backlog. Without anomaly detection tuned for agent patterns, legitimate high-volume agent activity triggers false positives — and malicious agent activity may not trigger anything at all.

The Foundation: SubmitEvent.agentInvoked

WebMCP provides one primary mechanism for server-side agent attribution: the agentInvoked property on SubmitEvent.

When an agent submits a form or triggers a tool that results in a form submission, the browser sets event.agentInvoked = true on the SubmitEvent. Your frontend can read this flag and pass it to your backend as part of the request.

// Frontend: capture the agentInvoked flag on form submissions
document.addEventListener("submit", (event) => {
  const isAgentAction = event.agentInvoked === true;

  // Add attribution header to the fetch request
  const headers = {
    "Content-Type": "application/json",
    ...(isAgentAction ? { "X-Agent-Invoked": "true" } : {})
  };

  // Or include it in the request body
  const payload = {
    ...formData,
    _agentInvoked: isAgentAction
  };
});

For tools using the Imperative API, add the attribution directly inside your execute() handler:

navigator.modelContext.registerTool({
  name: "createInvoice",
  description: "Create a new invoice for the specified client.",
  inputSchema: { /* ... */ },
  execute: async (params) => {
    const invoice = await api.invoices.create({
      ...params,
      _agentInvoked: true,    // Explicit attribution in payload
      _agentTimestamp: Date.now()
    });

    return {
      content: [{ type: "text", text: `Invoice ${invoice.id} created.` }]
    };
  }
});

On your backend, treat _agentInvoked: true (or the X-Agent-Invoked header) as an attribution signal and log it alongside every action.

Building a Tool Call Log

Every WebMCP tool invocation should produce a structured log entry. The minimum viable log record:

{
  "event_type": "webmcp_tool_call",
  "timestamp": "2026-04-14T09:23:41.000Z",
  "tool_name": "createInvoice",
  "user_id": "usr_08a7f2c1",
  "session_id": "ses_f4e923ab",
  "agent_invoked": true,
  "parameters": {
    "client_id": "CLT-004821",
    "amount": 4500.00,
    "currency": "USD"
  },
  "outcome": "success",
  "result_id": "INV-2026-00892",
  "latency_ms": 312,
  "ip_address": "203.0.113.42",
  "user_agent": "Mozilla/5.0 (compatible; BrowserAgent/1.0)"
}

Log both successes and failures. Failed tool calls are often more diagnostic than successful ones — they reveal misconfigured tools, permission gaps, and unexpected agent behavior.

What to include in error logs:

{
  "event_type": "webmcp_tool_call",
  "outcome": "error",
  "error_code": "INSUFFICIENT_PERMISSIONS",
  "error_message": "User role 'viewer' cannot create invoices.",
  "tool_name": "createInvoice",
  "agent_invoked": true,
  "user_id": "usr_08a7f2c1"
}

The Five Metrics That Matter

Once you have structured tool call logs, build dashboards around these five metrics:

| Metric | What it reveals | Alert threshold | |---|---|---| | Agent call rate | How often agents are using your product | Spike >5× baseline → investigate | | Tool success rate | Whether tool schemas are well-designed | Drop below 90% → review schema | | Agent vs. human ratio | How much agent traffic you're seeing | Track as trend, no fixed threshold | | Unique tools per session | Which workflows agents automate most | Informs which tools to invest in | | Error rate by tool | Which tools have design or permission problems | Error rate >10% → redesign the tool |

The agent vs. human ratio is particularly useful for product decisions. If agents account for 30% of all exportReport calls but 5% of all createProject calls, you know where agent automation is actually adding value — and where it is not yet being used.

Permission Auditing: The Security Layer

WebMCP's session inheritance model means agents operate with the user's full permissions. This is a feature — agents do not need separate auth — but it creates a visibility gap for security teams who track access by credential, not by session actor.

What your security team needs to see

For every agent-invoked action, your security team needs:

1. Actor attribution — Was this a human or an agent? Which agent (if identifiable from user-agent string)?
2. Permission scope — What operations did the agent perform, and were they within normal scope for that user role?
3. Frequency and volume — Did the agent perform unusual volumes of read or write operations?
4. Cross-resource access — Did the agent touch records outside the user's normal working set?

Implementing a permission audit trail

// Middleware pattern: log permission context alongside every agent action
async function logAgentAction(req, res, next) {
  if (req.headers["x-agent-invoked"] === "true") {
    await auditLog.record({
      actor_type: "agent",
      actor_session: req.sessionId,
      user_id: req.userId,
      user_role: req.userRole,
      action: req.path,
      method: req.method,
      resource_ids: extractResourceIds(req.body),
      timestamp: new Date().toISOString(),
      ip: req.ip
    });
  }
  next();
}

This middleware pattern runs for all API requests and adds agent attribution to your existing audit log infrastructure without touching individual endpoint logic.

Anomaly signals to monitor

Not all anomalies are attacks. Some are legitimate batch operations by agents. But these patterns warrant investigation:

• Bulk deletions initiated by an agent (multiple deleteRecord calls in rapid succession)
• Data exports initiated by an agent (agent calling exportData outside business hours)
• Permission boundary probes — agents calling tools they do not have permission to use, repeatedly
• Cross-tenant tool calls — any attempt by an agent to query resources outside the user's workspace

Agent Traffic in Your Analytics

Beyond security logging, agent traffic creates a new category of product analytics: how agents use your product differently from humans.

Questions your analytics should now answer:

• Which workflows do agents complete end-to-end that humans start but abandon?
• Which tools have the highest agent-to-human call ratio?
• Where do agents fail that humans succeed (schema errors, permission gaps, missing tools)?
• How does session depth compare for agent-assisted sessions vs. unassisted sessions?

A simple segmentation table to build:

| Metric | Human sessions | Agent-assisted sessions | |---|---|---| | Avg. workflows completed per session | X | Y | | Avg. time-to-completion (key workflow) | X min | Y min | | Feature adoption rate | X% | Y% | | Error rate | X% | Y% | | Retention (30-day) | X% | Y% |

Teams that have shipped agent features expect to see: higher workflow completion rates in agent-assisted sessions, lower time-to-completion for complex workflows, and higher feature adoption (because agents surface power features contextually).

If you do not see these patterns, the data tells you something specific: either your tool schemas are causing agent failures, your tools are too narrowly scoped to support full workflow completion, or the agent-assisted experience has friction that prevents users from delegating.

What Kn8 Provides Out of the Box

Building all of this infrastructure from scratch — structured logging, agent attribution, permission auditing, analytics dashboards — is months of work before you ship a single agent feature.

Kn8 provides it as the default observability layer for WebMCP-enabled products:

• Automatic attribution — every tool call is tagged with agentInvoked, user session, and tool name without any additional setup
• Real-time dashboard — tool call rates, success rates, error breakdowns, and agent vs. human ratios in a single view
• Anomaly alerts — configurable alerts for unusual agent behavior patterns
• Audit trail export — compliance-ready logs with full tool call history per user and per agent type
• Permission scope logging — every tool call is annotated with the user role and permission set active at the time of execution

Frequently Asked Questions

How do I distinguish which AI agent called my tool?

The agent's identity is not directly exposed in WebMCP's current spec — the browser does not pass agent identity to the page's JavaScript. What you can access is the user_agent string of the browser session, which may include agent-identifying strings (e.g., BrowserAgent/1.0). Over time, as the spec matures, explicit agent identity passing is expected. For now, the agentInvoked flag tells you that an agent acted; the user-agent string may tell you which agent type.

Does WebMCP observability require changes to my backend API?

Minimal changes. The primary addition is reading the X-Agent-Invoked header (or _agentInvoked body field) and including it in your existing logging calls. If you have a centralized middleware layer, this is a one-line addition. If you log at the endpoint level, you will need to add attribution to each logging call — but the data model does not change.

What if an agent performs a destructive action and I need to reverse it?

This is the idempotency and reversibility problem. For tools that perform destructive or hard-to-reverse operations, design them with reversal in mind: soft-delete instead of hard-delete, create a revision history, or require explicit confirmation via requestUserInteraction(). Your logs will show the agent-invoked action with full parameters — enough to reconstruct what happened. Whether you can reverse it depends on how your data model handles the operation, not on WebMCP.

Should I expose my agent observability data to users?

Many B2B products are already doing this: showing users an "agent activity log" that lists what their AI assistant did on their behalf. This increases trust and gives users control over delegation. If you build this, surface it in your product's activity/audit UI alongside human actions, with an "Agent" label for agent-invoked events.

Key Takeaways

• SubmitEvent.agentInvoked is the foundation of agent attribution — read it on the frontend and pass it to your backend
• Every WebMCP tool call should produce a structured log entry with tool name, user, session, parameters, outcome, and latency
• Track five core metrics: agent call rate, tool success rate, agent/human ratio, tools per session, and error rate by tool
• Security teams need actor attribution, permission scope, frequency anomalies, and cross-resource access patterns
• Agent analytics reveals which workflows agents complete that humans abandon — directly informing which tools to invest in
• Building this infrastructure from scratch delays your first agent feature by months

References and Sources

• W3C Web Machine Learning Community Group. WebMCP Draft Specification — Security Considerations. February 2026. https://webmachinelearning.github.io/webmcp/
• Valence Security. WebMCP Security: Why Every Browser Session Is About to Carry More Power. 2026. https://www.valencesecurity.com/resources/blogs/webmcp-browser-security

Further Reading

• What is WebMCP? — Kn8 Blog
• AI Agents in B2B SaaS — Kn8 Blog
• WebMCP Implementation Guide — Kn8 Blog
• Kn8 Observability Dashboard — Built-in agent analytics

Request access to Kn8 and get agent observability running alongside your WebMCP tools from day one.