Skip to content

AgentLoop Framework Integrations

For Hermes Agent, see the Hermes plugin guide. This integration is included in AgentLoop 0.1.1 and newer and exports native per-turn traces through Hermes 0.21.6 observer hooks without new Hermes runtime dependencies. AgentLoop 0.2.0 also supports 0.22.x and 0.23.x. The real runtime pilot documents exact host pins and deterministic-provider validation scope.

For OpenAI Codex CLI, see the Codex guide. Its example runner converts codex exec --json events into native traces for the guided workflow.

For framework-neutral pipeline and stage annotations, see workflow tracing.

For opt-in runtime controls, see the LangGraph harness adapter. The tracing integrations below remain observation APIs; recording a call does not itself enforce a policy.

For completed external execution files, use the dependency-free Harbor ATIF, Harbor trials, bounded OTLP JSONL and Omnigent adapters. They retain source receipts, privacy defaults, missingness and unverified policy observations. Capabilities, offline behavioral probes, paired external-quality cohorts and coordination consume that evidence. These adapters are included in version 0.1.0 and require no Harbor/Omnigent runtime, collector, credentials or new database schema.

AgentLoop integrations are designed for low-friction instrumentation. A team should be able to wrap an existing agent and inspect latency, token, retry, and tool-call waste without changing its framework.

This guide covers the dependency-free integration helpers currently shipped in agentloop.integrations and the generic decorator SDK for custom agents.

Generic Python decorator SDK

Use this path when the team has a custom agent loop rather than LangGraph, CrewAI, or another framework.

import agentloop

agentloop.init(
    project_id="agentloop-demo",
    export_dir="runs",
    auto_upload=False,
)

@agentloop.trace_model(name="planner_llm", model="gpt-4.1-mini")
def plan(question: str) -> str:
    return client.responses.create(model="gpt-4.1-mini", input=question).output_text

@agentloop.trace_tool(name="web_search")
def web_search(query: str) -> list[str]:
    return search(query)

@agentloop.traceable(root=True, agent_name="research_agent")
def run_agent(question: str) -> str:
    plan_text = plan(question)
    docs = web_search(question)
    return synthesize(plan_text, docs)

run_agent("Compare agent observability tools")

Available decorators:

  • @agentloop.traceable(root=True, agent_name="...") starts a full trace for a custom agent entrypoint.
  • @agentloop.trace_tool(name="...") records a tool/function span.
  • @agentloop.trace_model(name="...", model="...") records a model-call-like span.

For a custom Python agent, add the three decorators, run the agent once, and inspect or export the resulting trace.

The three decorators also support generator and async-generator functions. AgentLoop opens the generator span on first resume and records one event when the iterator exhausts, closes, or fails. Nested tracing calls execute under that span, but its trace context is removed whenever the iterator is suspended. This keeps unrelated sibling or interleaved work from being mis-parented and allows an async generator to be closed from another task. The wrappers preserve send(), throw(), asend(), and athrow() behavior instead of reducing the iterator to plain for or async for consumption.

A generator span's duration covers elapsed lifecycle time from first resume until exhaustion or close, including time the consumer pauses between yielded values. An explicit early close()/aclose() records a successful span; a propagated exception or cancellation records an error once. If a consumer stops iterating early without closing the iterator, cleanup is deferred until generator finalization. Code that may break, return, or abandon an iterator should close it explicitly, or use contextlib.closing() / contextlib.aclosing() so the span ends promptly and deterministically.

Async cancellation is recorded as a non-successful span before the original asyncio.CancelledError is re-raised unchanged. This contract also applies to LangGraph nodes, instrumented CrewAI task/agent methods, and OpenAI calls and streams. When cancellation has no message, the event error is CancelledError rather than an empty string.

Captured trace ownership and context

Adapters that finish work in a different callback or task can pass a captured AgentTrace to record_model_call(..., trace=trace) or record_tool_call(..., trace=trace). Capture current_trace() and agentloop.tracer.current_event_id() when the operation starts. Both completed event recorders use the same parent rules:

Arguments Trace and parent used
Omit trace or pass trace=None Active trace; inherit its current event ID unless a non-None parent_id is supplied.
Explicit trace, omitted/None parent_id Supplied trace; record a root event, ignoring the ambient parent even when that trace is active.
Explicit trace and parent_id Supplied trace and captured parent ID.

An explicit target works without an active trace and does not change the caller's context. Without one, recording still requires an active trace. The caller is responsible for supplying a parent ID from the target trace.

from agentloop import bind_trace_context, record_tool_call


def record_completed_tool(trace, parent_id, started_at, duration_ms):
    record_tool_call(
        "search",
        trace=trace,
        parent_id=parent_id,
        started_at=started_at,
        duration_ms=duration_ms,
    )


def run_callback(trace, parent_id, callback):
    with bind_trace_context(trace, parent_id):
        return callback()

Use bind_trace_context(trace, event_id=None) when nested instrumentation needs an ambient trace while the callback runs. Omitting event_id clears the ambient parent. The helper restores both previous bindings after success, exceptions, or cancellation; bindings are local to the current execution context. It does not emit events, start timers, finish traces, or export them.

Enter and exit a binding in the same execution context. Generator adapters must create a fresh binding around each resume/close operation and exit it before yielding a value to their caller. A with block spanning a generator's yield does not automatically unbind on suspension. AgentLoop's decorators handle these boundaries and retain their existing elapsed lifecycle duration semantics.

OpenAI SDK

from openai import OpenAI
from agentloop import trace_agent
from agentloop.integrations.openai import instrument_openai_client

client = instrument_openai_client(OpenAI())

with trace_agent("research_agent") as trace:
    response = client.responses.create(
        model="gpt-4.1-mini",
        input="Research three competitors and summarize the differences.",
    )

trace.export_json("runs/openai_agent.json")

Supported call sites when present:

  • client.responses.create(...)
  • client.chat.completions.create(...)

AgentLoop records:

  • latency
  • model name
  • input tokens
  • output tokens
  • token provenance
  • status/error
  • message count
  • tool count

Token counts come from the SDK's usage object, so they are recorded as provider provenance and a trace made only of these calls reports token_status: exact. A response that carries no usage records unavailable rather than a zero that would later read as a measurement of zero tokens. See Token provenance.

You can also wrap any OpenAI-like callable directly:

from agentloop.integrations.openai import instrument_callable

wrapped_create = instrument_callable(client.responses.create, name="openai.responses.create")

Instrumentation is idempotent: instrumenting the same client or callable more than once is a no-op, so one request records one event. Streaming responses (stream=True) are finalized when the stream is fully consumed, closed, fails, or is cancelled — not when the iterator is created — so the recorded latency covers consumption and final usage is captured when the SDK reports it in the stream. Calls made without an active trace are not recorded and do not raise.

LangGraph

from agentloop.integrations.langgraph import instrument_state_graph, trace_runnable

builder = StateGraph(State)
instrument_state_graph(builder)

builder.add_node("retrieve", retrieve)
builder.add_node("synthesize", synthesize)

app = trace_runnable(builder.compile(), name="research_graph")
app.invoke({"question": "What is agent-loop optimization?"})
app.export_last_trace("runs/langgraph_agent.json")

AgentLoop wraps nodes added after instrument_state_graph(...) and records each node as a tool-call span.

Call instrument_state_graph(builder) before the first add_node(). If a builder already contains nodes, instrumentation raises ValueError before modifying it; it cannot silently report only the nodes added later. Repeated instrumentation of an already instrumented builder is a no-op, including after nodes have been added. Compatible builders must expose a callable add_node and an inspectable nodes mapping or sequence; unsupported shapes raise TypeError.

For a graph whose construction you cannot change, decorate its node functions with trace_node(...) before registering them. trace_runnable(compiled_app) provides the root trace for invoke, ainvoke, stream, and astream, but does not retrofit node instrumentation into an already compiled graph. Node-level visibility requires the builder instrumentation or explicit node decorators. The optional conformance tests cover LangGraph 1.2.11 without making LangGraph a core dependency:

uv run --frozen --with langgraph==1.2.11 python -m pytest tests/test_langgraph_conformance.py -q

CrewAI

from agentloop.integrations.crewai import instrument_agent, instrument_crew, instrument_task

researcher = instrument_agent(researcher)
research_task = instrument_task(research_task)
crew = instrument_crew(crew, name="market_research_crew")

crew.kickoff()

Supported Crew-like methods when present:

  • Crew: kickoff, kickoff_async
  • Task: execute, execute_sync, execute_async, run
  • Agent: execute_task, run, invoke

The integration is dependency-free and works by patching object methods in-place.

Vercel AI SDK / JavaScript agents

For JS/TS applications, export telemetry events as JSON and convert them into an AgentLoop trace on the backend:

from agentloop.integrations.vercel_ai import trace_from_vercel_ai_events

trace = trace_from_vercel_ai_events(events, name="vercel_ai_agent")
trace.export_json("runs/vercel_ai_agent.json")

Expected event fields are flexible:

{
  "type": "generate",
  "name": "generateText",
  "duration_ms": 1200,
  "model": "gpt-4.1-mini",
  "input_tokens": 900,
  "output_tokens": 240,
  "metadata": {"step": "draft"}
}

Event type mapping:

  • text, generate, stream, model -> model_call
  • tool, tool-call, step -> tool_call

OpenTelemetry trace and span IDs

OTLP requires trace IDs to be 32 lowercase hexadecimal characters and span IDs 16, and forbids all-zero values. AgentLoop lets you choose custom run and event IDs (the local/API constructors and the OpenAI Agents bridge all allow them), so when exporting to OTLP (export-otel, trace_to_otel, or the OpenAI Agents processor) AgentLoop maps every identifier to a valid one:

  • An already-valid ID (correct width, non-zero, lowercase hex) is preserved exactly — so a trace imported from OTLP round-trips back out unchanged.
  • Every other ID — non-hex, wrong width, empty, or all-zero — is mapped deterministically with SHA-256. Distinct native IDs stay distinct (for example a and 0a do not collapse to the same value), preserving within-trace uniqueness and parent linkage.

The original native identifiers are preserved on the exported span attributes (agentloop.native_event_id, agentloop.native_parent_id, and, from the OpenAI Agents bridge, agentloop.native_span_id / agentloop.native_trace_id; the run ID is also on agentloop.run_id), so a remapped ID is still diagnosable.

Compatibility note. Importing an OTLP trace now preserves the full 32-hex trace ID rather than truncating it to the last 16, so a trace imported from OTLP has a run ID of the form run_ + 32 hex characters (previously run_ + 16). Native (non-imported) run IDs are unchanged.

Detecting available integrations

To check which framework integrations are available in the current environment, use detection. It reports whether each integration's SDK is importable and does not instrument anything — it never wraps, patches, or registers a call.

from agentloop import detect_integrations

result = detect_integrations()
# result.available   -> ["openai", ...]
# result.unavailable -> {"langgraph": "package not installed", ...}

Or from the CLI:

uv run agentloop detect-integrations

Detection uses importlib.util.find_spec, so it does not import the frameworks and is safe and cheap to run at application startup. To actually record traces, apply the helpers above (or the generic decorator SDK) from the application process you want to trace.

Migration note. detect_integrations() replaces auto_instrument(), and the CLI command detect-integrations replaces auto-instrument. The old name implied it enabled instrumentation, which it never did. auto_instrument() still works as a deprecated alias (it emits a DeprecationWarning), and the result fields were renamed from enabled/skipped to available/unavailable.

Typical workflow

Remote integrations use HTTP API v1. The Python client keeps the server origin or mount path in AGENTLOOP_API_URL and adds the version prefix:

from agentloop import AgentLoopClient

client = AgentLoopClient.from_env()
runs = client.list_traces(page_size=20)  # GET /v1/traces?page_size=20

Upgrade the server before adopting the current client. Existing unversioned HTTP callers retain compatibility throughout 0.x.

  1. Install AgentLoop.
  2. Instrument the framework already used by the team.
  3. Run one production-like agent task.
  4. Upload or store the trace.
  5. Open the dashboard.
  6. Use optimization cards to reduce latency and token spend.

These adapters let developers add AgentLoop without rewriting an agent around a new framework or replacing its existing tracing stack.