Skip to content

Explicit context transformations

This opt-in API is included in 0.1.0. Ordinary tracing does not rewrite prompts. The first supported transformation summarizes only tool results that the caller explicitly marks optional. It does not choose a policy, identify optional evidence, trim instructions or activate itself.

from agentloop.budgets import BudgetLimits, budget_policy
from agentloop.context_controls import ContextModel, context_policy
from agentloop.context_types import (
    ContextAdapter, ContextPolicyConfig, ContextRequest, SummaryBackend,
)
from agentloop.harness import Harness, HarnessConfig

settings = ContextPolicyConfig(
    "old-search-summary", "1", "answer", "my-independent-scorer-v1",
    max_summary_chars=512, max_summaries=1, on_invalid="error",
)
run = Harness(HarnessConfig(
    mode="shadow",
    policies=(context_policy(settings), budget_policy(BudgetLimits(max_model_calls=2))),
)).start_run("task-01")
model = ContextModel(
    run, my_synchronous_provider,
    config=settings,
    adapter=ContextAdapter("my-text-chat-adapter"),
    summarizer=SummaryBackend("my-summary-model-and-prompt-v1", my_summarizer),
    usage_reader=my_normalized_usage_reader,
)
answer = model(ContextRequest(
    provider_payload,
    optional_tool_results=("old-search-call",),
    protected_tool_results=("current-evidence-call",),
))

The host supplies the three callbacks and an actual provider payload in this integration sketch. my_summarizer receives SummaryInput with content, max_chars, source_hash and the fixed trust class tool. It returns SummaryResult(text, usage), where usage is a normalized ResourceUsage for that one summary call. The provider receives an owned copy of the JSON payload and returns its usual non-streaming result. The usage reader reports only that provider call, excluding its child summaries. Never call another model outside the supplied run to hide summary usage.

The supported message format is a flat messages list with textual system, developer, user, assistant and tool messages. Assistant tool calls may have null content. Every declared tool call must have exactly one result before the next non-tool message. The original roles, call declarations, IDs, all other messages and all provider options retain their values and dictionary ordering. Protected results take precedence over optional selections. Images, streaming requests and async providers are unsupported in the first adapter; unsupported transformations produce an explicit error or the configured original-request fallback.

The capability declaration is a host integration contract, not automatic SDK detection. A matching context_policy must be registered on the same harness. Before-model admission binds one explicit adapter invocation; generic calls on its selected branch cannot borrow the transformation admission. The summary runs after admission and outside policy callbacks. Policies receive metadata rather than raw request content.

Modes, validation and rollback

  • Disabled: pass an owned original payload to the provider; no summary or transformation receipt is created.
  • Shadow: validate and record a proposal, then send the original payload. No summary is executed, including for malformed or unsupported proposals.
  • Enforce: summarize selected long tool results, validate the complete candidate, then invoke the provider. Change modes by creating a new run.

Caller history is never mutated. A summary must be nonempty, shorter than the source and within max_summary_chars. At most max_summaries results may be selected; excess targets fail before any summary call. Summaries replace only the content of their original tool message. Text that resembles an instruction stays tool data. The host must preserve that trust class inside its summarizer too; structural preservation does not prove semantic fidelity or immunity to prompt injection.

on_invalid="original" restores the whole original request after an invalid candidate, including when a later summary fails. It cannot restore a partially modified request. Already incurred summary usage remains charged. A stopped run, cancellation, a streaming request or an unverified configured context limit does not receive this fallback. Provider failures are propagated without automatic retry. Disabling the policy is the rollback; it cannot undo a completed call.

Usage, limits and evidence

Main and summary calls share the same HarnessRun and its admission budgets. Supply DispatchOptions(Reservation(...)) for both the main provider and SummaryBackend when enforcing token or cost budgets. Bounds must include input and maximum output for the respective call. The parent reservation is acquired before preparing summaries, so simultaneous reservations are conservative. Model-call admission can count an attempted preparation even if a child summary blocks actual provider invocation. The receipt separately records provider_invoked; it is not proof of a remote network dispatch.

Summary ResourceUsage includes normalized tokens and cost with their provenance. Unknown usage follows the budget's existing unknown-usage policy; missing cost is never inferred to be zero. Do not count a provider's inclusive parent usage again as exclusive summary usage.

Optional token_counter(payload) returns ContextTokenCount. Without it, pre-dispatch token counts are explicitly unavailable. A configured max_input_tokens requires compatible exact provider, tokenizer or user_supplied counts, with a versioned reference. Word estimates cannot satisfy that bound. If exact before/after counts show no token reduction, the candidate fails validation. The host is responsible for using the provider's actual chat template and counting all relevant request fields. Character reduction alone does not establish a token reduction or quality preservation.

model.export_evidence() and active trace metadata at agentloop.context_transform retain policy/version/configuration, request hashes, summary reference, token provenance, attempted and admitted summary counts, normalized summary usage and preparation time. Preparation time includes the summarizer and admission work, and excludes the final provider invocation. The default receipts contain no message or summary text. Safe error categories do not include callback exception messages. Evidence storage is bounded; exhausting it stops new admission explicitly. Exported receipts are copies.

Provider prefix caching remains provider-dependent. These receipts label its outcome unobserved; a shorter prompt or repeated local hash proves neither a cache hit nor a lower bill. No cache option or schema option is silently changed.

Evaluation

The context study extends the native harness ablation protocol with independently scored, held-out public SQL tasks. Separate context, budget and combined arms permit direct comparisons. The report includes summary calls in total tokens and latency, retains failed transformations, and keeps unknown operating costs visible. A versioned quality reference identifies the host's scorer; the runtime does not run it or claim it validates a summary.

Offline coverage includes roles, pairing, protected evidence, malformed/empty histories, context limits, rollback, cancellation, bounded evidence, exhausted budgets, private error text and shared usage. Synthetic runner tests separately verify that wrong answers and summary failures cannot pass independent quality.