Changelog¶
Changes to AgentLoop follow Semantic Versioning.
The published version sequence starts at 0.1.0; routine releases increment the
patch number, such as 0.1.1 and 0.1.2. Package versions are separate from
trace, study, receipt, API and database schema versions.
Unreleased¶
Fixed¶
- Repeated-tool findings now ask developers to investigate when only the tool name repeats. Matching opaque input keys and declared read-only behavior can support testing request-local reuse ahead of concurrency. Plans and estimates identify which change the evidence supports.
Added¶
- Guided comparisons show trace latency and runner wall time in the CLI, JSON, Markdown, and HTML. Per-attempt timing identifies the clock source. Paired summaries show medians and coverage for completed, correct attempts. Latency gates continue to use the trace measurement.
- Guided comparisons can require a trusted local Python verifier selected by the developer. The frozen session binds the script and timeout, and both sides must pass the verifier and their built-in quality checks.
0.2.3 - 2026-11-17¶
Added¶
agentloop improve start --decision aggregatejudges speed or cost across all pairs: the median paired improvement must meet the frozen minimum and more pairs must improve than regress. Every pair must still complete and pass its quality and non-performance gates. Reports include the counts and a non-gating sign-test p-value. The default per-pair decision is unchanged.improve startcompares repetitions of each task and reports arepetition_detourfinding when an attempt took more model steps or tool calls than its typical repetition and used at least 25% more input tokens (runtime when tokens are unrecorded). Evidence lists the extra calls and intermediate model steps. Interleaved controls are diagnosed the same way.
Fixed¶
- The CLI no longer crashes with
UnicodeEncodeErrorwhen its output is piped or redirected on Windows and a task ID, trace name or other text falls outside the console code page. Redirected output is written as UTF-8. improve finishrejects an interleaved control captured before the session was frozen, matching the existing candidate check.- In aggregate decisions, report headlines count pairs that passed correctness and completion checks instead of implying every pair passed overall.
- The Codex example runner records one model span per model round. A message followed by a tool call in the same round was counted as two model calls.
repetition_detourcompares completed attempts only, states observed and typical counts instead of fractional differences, and keeps diagnosis counts in step with the merged findings.- The Codex runner closes a model round when a tool item arrives only on completion, and measures each round from its own start.
improve startacceptsper-pairas well asper_pairand names the CLI spelling in its error.docs/CODEX.mdattributes the first run's token spikes to extra model steps. It previously attributed them to an instruction-file read, which cost nothing extra when it ran in the same step as the requested read.
0.2.2 - 2026-11-17¶
Added¶
release-npm.ymlpublishes the TypeScript SDK to npm from annotatedtypescript-vX.Y.Ztags on main, after Node.js 22/24 build and tests, using trusted publishing with provenance. The SDK installs withnpm install agentloop-profiler.agentloop improve capture --control-arg ...captures an unchanged control runner alongside the candidate, alternating which runs first on each attempt.improve finishthen measures the candidate against that control, keeping findings frozen from the original baseline, so cache warm-up and host drift affect both conditions.examples/codex_exec_runner.pyruns OpenAI Codex CLI (codex exec --json) on guided tasks and converts its event stream into native traces, with a repository task set and a documented first run indocs/CODEX.md.
Changed¶
- Development, CI, release workflows and the Docker image use uv 0.13.7. The supported uv range is now 0.11.8 through 0.13, which lets Dependabot propose Python dependency updates again.
0.2.1 - 2026-11-12¶
Fixed¶
- Published README documentation and community links use absolute destinations that resolve from PyPI. Repository and installed-wheel checks reject relative file links in the package description.
- Installation guidance uses the supported
python -m pip show agentloop-profilercommand to inspect the installed version. - README HTML images and responsive sources use absolute asset URLs for PyPI. The shared description check validates Markdown links and HTML destinations.
0.2.0 - 2026-11-12¶
Added¶
- Reproducible guided repository, retrieval and tool-heavy pilot/held-out suite, with optional pinned local inference, real LangGraph execution, negative controls, native intervention/study artifacts and retained overhead controls.
improve initgenerates a runnable offline example, independent task data, a sync/async Python adapter and a data-only project configuration.- Guided plans show baseline quality, native finding rankings, estimate provenance, assumptions and missing evidence separately from measured results.
- Guided
improve capture,improve start, andimprove finishCLI workflow with fixed task inputs and quality checks, frozen baseline recommendations, content-hashed evidence, complete attempt inventories, native replay and intervention/study results, and readable Markdown/HTML comparisons. - Reproducible Hermes repository-question workflow with pinned local CPU model inference and an incorrect-answer control evaluated through the guided CLI.
- Reproducible Hermes v0.22.7 runtime pilot and CI coverage for streamed requests, real file tools, retries, interruption, default privacy, and delegated child sessions, using an isolated profile and a deterministic local provider.
- Counterbalanced offline instrumentation controls retain recording, buffered JSON export and native analysis timings, with independent arithmetic checks, source provenance and failed/cancelled attempts.
Changed¶
- Hermes observers accept
>=0.21.6,<0.24. CI exercises pinned Hermes 0.22.7 and 0.23.1 hosts with streaming, file tools, retries, cancellation, privacy and delegation. Evidence validation binds each run to one source pin. Published AgentLoop 0.1.1 retains its original Hermes 0.21.x compatibility range.
Fixed¶
- Rejected guided comparisons also fail the linked intervention gates, including candidates with failed execution or no positive measured improvement.
- Guided CLI diagnostics identify failed tasks, missing traces and quality gates.
- Guided capture retains conventional child interrupt exits as cancelled attempts.
- Patch planning prunes generated build, dependency and environment directories, keeping recommendations scoped to application source.
- Installed-wheel validation exercises guided success and rejection paths outside the checkout. Local model studies run through an explicit manual CI input.
- Replay and CI comparisons reject incomplete, interrupted, or failed Hermes captures even when their shorter duration appears to improve performance.
- Schema-repair recommendations require a declared structured-output failure, preserving transport, rate-limit and unknown retries in the underlying metrics.
- TypeScript API requests bind fetch to its global receiver so native browser requests work. Installed-tarball checks exercise esbuild and Chromium, including concurrent spans, cancellation, privacy and redirect rejection.
0.1.1 - 2026-11-05¶
Added¶
- TypeScript SDK for native model/tool/retry tracing and the API v1 routes, with explicit concurrent contexts, qualified usage, private credential scopes, cancellation, bounded responses, and Node/Python interoperability checks. The SDK is available from source; npm publication remains pending.
- Hermes 0.21.6 observer integration and a standalone directory-plugin exporter, with native turn traces, normalized request usage, retry and tool evidence, explicit incomplete capture, profile isolation, and opt-in output text.
Changed¶
- Source builds require Python 3.11 or newer so all supported installations can use patched Pillow and filelock releases. Published 0.1.0 predates this change.
Fixed¶
- Python 3.15 container installs use compatible binary dependencies and receive full CI coverage with released interpreters alongside the existing Python matrix.
- Updated Pillow, certifi, filelock, virtualenv, and setuptools to patched releases, with resolver floors to prevent vulnerable downgrades.
- Published optional-extra requirements and source dependency groups enforce security floors even when an installer does not read uv-specific constraints.
- Failed retry spans retain their error status, and exception formatting cannot replace the business failure when an exception's string conversion raises.
- Python API requests reject redirects, bound response reads, preserve credential scopes, and avoid exposing keys or error response bodies in diagnostics.
- TypeScript metadata copying validates and copies each value once, ignoring custom serializers and rejecting inherited values in sparse arrays.
- Package discovery prunes generated runs, SDK dependencies, tests, and build output before traversing their directory trees.
- Repository link checks exclude generated run artifacts, package smoke environments and JavaScript dependencies while retaining source documentation.
- CLI analysis, reports and study summaries reject destinations that alias source evidence or each other before writing, including symlinks and hard links.
- CI step-summary destinations are checked alongside reports so appending a summary cannot corrupt a trace, fixture or report.
- Release preparation restores the original version, changelog and lockfile bytes after a failed update or interrupted lock generation, so preparation can retry.
- Locked urllib3 and virtualenv dependencies now use patched versions for seven security advisories, with resolver constraints preventing vulnerable downgrades.
Documentation¶
- Reorganized the README around installation, tracing, reports and quality gates, with executable examples and corrected availability notes for version 0.1.0.
0.1.0 - 2026-10-11¶
Added¶
- Local tracing of model, tool and workflow operations, with dependency-free framework adapters, OpenTelemetry conversion and qualified usage provenance.
- Execution graphs, canonical findings, optimization estimates, local HTML and Markdown reports, sampling and incremental large-trace aggregation.
- Quality fixtures, replay gates, paired studies, intervention records, task-level uncertainty and reproducible offline optimization experiments.
- Optional dashboard, HTTP API, project-scoped authentication, SQLite/Postgres persistence, migrations, readiness checks and deployment scaffolding.
- Explicit wrapped-call harness controls, admission budgets, loop guards, completion checks, recorded decisions and optional pinned LangGraph adapters.
- Read-only Harbor ATIF/job/verifier and Omnigent telemetry imports, bounded OTLP JSONL, immutable source receipts, complete negative inventories and qualified session, policy, capability and coordination reports.
- Executable offline capability probes, external-quality cohort comparisons, synthetic examples and measured interoperability scalability results.
- Classified/routed/extracted AI workflow examples, frozen research evidence and calibrated finding evaluation with explicit source and measurement limits.
- Validated Python distributions and standalone CLI artifacts for Linux, Windows, macOS Intel and Apple silicon, with checksums and dependency notices.
Fixed¶
- Interrupted artifact writes remove their new partial files so complete exports can retry. Existing and competing writers' artifacts stay unchanged.
- Harbor study writers validate every generated condition directory before exporting, including existing/dangling symlinks and junctions.
Release process¶
- Release preparation validates the source and opens a reviewable version/changelog PR. Merging preparation does not automatically tag or publish a release.
- Explicit version tags validate the exact source commit before publishing the validated artifact bytes to GitHub and PyPI through trusted publishing.
- Optional live-provider validation, Harbor post-job plugins, active approval resolution and research trajectory export remain outside this release.