Canonical finding rules¶
The investigation ranking contract preserves canonical predictions while exposing evidence, quality risk and validation effort. Rule authors can declare required ranking inputs without adding domain logic to the ranking engine.
The finding trust benchmark evaluates versioned rules against independently labeled fixtures, preserving false positives, abstentions, unknown labels and backend provenance in a release regression gate.
agentloop.rules owns built-in detection. A FindingRule has a stable rule_id,
a version, and a detector accepting AnalysisContext (metrics plus execution
graph). It returns FindingCandidate objects containing wording, confidence,
affected spans, estimates, and available evidence/assumptions.
The registry has a fixed order: parallelization, context caching, schema/retry
validation, batching, routing, large steps, runaway loops, and tool oscillation.
Rule versions are 1.0 for the initial registry. Change a version when changing
a rule's interpretation or detection semantics. There is no remote rule loading,
plugin discovery, or executable rule configuration in the HTTP API.
Reports expose finding_candidates and derive recommendations from them.
Optimization consumes those same candidates instead of applying another set of
thresholds. Callers supplying an older report without candidates use the same
registry as a fallback. Diagnosis, patch planning, CLI, dashboard, API, and CI
therefore use the same finding types and wording. For example, report context
caching now uses the optimizer's title, Cache repeated prompt/context prefix.
Schema repair eligibility¶
add_schema_validation version 1.2 requires a retry span with explicit
retry_reason: invalid_output_schema or reason: schema_validation_failed
metadata. The more specific retry_reason field takes precedence when both
are present. Transport failures, rate limits and missing or unknown causes
remain visible in retry counts and durations. Schema recommendations and their
heuristic savings cover only the spans with declared schema failures.
These declarations identify the recorded cause. The recommendation retains medium confidence, and correctness must be verified using the developer's independent task checks. Existing traces remain readable. Older traces with unclassified retry spans can produce fewer schema-repair recommendations.
Batching eligibility¶
batch_model_calls version 1.2 groups at least three calls with the same name,
model identity, and parent scope. It excludes groups where a call depends on
another group member, including dependencies through tools or other spans.
Native depends_on metadata, declared workflow-stage dependencies, explicit
graph dependency edges, and parent relationships all contribute to this check.
Missing references, malformed declarations, duplicate span IDs, and cyclic
ancestry prevent recommendations for affected groups. A shared external input
does not by itself make the calls dependent on one another.
Inferred sequence edges describe observed order and are not treated as causal dependencies. Traces without dependency declarations can still produce batching hypotheses; verify output independence, shared-state safety, and task quality before applying them. This rule does not infer hidden dependencies from prompts or tool outputs, and it does not rewrite historical findings or study evidence.
Failures and completeness¶
Each detector failure is isolated. Other rules still return their findings, and
rule_errors records the failed ID/version, a safe error category, and a generic
message. Raw exception messages and implementation class names are not serialized
into reports or HTTP responses. For local debugging, enable DEBUG logging for
agentloop.rules; detailed exception logs may contain private inputs and should
remain restricted to operators.
Reports, plans, and diagnoses mark analysis_complete: false; human reports and
dashboard views show the diagnostics. CI fails incomplete diagnosis even when its
performance gates pass, so missing analysis cannot produce a clean merge signal.
When persisting an incomplete diagnosis, returned findings may be updated but missing historical findings are not superseded. A failed rule cannot establish that its earlier findings disappeared. A later complete diagnosis applies the normal lifecycle rules.
Compatibility¶
agentloop.optimizer.OptimizationCard and RecommendationType remain available
as compatibility imports. Finding IDs keep the existing type/title/span fingerprint;
adding rule IDs or versions does not itself replace historical identities. Built-in
detector titles, thresholds, savings calculations, and ordering are retained;
report recommendation wording now follows the canonical detector rather than the
old separate report-only summaries. Rule metadata is additive in JSON and persists
with findings; no database migration is required.