Integration conformance harness¶
The shared protocol harness and parameterized contracts exercise adapter behavior with local fakes. They require no provider account, framework service, or optional framework SDK.
The current factories cover OpenAI client resource methods, LangGraph builder node registration, and CrewAI task execution. Each declares its tested surface, capabilities, event type, and behavior without an active trace.
| Surface | Sync/async calls | Stream lifecycle | Model usage | Without active trace |
|---|---|---|---|---|
| OpenAI client methods | Supported | Sync and async | Supported; absent usage is unavailable | Original call passes through |
| LangGraph builder nodes | Supported | Unsupported for node wrappers | Unsupported for node spans | Requires an active trace |
| CrewAI task methods | Supported | Unsupported for task wrappers | Unsupported for task spans | Requires an active trace |
The LangGraph declaration concerns instrument_state_graph node wrappers;
compiled TracedRunnable streaming is a separate surface with existing dedicated
tests. Crew-level trace creation is also tested separately from task wrappers.
Unsupported capabilities are named skips with reasons, not implicit omissions.
Run with -ra to display those declarations in the test result:
Shared checks assert exact return-object identity and original exception or cancellation identity. They also verify argument forwarding, repeated instrumentation without duplicate events, parent linkage/context restoration, missing model usage, and exclusion of raw arguments, credentials, and returned content from default telemetry. Stream cases cover deferred recording, full consumption, close/aclose before and after consumption, iteration failure, close failure, cancellation, and final usage.
Add an adapter¶
- Add a factory to
tests/integration_conformance.pytaking(callable, instrumentation_repetitions)and returning the instrumented callable. Construct the smallest protocol-compatible object needed by your adapter. Reapply the real instrumentation function for every repetition. - Add an
AdapterContracttoADAPTERS. Name the precise surface being tested. Every key inCAPABILITIESmust beTrueor a nonempty explanation of why that surface does not support it. Declarewithout_traceaspassthroughorrequires_active_traceand set the expected event type. - Run the shared suite. Add an adapter-specific fixture or contract when a protocol exposes a distinct behavior; do not weaken shared assertions to hide a regression. Keep the existing integration-specific tests for native object shapes and APIs that are outside this common call/stream protocol.
- Document the adapter's supported surface and run its offline example before the full repository checks.
The harness is test-only. It is not a runtime plugin API or automatic package discovery mechanism, and passing these fakes does not establish compatibility with every future upstream SDK release.