Vetting an Inference Provider's SDK Before You Add It

Start with the manifest, not the README. In Python that is pyproject.toml, setup.py or requirements.txt; in JavaScript it is package.json and the lockfile. What you are looking for is not the list of direct dependencies, which the vendor shows you, but the transitive graph underneath it.

Three things should stop you and make you look harder:

• A bundled or pinned HTTP client. If the SDK requires a specific major version of requests, httpx, axios or urllib3, it may override your own transport, ignore your proxy settings or bypass your instrumented client. That is how an SDK becomes an unmonitored network boundary.
• Native bindings. Packages with compiled extensions (crypto libraries, tokenizers, ONNX runtimes) ship binaries that a source review cannot fully inspect. Check whether wheels are published with hashes, whether builds are reproducible, and whether an install script runs arbitrary code.
• Install-time scripts. Node and Python both allow code to execute on install (NCSC dependency blog). An install hook that downloads a model file or fetches configuration at install time is doing a network call before your review even begins.

Then freeze the graph. Version locking is not glamorous, but it is the control that makes the rest of the audit repeatable. A lock file records every direct and transitive package at an exact resolved version with a content hash, so a restore either reproduces that graph or fails loudly (NuGet version locking write-up). The same principle carries across ecosystems: pin, commit the lock, and run CI in a locked mode so that a dependency change is a reviewable diff rather than a silent drift.

A practical rule for the graph itself: any transitive package that arrives with the SDK and cannot be justified by reading its name is a package you should be able to name and explain. If you cannot explain why foo-utils is in your tree, treat it as suspicious until you can. The academic literature on supply chain risk now catalogues nineteen distinct attack types mapped to twenty-five controls (Journal of Information Security and Applications taxonomy); you do not need to apply all of them to one SDK, but dependency confusion and typosquatting are the two that hit model SDKs most often.

This is the part most teams skip. An AI SDK can send more data outbound than any other library you own: prompts, completions, token counts, error traces and sometimes the full request and response bodies.

Do the grep yourself. For a Python package:

# outbound endpoints and hosts
grep -RnE "https?://|requests\.(get|post|put)|httpx\.|urllib" ./sdk_package
# logging and print calls
grep -RnE "logging\.|logger\.|print\(|console\.log" ./sdk_package
# environment variables and credential reads
grep -RnE "os\.environ|getenv|process\.env" ./sdk_package

For a JavaScript package, substitute fetch( and axios for the HTTP patterns. Triage the output by host. The API endpoint is expected. A second host, a crash-reporting service or a version-check URL is a decision for your security team, not the vendor's default. Any log line that writes the request body, a prompt, or a response will land in your aggregator, and from there in whoever has access to it.

Pay attention to where credentials are read and stored. An SDK that pulls keys from os.environ is normal and good. An SDK that writes the key to a config file, a cache directory or an error payload is not. A common failure mode in agent SDKs is an exception handler that serializes the whole request object, headers included, into the log stream.

A useful habit is to run the SDK once in an isolated container with egress monitoring and a dummy key, then read the connection log. That gives you a factual list of what it contacted, which is more reliable than any marketing page about privacy.

Model APIs fail transiently. Rate limits, timeouts and 5xx responses are normal. The question is whether the SDK's retry loop is safe for an agent that may itself be running many concurrent calls.

Check four things in the retry code:

1. A hard attempt cap. Retries without a cap turn a rate-limit storm into a self-inflicted denial of service. A small default (a handful of attempts) is reasonable; unlimited or configurable-to-unlimited is not.
2. Exponential backoff with jitter. Straight exponential backoff keeps clients synchronized, which recreates the thundering herd on every retry window. Jitter spreads them out. Retry frameworks treat exponential backoff with jitter as the baseline practice for exactly this reason (retry logic guide).
3. Respect for Retry-After. If the server tells the client when to come back, the SDK should use that value. Ignoring it means the retries start before the window opens.
4. A retriable versus fatal split. 429 and 5xx responses are usually retriable. 400 and 401 are not, and retrying them burns velocity and money for no result. A good SDK classifies errors before acting; a bad one retries everything.

Also check what the SDK retries: the request or the whole logical operation. If a tool-calling loop is retried wholesale, a side effect (a write, a payment, a ticket) may execute twice. That is the difference between an idempotent retry and a duplicate action in production.

An error that says "request failed" is useless in an incident. You want typed exceptions, an HTTP status where one applies, a request ID for the provider's support team, and any partial result the response already produced before the failure.

The minimum bar for a model SDK in an agent:

• Typed, distinguishable errors (rate limit, authentication, validation, server, network) rather than a single generic exception.
• Status code and request or trace ID on every network-derived error.
• Partial output preserved for streaming failures, so an agent can resume or report rather than lose work.
• A configurable timeout, with separate connect and read values if available.
• Explicit cancellation for streams, so a stopped tool call closes the connection instead of leaving a generator hanging.

If timeouts are hardcoded, the SDK decides how long your request may hang, not you. If streaming cancellation is absent, an abandoned stream may keep consuming quota until it finishes. Both are direct costs.

NIST's practice groups map cleanly onto this checklist: protecting the software means controlling how components talk to the network and where secrets live, while producing well-secured software means tested error paths and configuration (Black Duck SSDF summary). You can borrow that structure for the review itself: record each finding under a practice so the audit reads like a control check rather than a set of opinions.

Use a two-part test.

• Pass: every outbound host is explainable, retries are capped with jitter and Retry-After handling, errors are typed with status codes and request IDs, timeouts and cancellation are configurable, credentials are read from the environment and never logged, and the version graph is locked and hash-verified.
• Fail: any of those is missing, undocumented or hardcoded.

If any part fails, do not necessarily reject the SDK. Treat it as an untrusted network boundary and wrap it. The wrapper does four things: own the transport and proxy settings, normalize errors into your own typed exceptions, enforce your retry policy in one place instead of trusting the library's, and redact before logging. That wrapper is also where you inject your own request ID, which makes vendor support conversations tractable.

For teams running several model providers, this is where a single interface pays for itself: one transport, one retry policy, one error taxonomy and one logging path, regardless of vendor. Quanta Inference provides text, image, video, audio and embedding models through one OpenAI-compatible API key with prepaid US-dollar pricing, which lets you keep your own wrapper and swap the backend (Quanta Inference).

Smaller teams without a security reviewer can get the same answer from a structured conversation about their goals and constraints rather than a checklist read cold (Quanta Consulting).

Consider an agent that makes 1,000 model calls in a batch, and a provider that starts returning 429 responses at 20 percent of requests for ten minutes. The naive SDK retries each 429 five times immediately, with no cap on concurrency and no jitter. The load almost triples: 200 failed calls become up to 1,000 extra attempts, which pushes the rate limiter into sustained rejection and holds your batch open for the whole window.

The second SDK caps attempts at five, backs off exponentially with jitter, and honors Retry-After. Attempts also arrive spread across the window rather than in a synchronized burst. Once the limiter lifts, the queue drains, and the batch finishes on a predictable timeline.

The third SDK is your wrapper around either of the first two. It enforces the cap, jitter and Retry-After handling in your code, emits a metric per attempt, and logs the request ID on each failure. The SDK's defaults stop mattering, which is the point: you reviewed the vendor once, and the behavior you reviewed is the behavior that ships.

The numbers are illustrative, but the direction holds: retry behavior is a policy decision, and the party that should own it is the party that knows its own traffic.