qfa.adapters.llm_client#

LLM client adapter using LiteLLM for unified provider access.

Classes

LiteLLMClient(model, api_key, api_base, ...)

LLM adapter satisfying LLMPort via LiteLLM.
class qfa.adapters.llm_client.LiteLLMClient(model: str, api_key: str, api_base: str, api_version: str, chars_per_token: int, max_total_tokens: int)[source]#

Bases: LLMPort

LLM adapter satisfying LLMPort via LiteLLM.

Routes to any LLM provider based on the model string prefix (e.g. "azure/gpt-4", "azure_ai/mistral-large-2411"). Calculates per-call cost using LiteLLM’s built-in cost map or custom pricing registered via litellm.register_model().

Parameters:

  • model (str) – LiteLLM model identifier (e.g. "azure_ai/mistral-large-2411").

  • api_key (str) – API key for the provider.

  • api_base (str) – Base URL for the provider endpoint. Empty string if not needed.

  • api_version (str) – API version string. Empty string if not needed.

async complete(system_message: str, user_message: str, tenant_id: str, response_model: type[T_Response], timeout: float = 40.0) → LLMResponse[source]#

Send a completion request via LiteLLM, retrying transient failures.

timeout is the budget for a single attempt. Transient failures (timeout, rate-limit) and content-policy rejections are retried with exponential backoff up to a total wall-clock budget of LLM_RETRY_BUDGET_MULTIPLIER * timeout; the retry wraps only the provider call, so injection/token checks and response parsing happen exactly once. Callers that enforce a deadline must size timeout so this worst-case budget still fits (the orchestrator does this in _check_deadline_and_get_timeout). Content-policy rejections are retried because Azure’s filter severity classification is not guaranteed deterministic for identical input (#293); other bad-request and generic API errors are not retried — they are not transient. Azure signals a rejection two ways, both mapped to LLMContentPolicyViolationError and both retried: a synchronous BadRequestError (sniffed by _to_domain_error), or a 200 response whose choices[0].message.content is None with its content_filter_results flagging a category (Azure’s asynchronous filter, which lets the call through and blocks the completion after generation). The asynchronous path bills a completion before rejecting it, so usage from every discarded attempt is accumulated and folded into whichever outcome this call ultimately produces: the returned LLMResponse’s token/cost fields on eventual success, or the raised LLMContentPolicyViolationError’s discarded_* fields if every attempt is blocked.

Parameters:

  • system_message (str) – The system-level instruction for the model.

  • user_message (str) – The user-level message to complete.

  • timeout (float) – Maximum time in seconds to wait for a single attempt.

  • tenant_id (str) – Tenant identifier passed as user for audit trail.

Returns:

The model’s response including token usage and cost.

Return type:

LLMResponse

Raises: