qfa.services.analyze#

Application service for the POST /v1/analyze-bulk use case.

Holds both analyse modes — single_pass (AnalyzeService.analyze_bulk()) and hierarchical (AnalyzeService.analyze_hierarchical()) — because they are two modes of one endpoint, selected on the request, and they share the retained-placeholder guardrail (AnalyzeService._ANALYZE_RETAINED_PLACEHOLDER_TYPES). Splitting them would either duplicate that rule or need a third object to hold it.

Per ADR-017 this class has no base class: the shared LLM-call scaffolding (anonymise a batch of records, derive a per-call timeout from the deadline, guard the token budget, run a semaphore-bounded completion) arrives as the injected LLMCallExecutor collaborator. The embedder the hierarchical path needs sits on this constructor rather than on one shared by every use case — it is the dependency that motivated the composition-only decomposition in the first place.

Classes

AnalyzeJudgeResult(*, quality_score, ...)

Structured output of the analyse-judge LLM call.

AnalyzeService(executor, llm, anonymizer, ...)

Free-text analysis of a batch of feedback records.
class qfa.services.analyze.AnalyzeJudgeResult(*, quality_score: Annotated[float, Ge(ge=0.0), Le(le=1.0)], uncertainty_explanation: Annotated[str, MinLen(min_length=1)])[source]#

Bases: BaseModel

Structured output of the analyse-judge LLM call.

The judge returns both a quality score in [0,1] and a short natural-language uncertainty_explanation the analyst can read to understand why the score is what it is.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

model_config = {'frozen': True}#

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

quality_score: float#
uncertainty_explanation: str#
class qfa.services.analyze.AnalyzeService(executor: LLMCallExecutor, llm: LLMPort, anonymizer: AnonymizationPort, settings: OrchestratorSettings, max_total_tokens: int, analyze_settings: AnalyzeSettings | None = None, embedder: EmbeddingPort | None = None, judge_llm: LLMPort | None = None)[source]#

Bases: object

Free-text analysis of a batch of feedback records.

Assembles prompts from feedback records, calls the LLM through the LLMPort, and grades the result with an LLM-as-judge call. Deadline arithmetic, the token-budget guard, batch anonymisation and semaphore-bounded completions are delegated to the injected LLMCallExecutor (executor below), so this class holds use-case logic rather than call scaffolding.

Parameters:

  • executor (LLMCallExecutor) – The shared LLM-call scaffolding (anonymise-records, deadline→timeout derivation, token-budget guard, semaphore-bounded completion) this service delegates to, per ADR-017. It is an injected collaborator, not a base class: the composition root (qfa.api.composition.build_services()) constructs one and shares it with every use-case service.

  • llm (LLMPort) – The LLM provider adapter used for every generation call (the single-pass analysis, the hierarchical map and reduce calls).

  • anonymizer (AnonymizationPort) – The anonymisation adapter used to redact PII before LLM calls.

  • settings (OrchestratorSettings) – Cross-cutting configuration; this service reads chars_per_token for the reduce-phase token estimate.

  • max_total_tokens (int) – Maximum estimated total tokens for a single request, used to size map chunks and reduce groups.

  • analyze_settings (AnalyzeSettings | None) – Configuration for the POST /v1/analyze-bulk endpoint (clustering knobs, coding-trend table inputs, default period). Defaults to AnalyzeSettings with environment-loaded values so tests and callers that don’t care about analyze tuning can omit it.

  • embedder (EmbeddingPort | None) – Optional embedder for mode=hierarchical. None makes the hierarchical path raise AnalysisError at request time, leaving single_pass fully usable.

  • judge_llm (LLMPort | None) – Optional separate adapter for the LLM-as-judge quality-score calls, so judging can run on a different model than generation. None (the default) routes judge calls to llm, which is the behaviour when no JUDGE_LLM_MODEL is configured. Configured via JUDGE_LLM_* and resolved in qfa.api.composition.resolve_judge_llm_settings().

async analyze_bulk(request: AnalysisRequestModel, deadline: datetime, anonymize: bool = True) → AnalysisResultModel[source]#

Analyze a batch of feedback records.

Two LLM calls are issued: the analysis itself, then a judge call that produces quality_score and uncertainty_explanation.

Also computes the deterministic coding_trends table from record metadata (no LLM, no chunking) and returns it. The table is a free win for the single-call path: it depends only on metadata and date parsing, not on map-reduce. When metadata is absent the field comes back as None rather than failing.

Edge cases#

  • mode other than "single_pass" → 422.

  • Judge call failure → 200 with quality_score=null and the constant unavailable-judge explanation.

  • Estimated tokens above the cap → 413 payload_too_large; reduce the batch size. Hierarchical / map-reduce is tracked in #124.

  • Existing regex prompt-injection tripwire still applies and returns 422 prompt_injection_detected.

async analyze_hierarchical(request: AnalysisRequestModel, deadline: datetime, anonymize: bool = True) → AnalysisResultModel[source]#

Analyse a corpus larger than the single-call token cap.

Flow: anonymise each record → deterministic coding-trend table → embed record texts (synchronous, CPU-bound) → cluster (HDBSCAN) → MAP each chunk to a partial (leaf LLM call) → REDUCE the partials (with the trend table), recursing when a chunk or the partial set overflows the token budget → leaf-JUDGE each partial for the confidence. Reduce runs before the judges: the synthesis is the deliverable and gets slot priority, while the judges only feed the secondary confidence signal. The returned confidence is the coverage-weighted mean of the per-chunk judge scores, computed over only the chunks that were successfully judged — chunks whose map or judge call failed are excluded (not scored 0.0) and their count is reported in uncertainty_explanation. confidence is None when no chunk could be judged.

Anonymisation happens before embedding and before every LLM call. Guardrails are applied at both the map and reduce prompts.

Raises:

AnalysisError – When no embedder is configured or the corpus cannot be analysed.