Hybrid Retrieval in MailTrace: Combining Semantic Search with Keyword Matching

Retrieve within the same authorised scope

Hybrid retrieval is not an excuse to search everything and filter later in the prompt. Both retrieval paths should begin with case ownership and current evidence eligibility. Otherwise the lexical branch can reintroduce a document that the vector branch correctly excluded.

The reported fallback reads canonical evidence documents from the application database and current manifest. It does not revive retired vectors or silently present an unavailable index as current. The interface should explain when search coverage is limited, particularly when extraction is missing for a scanned document.

Combine ranks without equating incompatible scores

Raw keyword scores and cosine similarities do not share a universal scale. Reciprocal rank fusion is one possible way to combine result orders. This educational example is a candidate fusion method; it is not a claim that MailTrace’s production service uses this exact formula.

from collections import defaultdict

def fuse(*rankings, k=60):
    scores = defaultdict(float)
    for ranking in rankings:
        unique = list(dict.fromkeys(ranking))
        for rank, source in enumerate(unique, start=1):
            scores[source] += 1 / (k + rank)
    return sorted(scores, key=lambda source: (-scores[source], source))

assert fuse(["email-a", "email-b"], ["email-b", "email-c"])[0] == "email-b"

Measure coverage before polishing the answer

Evaluate questions containing exact request references, informal wording and multiple plausible topics. Compare lexical-only, vector-only and combined retrieval on the same candidate budget. If a reranker is added, measure its additional benefit separately so that extra latency has an identifiable purpose.

A fallback can keep the workflow useful without delivering equivalent recall. The user should still see which evidence supported the answer and which items were not searchable. No retrieval strategy justifies answering a question whose supporting material is absent. In that case, an explicit gap is a better product result than a fluent guess.