VITRINE · Specialist deep dive

Retrieval without a hidden network dependency

How 99 products support reproducible exploration of realistic e-commerce scenarios through an authored concept space, an optional real embedding space, a lexical index, reciprocal-rank fusion, and post-retrieval guardrails—and why retrieval is neither a recommendation-quality verdict nor a safety measurement.

← Back to the VITRINE documentation hub · Value proposal · Operator walkthrough · Evaluation protocol
99product documents
24authored concept dimensions
1,536optional real-vector dimensions
24candidates requested per leg
k = 60RRF rank constant
1–12allowed final top-k range
01 · Space selection

Choose once, report it every time

EmbeddingSpace resolves one vector space for the process. The default and automatic choice is the authored concept space. The real-vector path is explicit because a run must not silently change meaning when credentials happen to be present. It also requires --confirm-paid because every query may be embedded by a live provider. Provisioning and the exact four-variable contract live in the Microsoft Foundry live-run setup.

Embedding-space selection Automatic and concept-vector requests select the authored 24-dimensional space with no egress. A confirmed real-vector request loads committed 1536-dimensional product vectors and requires a live matching query embedder. Missing credentials or a failed identity probe falls back visibly to concept vectors. Requested spaceCLI sets before retrieval Concept vectors24 dims · authored · localAUTO PREFERS THIS Real product vectors1,536 dims · committed index--real-vectors + confirmation Live query embeddercredentials requiredmemoized per exact text Resolved sourcename · model · dimsidentity proof printed auto / concept real no key / mismatch → declared fallback
The fallback is visible in the banner. It is not a zero vector and it does not pretend a real-space search happened.
Default

Concept space

Every product and query is projected locally through a committed 24-concept lexicon. No endpoint or API key is read for this path.

Explicit

Real-vector space

The product index is committed, but a new query must be embedded into the same model space. A product identity probe checks that the live query embedder and index agree before use.

02 · Hybrid retrieval

Meaning and exact identity need different legs

Dense search helps when the customer's wording differs from the catalogue. Lexical search protects model numbers, GTINs, brands, and literal product facts. Their scores are not comparable, so VITRINE fuses ranks rather than raw values.

VITRINE hybrid retrieval pipeline A normalized retrieval query and its filters split into dense and lexical legs. Each leg returns up to 24 candidates. Dense results below the selected space's floor are removed. Reciprocal rank fusion with k 60 merges the rank positions, returning one to twelve candidates with provenance. RetrievalQueryneed · market · categoryattributes · exclusions · k Dense legcosine in selected spacetop 24 · then floor Lexical legweighted tokens + exact booststop 24 Reciprocal rank fusionΣ 1 / (60 + rank)raw scores stay diagnostic RetrievalResultranked hits + leg ranksdegradation + evidence line filters run before the cut—not after ranking
RRF deliberately ignores score scale. A cosine and an IDF-weighted token score do not become comparable merely because both are numbers.
Lexical signalCurrent weight/boostPurpose
Name3.0Product identity and strong title terms
Brand3.0Exact brand preference
Specification value1.5Mounts, capacities, dimensions, and compatibility values
Specification key1.0The kind of fact being requested
Description0.75Recall without letting prose dominate identity
Model-number match+6.0Dense retrieval is weak at arbitrary identifiers
Exact GTIN+12.0An exact barcode is an identity, not a semantic hint
03 · Authored concept vectors

Deterministic meaning, with a visible ceiling

The key-free embedding source is a small authored projection—not a miniature claim of general language understanding. Text is tokenized, phrases of up to three tokens are looked up, concept weights accumulate, and the result is normalized.

Authored concept-vector projection Input text is tokenized and scanned for up to three-token phrases. An authored lexicon maps matched phrases to 24 concept dimensions. Weights are accumulated and normalized. If no phrase is recognized, an all-zero vector is returned and retrieval reports the missing signal. Input textquery or product Tokenizeshared lexical rules Phrase lexicon1–3 token windowsauthored weightsgalaxus-concept-v3 24 dimensionsaccumulatethen normalizeunit length Cosineor zero signal unknown vocabulary → all-zero vector → explicit degraded reason, never invented similarity

What it is good at

Stable demonstrations, known retail concepts, understandable explanations, and repeatable offline verdicts. Artifact bytes differ because receipts deliberately contain run IDs and timestamps.

What it is bad at

Unanticipated vocabulary, linguistic nuance outside the lexicon, and any claim of broad semantic generalization.

Why zero is visible

An unrecognized query has no dense signal. The retriever records a degraded reason rather than ranking 99 identical zero cosines.

04 · Space-dependent operating points

A threshold belongs to a vector space

VITRINE carries separate rows because the numeric shape of an authored 24-dimensional space is not transferable to a 1,536-dimensional provider space. These are the values in current code, not claims of universal accuracy.

Resolved spaceDense floorAttribution floorPrimary confidenceSecondary confidence
Concept0.2800.2000.7030.455
Real vectors0.2230.2210.5200.437

Do not compare scores across spaces as though only the vector length changed. Query embedding, cosine distributions, dense cut behavior, attribution, and confidence routing all change together. The runtime prints the resolved space so the provenance cannot disappear.

Calibration limitation: these are committed operating constants, not a reproducible public calibration package. The original derived calibration JSON and split are not present in this sanitized repository. The recorded held-out bands did not validate every row: the concept dense floor was cleared by roughly 57% of arbitrary products and the real-vector dense floor by roughly 24%. Treat these cuts as sample operating points that need re-derivation on governed production data, not as universal similarity thresholds.

05 · From hit to recommendation

Retrieval is necessary, not authoritative

A fused hit is only a candidate. Recommendation authority arrives later, after the subject can connect the product to a customer signal and to a resolvable catalogue fact.

  1. Pre-filter

    Market, category, attribute, ownership, and exclusion constraints are applied before each leg ranks.

  2. Retain provenance

    Each hit keeps dense rank, lexical rank, raw diagnostic scores, and the best matching catalogue line.

  3. Attribute to an interest

    The subject must identify which independently derived or explicitly stated customer need the product answers.

  4. Resolve catalogue evidence

    The cited review or attribute must resolve against the selected SKU. Plausible prose is insufficient.

  5. Run guardrails

    Sensitive-category, confidence, candidate-containment, compatibility, ownership, and price/stock checks may drop or demote it.

  6. Re-read price and stock

    Presentation uses catalogue authority at render time. A price stated in model-authored reason text is rejected.

06 · Interpretation limits

What this subsystem does—and does not—show

Shown structurally

Offline reproducibility

The default search can run entirely in-process over committed product data and the authored concept projection. Its graph, counts, floors, and failure semantics are testable without credentials.

Shown historically

Stated constraints

A predecessor Galaxus-themed synthetic-sample run measured 0.889 live versus a tag-join oracle's 1.000 at matched k. It is not a Digitec Galaxus production or business metric, and this deep dive does not regenerate that paid measurement. Inspect the bounded receipt →

Not shown

Next-purchase prediction

Four informative pairs make the minimum attainable two-sided p-value 0.125. Better retrieval mechanics do not repair an underpowered design.

Baseline warning

Model value

A tag join reaches 1.000 on several questions with no model calls. Hybrid retrieval and a richer architecture must earn their complexity on matched comparisons.

Security boundary: the offline VITRINE injection gate runs AgentEval's real RedTeamRunner with direct, indirect, and tool-output attacks against deterministic guarded and deliberately vulnerable calibration arms; its poison-dependent forbidden call is not Robin. The separate, explicitly paid Eval 06 plan runs jailbreak and canary-backed system-prompt-extraction probes against fresh real Robin targets and persists a redacted census. The offline gate is not a live-agent resistance rate, and neither lane is a retrieval-quality rate. See the evaluation protocol for their exact evidence and cost boundaries.

For implementation-level details, start with architecture reference and the source files under src/AgentEval.VitrineDemo/Retrieval.