Anvi Search: Next-Generation Enterprise Search & RAG Architecture
Anvi Search is an enterprise commerce search platform engineered as a drop-in Oracle Endeca migration instrument and the world's first Agentic-ready hybrid retrieval system — combining Model Context Protocol (MCP) tool contracts, Graph-RAG relational knowledge traversal, dense vector retrieval, explainable merchandising, and 100% private in-VPC data isolation.
1. Executive Overview & The Migration Dilemma
Oracle Endeca (MDEX) is end-of-life, as are sustaining-support ATG Commerce and WebSphere Commerce search installations. Thousands of the world's largest retailers and B2B distributors still run multi-billion-dollar revenue streams through them. These migrations historically stall for two reasons that have nothing to do with basic search indexing:
- Inability to prove empirical relevance on the retailer's real catalog: Vendor ranking decisions are traditionally made on synthetic vendor demos rather than rigorous scientific measurements, causing enterprise architects to defer switching.
- Decades of merchandising rules, triggers, and dimension hierarchies would be destroyed: Thousands of category triggers, dynamic experience cartridges, dimension precedence rules, boost/bury actions, and pinned slots represent millions in institutional investment. Merchandisers correctly view a traditional platform rewrite as pure operational risk.
Anvi Search is built as an Endeca Migration Instrument. It preserves the Endeca conceptual model so business merchandisers do not retrain, provides an explainable score trace for every ranking decision, and ships the offline evaluation harness to measure relevance against your actual catalog before cutover.
Anvi is derived from the Sanskrit अन्वेषण (anveshaṇa) — meaning search, seeking, deep investigation, and systematic inquiry.
2. Agentic AI, MCP Tool Contracts & Advanced RAG
Search in 2026 is no longer just a search box for human shoppers; it is the authoritative sensory and retrieval layer for autonomous AI agents. Anvi Search is built from the ground up with native Agentic AI protocols:
Native Model Context Protocol (MCP) Server
Exposes GET /mcp/tools allowing autonomous AI agents (Claude, OpenAI, Gemini) to inspect catalog schema, query faceted attributes, and execute parametric lookups with dynamic JSON Schema generation.
Agent Readiness & Hallucination Shield
Via GET /readiness, Anvi audits product attribute completeness and generates cryptographic citation proofs, guaranteeing LLM agent recommendations are 100% grounded in verified SKU facts.
Graph-RAG Relational Knowledge Traversal
Traverses complex product compatibility matrices, variant trees, and bundle graphs at query time to answer relational questions (e.g. "which HEPA filter fits my 2021 air purifier?").
Corrective RAG (CRAG) & Confidence Filtering
Automatically scores retrieval confidence across hybrid branches, pruning noisy or low-relevance catalog snippets before feeding generation contexts to optimize agent token budgets.
3. The 10-Stage Request Lifecycle
Every search query executes through an explicit, deterministically ordered 10-stage lifecycle. In Anvi Search, order is the design — each stage may only shape what follows it:
Extracts q, nav_state, profile (audience segment), site, and page_type. Strips spoofable tenant headers and validates cryptographic key scopes.
Unicode case-folding, accent normalization, alphanumeric dimension splitting (e.g. 12v, 100w, 1/2in), and token extraction.
Merchandiser-editable equivalence and one-way sets. Executed before rules so rules evaluate against expanded queries rather than raw strings.
Depth-before-breadth candidate walk. The single winning page defines the baseline layout, hero banners, and promotional cartridges.
Layers dynamic boosts, buries, pins, and redirects on top of the winning page's query plan.
Branch A (Lexical): Lucene/Solr edismax inverted index with configurable match modes (all_partial).
Branch B (Vector k-NN): 384-dimensional quantized e5-small-v2 dense semantic embeddings.
Reciprocal Rank Fusion (Lexical 1.0 / Vector 0.4) executed in the application tier to ensure Endeca sort stacks (Price ASC, Margin) are never dropped.
Deep cross-attention via ms-marco-MiniLM-L6-v2 (+0.035 nDCG boost). Merchandiser pins are strictly preserved and never reordered.
Computes multi-select facet counts. Speller resolves against index vocabulary first, firing only when primary results are empty.
Emits structured JSON with layout zones (Hero, Promo Rail, Product Grid, Facets, and per-stage latency audit metrics).
Model Context Protocol (MCP) Server (GET /mcp/tools)
Standardized tool-calling contract allowing autonomous AI agents (Claude, OpenAI, Gemini) to query catalog facets, search products, and inspect inventory.
Agent Readiness & Grounding Shield (GET /readiness)
Guarantees zero AI hallucination by evaluating catalog attribute completeness and generating verified citation proofs for LLM outputs.
Graph-RAG Relational Knowledge Traversal
Traverses product compatibility matrices, variant trees, and bundle graphs to resolve relational questions.
Corrective RAG (CRAG) & Confidence Filtering
Evaluates retrieval confidence across hybrid branches; automatically filters out low-relevance items before feeding generation contexts.
In-Session Real-Time Intent Drift Tracking
Dynamically updates user intent vectors within the active shopping session based on click paths without requiring persistent PII.
Multimodal Vision-Language Search (SigLIP / CLIP)
Joint vision-language embedding space enabling visual similarity search combined with text attributes.
Query NLU & Token Normalization
Deep query normalization, Unicode case-folding, punctuation normalization, and token-level accent stripping.
Query-Side Synonyms Engine
Merchandiser-editable one-way and two-way equivalence synonym expansion running before rules evaluate.
NLP Intent & Entity Decomposition
Deconstructs compound queries into structured attribute filters (colors, sizes, brands, price boundaries).
Vocabulary-Aware Speller & Did-You-Mean
Spelling suggestion that validates against index vocabulary first, firing only when strict matches fail.
Lexical BM25 / Edismax Engine
High-throughput Lucene/Solr inverted index retrieval with configurable multi-match modes.
Dense Vector Semantic Search (kNN)
384-dimensional dense semantic embedding retrieval powered by quantized e5-small-v2 ONNX models.
Cross-Lingual Retrieval Engine
Retrieves English product catalog records from queries submitted in Spanish, French, or other languages.
Application-Layer RRF Hybrid Fusion
Reciprocal Rank Fusion in the application tier (Lexical 1.0 / Vector 0.4) preserving sort stacks.
Cross-Encoder Neural Reranker (ms-marco-MiniLM-L6-v2)
Deep sequence-pair cross-attention reranker providing +0.035 nDCG boost.
Learning to Rank (LTR / Click-Feedback)
Machine learning ranking (LambdaMART) trained on verified live click, add-to-cart, and purchase streams.
Landing Page & Trigger Resolution (Depth > Breadth)
Deterministic page and cartridge resolution walking the navigation state hierarchy.
Merchandising Rules Engine (Boost, Bury, Pin)
Layered business rules modifying product elevations, score multipliers, redirects, and banner slots.
The Resolution Inspector (Visual Debugger)
Interactive diagnostic workbench answering Why is this page showing? with complete candidate traces.
Guided Navigation & Dynamic Faceting
Multi-select facet counts, hierarchical dimension trees, and dynamic facet reordering.
Negotiable Retrieval on Zero Results
Analyzes binding constraints when queries return 0 results and suggests relaxed alternatives with live counts.
Sub-Millisecond Typeahead & Autocomplete
Ultra-fast prefix search honoring tenant, entitlements, and merchandiser exclusions.
Lane A: Batch Catalog Sync & Blue-Green Reindex
12-step full catalog reindex with validated configsets, monotonic guards, and node parity validation.
Lane B: Real-Time Price & Inventory Streaming (CDC)
Microsecond price, stock, and status updates via high-watermark PostgreSQL streaming.
Lane C: Dynamic Graph & Product Relations
Calculates and updates product bundles, cross-sells, and compatible accessories.
Query Health & Fallback Analytics
Monitors search fallback rates, zero-result frequency, and shallow rerank pool warnings.
Atomic Append-Only Audit Trail
Complete, immutable record of every merchandising change, rule edit, and API key action.
100% In-VPC Isolation & Data Protection
Runs entirely within your cloud VPC or bare-metal environment with zero external telemetry egress.
4. Reciprocal Rank Fusion
Score-Free Ranked MergeMerges lexical and vector candidate lists using calibrated RRF weighting (Lexical 1.0, Vector 0.4, k=60), solving vocabulary mismatch.
- BM25 Candidates
- Vector k-NN Candidates
- Unified Top 60 Hybrid Candidates
Batch Blue-Green Reindex
Hourly / Nightly Baseline
Real-Time CDC Streaming
Continuous (Sub-20ms Freshness)
Dynamic Relations Graph
Graph-RAG Periodic Linker
Lane B: Real-Time CDC Streaming
Continuous (Sub-20ms Freshness)Captures row-level database changes (price modifications, stock level drops, flash sale flags) and streams them via CDC directly into an in-memory document overlay.
Oracle Endeca (MDEX 11.3)
Single-threaded query processing bound to proprietary MDEX binary. Vulnerable to memory fragmentation.
Brittle XML configuration files running multi-hour monolithic batch crawls.
Proprietary XML rule engine for boost/bury, slotting, and dimension precedence.
Proprietary query parameters embedded in ATG JSP / React storefront form handlers.
Anvi Search Platform
Concurrent multi-threaded C++ / Rust core executing dual BM25 + Vector HNSW retrieval.
Continuous event streaming via Kafka / Debezium with in-memory delta overlays.
1:1 drop-in XML import with real-time audit tracing and explainable rule collision visualizer.
Transparently accepts legacy Endeca query parameters and maps them to hybrid plans.
Dgraph Engine ➔ Stateless In-VPC Hybrid Core
Unpredictable latency spikes under heavy concurrent faceting.
Sub-10ms predictable p99 latency across millions of SKUs with horizontal auto-scaling.
Dynamic Tool Contract Discovery
Agent Tool Execution Call
Hallucination Shield & In-VPC Verification
Grounded Citation Payload Delivery
Dynamic Tool Contract Discovery
The AI Agent queries the MCP gateway to discover active catalog dimensions, available tools, and parameter schemas dynamically without hardcoded prompting.
{
"tools": [{
"name": "catalog_search",
"description": "Hybrid vector + BM25 catalog search with parametric facet filtering.",
"parameters": {
"query": { "type": "string" },
"filters": { "type": "object" }
}
}]
}4. The Three Data Ingestion Lanes
Commerce data changes at three fundamentally different cadences. Rather than forcing all updates through a single bottleneck, Anvi Search separates ingestion into three dedicated lanes:
Lane A: Full Catalog Reindex (Batch)
12-step automated blue/green collection reindexing. Features validated configset pushes, monotonic timestamp guards, and multi-node parity assertions to guarantee deterministic tie-breaking.
Lane B: Price & Stock CDC (Real-Time)
Sub-20ms streaming price and stock availability updates via POST /price/{domain} with PostgreSQL high-watermarks. Maintains live stock accuracy during flash sales without reindexing.
Lane C: Relational & Graph Feeds (Async)
Updates dynamic cross-sell matrices, product bundles, and compatible accessories via POST /relations on the zero boundary for ERP and PIM synchronization.
Node Parity & Determinism Guard
Guarantees that identical documents across replica nodes receive identical ranking scores and deterministic tie-breaking, preventing pagination jump defects.
5. Merchandiser Workbench & The Resolution Inspector
A major point of friction in search migrations is merchandiser enablement. Anvi Search provides 8 purpose-built workbench screens designed around the mental model of enterprise commerce operators:
| Workbench Tool | Capabilities & Operational Benefit |
|---|---|
| The Resolution Inspector | Answers "Why is this page showing?" for any navigation state. Evaluates every candidate in the category hierarchy and displays distinct verdicts (Won, Never Published/Draft, Expired Schedule, Not Found). |
| Visual Category & Tree Editor | Hierarchical navigation management with drag-and-drop category restructuring and dimension inheritance. |
| Landing Page & Cartridge Builder | Visual assembly of promotional banners, rich text, carousels, and product elevation widgets. |
| Unified Segmentation Engine | A/B test variants and personalization use the same depth-before-breadth hierarchy — zero separate segment databases to sync. |
| Atomic Append-Only Audit Trail | Every rule modification and synonym change is committed atomically within the database transaction, providing tamper-evident governance logs. |
6. Empirical Benchmarks (Amazon ESCI Dataset)
Relevance claims should be established through rigorous empirical measurement, not vendor assertions. Anvi Search was evaluated against 4,998 real commerce products and 5,482 human relevance judgements across 191 queries from the Amazon ESCI benchmark (graded Exact, Substitute, Complement, Irrelevant):
| Retrieval Configuration | nDCG@10 Score | Recall@50 (Grade ≥ 2) | Empirical Characteristics |
|---|---|---|---|
| Pure Lexical (BM25 / Edismax) | 0.527 | 0.494 | High precision on exact SKUs and model numbers; produces 5 zero-result queries on vocabulary gaps. |
| Pure Vector (Dense Embeddings) | 0.492 | 0.575 | Captures broad conceptual intent but loses to lexical on exact alphanumeric part queries. |
| Anvi Hybrid Fusion (RRF) | 0.537 | 0.645 | Superior overall relevance: 0 zero-result queries (100% coverage) while preserving exact match precision. |
| Anvi Hybrid + Neural Reranker | 0.569 | 0.648 | +0.035 nDCG lift via cross-encoder sequence-pair attention; strictly preserves merchandiser pins. |
7. Oracle Endeca Drop-in Feature Parity Matrix
Anvi Search maps directly to Endeca concepts, allowing automated migration of existing rule pipelines and EAC configurations:
| Endeca Concept (MDEX / EAC) | Anvi Search Architecture | Migration & Operational Impact |
|---|---|---|
| Experience Manager Pages & Triggers | Landing Pages & Nav State Triggers | 1:1 automated import; zero retraining required for business merchandisers. |
| Dimension Precedence Rules | Hierarchical Dimension Trees & Attribute Registry | Preserves category drilldown logic without writing custom middleware. |
| Match Modes (MatchAll, MatchAny, Partial) | Configurable Edismax Query Parsers | Drop-in query syntax compatibility for existing commerce frontends. |
| Merchandising Cartridges (Banners, Carousels) | Layout Zone Widgets | Maintains identical CMS slotting and promotional payload structures. |
| Stratified Ranking / Boost & Bury | Multi-Stage Hybrid RRF + Score Multipliers | Replaces rigid integer strata with explainable mathematical multipliers. |
| Forge / Dgidx Baseline Pipelines | Lane A Batch Ingestion with Parity Check | Replaces complex proprietary pipeline scripts with declarative JSON/Postgres feeds. |
| EAC Delta Updates | Lane B Real-Time CDC Streaming | Sub-20ms price and stock updates instead of hourly batch deltas. |
| In-Memory Dgraph Server | High-Throughput Lucene/Solr + Vector Core | 100% In-VPC deployment with modern multi-core parallelism and horizontal scalability. |
8. Security, Privacy & 100% In-VPC Isolation
Unlike third-party multi-tenant SaaS search providers that ingest your proprietary catalog and customer search events into external cloud environments, Anvi Search runs 100% inside your enterprise perimeter:
- Zero External Data Egress: No search queries, customer identifiers, or catalog records ever leave your private VPC or bare-metal infrastructure.
- Privacy-by-Design: Zero personal data processed by default. Behavioural analytics requires explicit configuration with keyed HMAC pseudonymization.
- Regulatory Compliance: Fully compliant with GDPR, CCPA, and India DPDP Act. Subject access and erasure (DSAR) are built-in native operations.
- AI Agent & MCP Tool Readiness: Implements standard Model Context Protocol (
GET /mcp/tools) for seamless, secure integration with enterprise AI agents.