/amazon-opensearch-service

Search capability — entry point and methods guide

This file is the entry point for the search capability. It covers vector / semantic / hybrid / sparse / dense / RAG retrieval on Amazon OpenSearch Service or Serverless. Supports Bedrock connectors (Titan, Cohere), self-hosted embedding models, FAISS HNSW vs Lucene, ELSER alternatives, and hybrid scoring.

When to use this capability

SKILL.md routes here when the user is asking about search retrieval setup or design. Concrete triggers:

• Phrases: "semantic search", "hybrid search", "vector index", "k-NN", "build a RAG app", "Bedrock embeddings", "sparse vectors", "dense vectors", "ELSER", "neural search", "FAISS or Lucene"
• Tasks: pick an embedding model, set up a Bedrock connector, configure a vector index, design hybrid scoring, evaluate retrieval quality, troubleshoot relevance

All search files (capability index)

After loading this entry, you can discover every search-capability file from this list.

Cross-cutting refs you may also load: vector-knn.md (vector sizing math, k-NN engines), sizing.md, security.md.

Vector / k-NN target shape

• Serverless NextGen Vector Search collections use a simplified API — no engine/mode selection (system auto-picks); supports custom document IDs and 32x compression by default.
• Serverless Classic Vector Search collections require explicit engine: faiss; Lucene/IVF/PQ are NOT supported on Classic Serverless.
• Managed Domain supports all engines: Lucene, FAISS HNSW, FAISS IVF, FAISS PQ.
• NMSLIB is removed in OS 3.x. For the engine-by-engine breakdown, see vector-knn.md.

Sizing-related universal rules (apply when this capability sizes a vector index)

• Current-generation instances. Default to Graviton (r7g/r8g for memory-optimized; m7g/m8g for cluster managers). r6g/r6gd only with explicit justification.
• Input honesty. When sizing on UNKNOWN inputs, lead with [BLOCKER — need input] OR present 2–3 tiered bands. Never present a single point estimate built on invented numbers.

Cross-capability handoff

• For provisioning the underlying domain or collection: see provisioning-reference.md.
• For migrating an existing search workload into AOS: see assessment-workflow.md.
• For post-deploy log analytics on the same domain: see log-analytics-guide.md.
• For embedding model selection beyond Bedrock: see search-dense-vector-models.md and search-sparse-vector-models.md.

1. BM25 (Lexical Search)

1.1 Overview

BM25 is the default ranking algorithm in OpenSearch. It calculates relevance based on term frequency (TF), inverse document frequency (IDF), and document length normalization.

1.2 Accuracy Characteristics

Strengths:

• Perfect for exact keyword matching
• Handles rare/domain-specific terminology well
• No vocabulary mismatch between query and index

Weaknesses:

• Cannot understand semantic meaning
• Fails on synonyms (e.g., "car" vs "automobile")
• Language-dependent (requires language-specific analyzers)

1.3 Cost Profile

Storage Estimation:

Index Size ≈ Raw Text Size × 0.1 to 0.3
Example: 1GB text → 100-300MB index

Scaling Behavior:

• Cost&Latency grows sub-linearly with data size
• Horizontal scaling is straightforward
• Query complexity significantly affects latency

1.5 Unique Features & Query Types

BM25 supports several special query types that vector search cannot:

1.6 Language Support

1.7 When to Use BM25

Recommended:

• Exact keyword/phrase search requirements
• Autocomplete and typeahead features
• Domain-specific terminology search
• Regex or wildcard pattern matching
• Maximum cost efficiency required
• Low-latency requirements at any scale

Not Recommended:

• Semantic similarity search
• Cross-lingual search
• Synonym handling without manual configuration
• User queries differ significantly from document terminology

2. Dense Vector Search

2.1 Overview

Dense vector search uses neural network embeddings to represent text as dense floating-point vectors (typically 384-1536 dimensions). Similarity is computed using cosine similarity, dot product, or L2 distance.

2.2 Accuracy Characteristics

Strengths:

• Understands semantic meaning
• Handles paraphrases and synonyms naturally
• Supports cross-lingual search with multilingual models
• Zero-shot transfer to new domains

Weaknesses:

• May miss exact keyword matches
• Requires embedding model
• Higher computational cost
• Quality depends heavily on embedding model choice

2.3 Index Algorithms (Core Structure)

2.3.1 HNSW (Hierarchical Navigable Small World)

Overview: Graph-based approximate nearest neighbor (ANN) algorithm. Default and most popular choice.

Memory Estimation (Raw):

Memory = num_vectors × (dimensions × 4 bytes + m × 8 bytes + overhead)
Example: 10M vectors × 768 dims, m=16
Memory ≈ 10M × (768 × 4 + 16 × 8) ≈ 32GB

Best For:

• Small to medium datasets that fit in memory
• Low-latency requirements
• High accuracy requirements

2.3.2 IVF (Inverted File Index)

Overview: Clustering-based approach that partitions vectors into clusters (buckets).

Best For:

• Larger datasets where memory is constrained
• Can tolerate slightly lower accuracy
• Batch search workloads

2.3.3 Disk-based Vector Search (mode: on_disk)

Overview: OpenSearch's solution for billion-scale vector search with limited memory (requires OpenSearch 2.17+). Uses Binary Quantization (BQ) to keep a compressed index in memory while storing full-precision vectors on disk.

Memory Estimation:

Memory = num_vectors × dimensions / 8 (bits to bytes) + HNSW graph overhead
Example: 1B vectors × 768 dims (using BQ)
Memory ≈ 1B × 96 bytes ≈ 96 GB (manageable on a cluster)
vs. ~3TB for float32 vectors

Best For:

• Billion-scale datasets
• Cost-efficiency (trading RAM for SSD)
• High-throughput scenarios where RAM is the bottleneck

2.4 Compression & Quantization

2.4.1 Product Quantization (PQ)

Compression technique that breaks vectors into sub-vectors and encodes them.

2.4.2 Binary Quantization (BQ)

Extreme compression using 1-bit representations.

2.5 Total Latency Composition

Total Latency = Embedding Inference Time + Vector Search Time (KNN)

1. Embedding Inference: 5-200ms depending on deployment (API vs GPU vs CPU)
2. Vector Search (KNN): 1-100ms depending on algorithm

Critical Note: Often, inference time dominates the total latency.

2.6 Language Support

2.7 When to Use Dense Vector

Recommended:

• Semantic similarity search
• Cross-lingual search requirements
• Synonym and paraphrase handling needed
• Natural language queries from users
• Question-answering systems
• RAG (Retrieval Augmented Generation) applications

Not Recommended:

• Exact keyword matching is critical
• Highly specialized domain vocabulary not covered by model
• Extremely cost-sensitive deployments
• Real-time autocomplete/typeahead
• Sub-millisecond latency requirements

3. Sparse Vector Search

3.1 Overview

Sparse vector search uses learned sparse representations where most dimensions are zero. Unlike dense vectors with 384-1536 dimensions all populated, sparse vectors may have 30,000+ dimensions but only 100-500 non-zero values.

3.2 How Neural Sparse Works

Uses neural networks to learn sparse representations with semantic meaning:

1. Documents and queries are encoded into sparse vectors
2. Each dimension corresponds to a vocabulary token
3. Weights indicate semantic importance (not just term frequency)

Advantages over BM25:

• Learns semantic term expansion (e.g., "dog" activates "puppy", "canine")
• Trained on relevance signals
• Better zero-shot domain transfer

3.3 Search Modes: Doc-only (Recommended) vs Bi-encoder

3.3.1 Doc-only Mode (Recommended)

• Ingestion: Documents encoded using a specialized doc-only model
• Search: Query processed using a simple tokenizer (not full model inference)

Why recommended: Zero query inference, low latency (10x+ faster), lower cost.

3.3.2 Bi-encoder Mode

• Both documents and queries processed by the same deep neural network
• Higher relevance but higher latency

3.4 Index Backends

3.4.1 rank_features Field (Inverted Index Based)

• Exact search (no approximation)
• Best for smaller datasets (< 50M documents)

3.4.2 SEISMIC (ANN-based Sparse Search)

• Approximate nearest neighbor for sparse vectors
• Best for large-scale datasets (> 10M documents) with latency sensitivity

3.5 Accuracy Characteristics

3.6 When to Use Sparse Vector

Recommended:

• Balance between lexical and semantic search
• Users want semantic search without query-time model inference
• Extreme fast semantic search (doc-only + SEISMIC)
• Interpretability is important
• Lower memory budget than dense vectors

Not Recommended:

• Cross-lingual search
• Maximum semantic understanding needed

4. Hybrid Search

4.1 Overview

Hybrid search combines multiple retrieval methods (BM25, dense vector, sparse vector) to leverage the strengths of each. OpenSearch supports hybrid search through the hybrid query type and score normalization.

4.2 Score Normalization

OpenSearch provides several normalization techniques (Min-Max, L2, Harmonic Mean, etc.) to ensure scores are comparable before combination.

4.3 Combination Strategy for Relevance

• Hybrid Scope Rule: Use at most two retrieval methods per hybrid query.

• Recommended Combinations:

  • Dense + Sparse: Best search relevance. Two layers of semantic understanding.
  • Dense + BM25: Robust baseline combining semantic understanding with exact keyword precision.

• Not Recommended:

  • Sparse + BM25: Generally redundant. Sparse vectors already capture keyword information.

4.4 When to Use Hybrid Search

Recommended:

• Maximum Relevance: When accuracy and recall are the top priorities.
• Mixed query types (some exact, some semantic).
• Unknown query distribution.
• Can afford additional infrastructure cost.

Not Recommended:

• Strict cost constraints
• Simple use cases where one method suffices
• Sub-10ms latency requirements
• Development/prototype phase (start simple)

5. Agentic Search (Natural-Language Query Translation)

5.1 Overview

Agentic search uses an LLM to translate a natural-language question into OpenSearch DSL, then executes it — the caller asks a question in plain language instead of authoring a query. It is a standalone retrieval method that returns synthesized answers, implemented as a natural-language query-authoring layer over BM25/dense/hybrid retrieval.

5.2 When to Use Agentic Search

Recommended:

• Callers who cannot author DSL/PPL (business users, natural-language front-ends).
• Ad-hoc exploratory questions over a well-mapped index.

Not Recommended:

• Latency-sensitive paths, because each query adds an LLM round-trip (typically 200–2000 ms) plus per-query model cost on top of retrieval.
• High-QPS production traffic with fixed query shapes — author the DSL once instead, because a static query avoids the per-call LLM cost.

5.3 Conversational vs Flow

• Conversational (AOS domain, stateful): multi-turn with conversation memory + RAG. Setup: provisioning-agentic-setup.md.
• Flow (AOSS Serverless, stateless): each query planned independently, lower latency/cost, no memory. Setup: provisioning-serverless-agentic-setup.md.

5.4 Security Considerations

• Field exposure: the LLM can generate DSL that surfaces any field in the index, including PII. Apply field-level access control (document/field security) so an agentic query cannot return fields the caller is not authorized to see.
• Cost abuse: each agentic query incurs a Bedrock model invocation. Throttle/rate-limit the agentic path to prevent runaway per-call model cost.
• Logging & monitoring: enable CloudTrail for _plugins/_ml model/agent calls and set CloudWatch alarms on agentic query error rates and unusual invocation volume. See the detailed monitoring guidance in provisioning-agentic-setup.md and provisioning-serverless-agentic-setup.md.
• Log confidentiality: when agentic_context is enabled for debugging, the generated DSL surfaces in the response metadata (and thus in any logs that capture full responses). Encrypt any CloudWatch log group that receives agentic responses with a customer-managed KMS key and restrict its access policy, because those logs can contain the same sensitive field values the query returned.
• AWS WAF (defense in depth): if the endpoint is internet-facing, front it with AWS WAF using request-size limits and rate-based rules, because WAF filters oversized and abusive free-form input at the edge before it reaches the LLM query planner.
• For setup-time security guidance (IAM scoping, model/agent access policies), see the security sections in provisioning-agentic-setup.md (conversational) and provisioning-serverless-agentic-setup.md (flow).