Dense Vector Models Guide
This document lists model options for Dense Vector Search in OpenSearch, categorized by deployment mode, with practical recommendations.
Key takeaways:
- OpenSearch node (CPU) pretrained models tend to be older baselines: convenient for quick starts, but not SOTA for retrieval quality.
- Default recommendation for most users: Amazon Titan Embeddings (via Amazon Bedrock) for strong quality + managed ops.
- External Embedding API Services: OpenSearch can work with any embedding service via ML Commons Connectors; the list below is just common examples.
1. OpenSearch Node Deployment (CPU)
Deploy models directly on OpenSearch nodes using CPU inference.
When to use
- Dev / POC / low QPS workloads
- Environments where you cannot run GPU endpoints
- You prioritize simplicity over best retrieval quality
Caveat
- The pretrained models available on OpenSearch nodes are generally older and may not match the quality of newer retrieval-optimized models (e.g., E5/BGE or vendor-managed models like Titan).
1.1 Supported Pre-trained Models (examples)
OpenSearch provides a repository of pre-trained models that can be registered directly.
| Model Name | Dimensions | Description | Size | Latency (Approx) |
|---|---|---|---|---|
huggingface/sentence-transformers/all-MiniLM-L6-v2 |
384 | Good speed/quality tradeoff for English. | 22M | Low (5–15ms) |
huggingface/sentence-transformers/all-mpnet-base-v2 |
768 | Often higher quality than MiniLM, slower. | 110M | Medium (20–50ms) |
huggingface/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 |
384 | Multilingual baseline for many languages. | 120M | Medium (10–30ms) |
huggingface/sentence-transformers/multi-qa-MiniLM-L6-cos-v1 |
384 | Tuned for QA-style semantic search. | 22M | Low (5–15ms) |
1.2 Custom Models
Not Supported. Custom or fine-tuned dense embedding models cannot be deployed on OpenSearch Nodes. You must use a SageMaker GPU Endpoint.
2. SageMaker GPU Endpoint (Recommended for Custom / High-QPS)
Deploy models on AWS SageMaker with GPU acceleration for high throughput and low latency. This is the recommended approach for:
- High QPS / large batch ingestion
- Larger or retrieval-optimized models (E5/BGE family, etc.)
- Custom/fine-tuned models and custom inference logic
2.1 Recommended Models (examples)
Any model compatible with Hugging Face Text Embeddings Inference (TEI) or a custom SageMaker inference script can be used.
| Model Name | Dimensions | Description | Recommended Instance |
|---|---|---|---|
intfloat/e5-base-v2 |
768 | Strong retrieval performance; widely used. | ml.g5.xlarge |
intfloat/multilingual-e5-base |
768 | Strong multilingual retrieval. | ml.g5.xlarge |
BAAI/bge-base-en-v1.5 |
768 | High-quality English retrieval. | ml.g5.xlarge |
BAAI/bge-m3 |
1024 | Multilingual + multi-granularity; heavier. | ml.g5.xlarge |
2.2 Custom Models
If you have a custom or fine-tuned dense embedding model, deploy it using a SageMaker GPU Endpoint. This mode supports custom model weights and custom inference logic that you control.
3. External Embedding API Services (Managed Providers)
Use managed API services to generate embeddings. OpenSearch connects via the ML Commons Connector.
Important: OpenSearch can integrate with any embedding provider/service as long as:
- You can call an HTTP endpoint from OpenSearch (or from the connector runtime),
- The service returns a numeric embedding vector,
- You can configure authentication and request/response transformation.
So the providers below are examples of common choices, not an exhaustive list.
3.1 Common Providers (Examples)
| Provider | Model Names (Examples) | Dimensions (Typical) | Notes |
|---|---|---|---|
| Amazon Bedrock (Default recommendation) | amazon.titan-embed-text-v2, cohere.embed-english-v3, cohere.embed-multilingual-v3 |
1024, 1024, 1024 | Fully managed, integrated with AWS IAM. Titan v2 supports variable dimensions. |
| OpenAI | text-embedding-3-small, text-embedding-3-large, text-embedding-ada-002 |
1536, 3072, 1536 | Widely adopted; requires API key. |
| Cohere | embed-english-v3.0, embed-multilingual-v3.0 |
1024 | Strong retrieval-focused embeddings. |
Why default recommend Amazon Titan
- Strong general-purpose embedding quality
- Fully managed + straightforward operations on AWS
- IAM-based auth and Bedrock integration reduces operational overhead
Summary of Trade-offs
| Deployment Mode | Latency | Cost | Maintenance | Scalability | Best For |
|---|---|---|---|---|---|
| OpenSearch node (CPU) | Medium/High | Low (shared) | Medium | Limited by cluster | Dev/POC, low QPS, simple setups |
| SageMaker (GPU) | Low | High (dedicated) | Low/Medium | High | Production ingestion + high QPS + custom models |
| External API | Medium/High (network) | Usage-based | Very Low | High | Fast rollout, managed quality, minimal ops |
Practical Tips (Common Gotchas)
- Dimensions must match your index mapping (
knn_vectordimension). - If your model recommends normalization (common for cosine similarity), apply it consistently at ingestion and query time.
- For E5/BGE-style retrieval models, follow their recommended query/document formatting (e.g., prefixes) for best results.
