/seek

Search Design Patterns

Purpose: Core design patterns for full-text, vector, and hybrid search systems. Read when: Designing index mappings, query strategies, or ranking pipelines.

Full-Text Search Patterns

Mapping Example (Elasticsearch/OpenSearch)

Field types, analyzers, and multi-fields for language-aware search:

{
  "mappings": {
    "properties": {
      "title": {
        "type": "text",
        "analyzer": "custom_analyzer",
        "fields": {
          "keyword": { "type": "keyword" },
          "ngram": { "type": "text", "analyzer": "ngram_analyzer" }
        }
      },
      "content": { "type": "text", "analyzer": "content_analyzer" }
    }
  },
  "settings": {
    "analysis": {
      "analyzer": {
        "custom_analyzer": {
          "type": "custom",
          "tokenizer": "standard",
          "filter": ["lowercase", "synonym_filter", "stemmer"]
        }
      }
    }
  }
}

Analyzer Selection Guide

Pattern 1: Multi-Field Mapping

Use multi-fields for different analysis strategies on the same content.

{
  "title": {
    "type": "text",
    "analyzer": "standard",
    "fields": {
      "exact": { "type": "keyword" },
      "autocomplete": { "type": "text", "analyzer": "edge_ngram_analyzer" },
      "search_as_you_type": { "type": "search_as_you_type" }
    }
  }
}

When to use: Content needs keyword search, autocomplete, and exact match simultaneously.

Pattern 2: Japanese Full-Text Search

{
  "settings": {
    "analysis": {
      "tokenizer": {
        "kuromoji": {
          "type": "kuromoji_tokenizer",
          "mode": "search"
        }
      },
      "analyzer": {
        "ja_analyzer": {
          "type": "custom",
          "tokenizer": "kuromoji",
          "filter": [
            "kuromoji_baseform",
            "kuromoji_part_of_speech",
            "ja_stop",
            "kuromoji_stemmer",
            "lowercase"
          ]
        }
      }
    }
  }
}

When to use: Japanese text indexing. Always use mode: search for query-time segmentation.

Pattern 3: Synonym-Aware Search

{
  "filter": {
    "synonym_filter": {
      "type": "synonym_graph",
      "synonyms_path": "analysis/synonyms.txt",
      "updateable": true
    }
  }
}

When to use: Domain-specific terminology (e.g., "laptop" = "notebook computer").

Vector Search Patterns

Pattern 4: Matryoshka Embedding Reduction

from openai import OpenAI

client = OpenAI()
response = client.embeddings.create(
    model="text-embedding-3-large",
    input="search query",
    dimensions=256  # Reduce from 3072 to 256
)
# Trade-off: ~5% quality loss for 12x storage reduction

When to use: Storage/latency constraints outweigh marginal quality gains.

Pattern 5: Two-Phase Vector Search

-- Phase 1: Approximate kNN with pre-filter
SELECT id, embedding <=> $1::vector AS distance
FROM documents
WHERE category = ANY($2)
ORDER BY embedding <=> $1::vector
LIMIT 100;

-- Phase 2: Rerank with cross-encoder score
-- (application layer)

When to use: Filtered vector search where filter selectivity varies widely.

Pattern 6: Multi-Vector Retrieval

Store separate embeddings for title, content, and metadata. Query with field-weighted combination.

# Weighted multi-field vector search
title_results = search(query_embedding, index="title_vectors", top_k=50)
content_results = search(query_embedding, index="content_vectors", top_k=50)
combined = rrf_fusion(title_results, content_results, k=60)

When to use: Documents with semantically distinct sections (e.g., title vs body).

pgvector Configuration

-- Create vector column
ALTER TABLE documents ADD COLUMN embedding vector(1536);

-- HNSW index (recommended for most cases)
CREATE INDEX idx_documents_embedding ON documents
  USING hnsw (embedding vector_cosine_ops)
  WITH (m = 16, ef_construction = 200);

-- Query with distance
SELECT id, title, embedding <=> $1::vector AS distance
FROM documents
WHERE category = $2
ORDER BY embedding <=> $1::vector
LIMIT 20;

Hybrid Search Patterns

Reciprocal Rank Fusion (RRF)

RRF_score(d) = Σ 1 / (k + rank_i(d))

Default k = 60. Combine BM25 rank and vector rank for each document.

Hybrid Search Pipeline

Query → [BM25 Search] → Top-N₁ results (ranked by BM25)
     ↘ [Vector Search] → Top-N₂ results (ranked by similarity)
         ↓
     [Fusion Layer (RRF / Weighted)] → Combined Top-K
         ↓
     [Optional Reranker (Cross-Encoder)] → Final Top-K

Fusion Strategy Selection

Pattern 7: Elasticsearch Hybrid with RRF

{
  "retriever": {
    "rrf": {
      "retrievers": [
        {
          "standard": {
            "query": {
              "multi_match": {
                "query": "search terms",
                "fields": ["title^3", "content"]
              }
            }
          }
        },
        {
          "knn": {
            "field": "embedding",
            "query_vector_builder": {
              "text_embedding": {
                "model_id": "my-embedding-model",
                "model_text": "search terms"
              }
            },
            "k": 50,
            "num_candidates": 100
          }
        }
      ],
      "rank_constant": 60,
      "rank_window_size": 100
    }
  }
}

Pattern 8: pgvector Hybrid Search

-- BM25 via pg_search + vector via pgvector
WITH bm25 AS (
  SELECT id, ts_rank(tsv, plainto_tsquery($1)) AS bm25_score
  FROM documents
  WHERE tsv @@ plainto_tsquery($1)
  ORDER BY bm25_score DESC
  LIMIT 100
),
vector AS (
  SELECT id, 1 - (embedding <=> $2::vector) AS vector_score
  FROM documents
  ORDER BY embedding <=> $2::vector
  LIMIT 100
),
fused AS (
  SELECT COALESCE(b.id, v.id) AS id,
    COALESCE(1.0 / (60 + b.rank), 0) + COALESCE(1.0 / (60 + v.rank), 0) AS rrf_score
  FROM (SELECT id, ROW_NUMBER() OVER (ORDER BY bm25_score DESC) AS rank FROM bm25) b
  FULL OUTER JOIN (SELECT id, ROW_NUMBER() OVER (ORDER BY vector_score DESC) AS rank FROM vector) v
  ON b.id = v.id
)
SELECT * FROM fused ORDER BY rrf_score DESC LIMIT 20;

Scaling Patterns

Pattern 9: Index Sharding Strategy

Pattern 10: Search Result Caching

CACHE_STRATEGY:
  layer_1:
    type: "query result cache"
    ttl: "5m"
    invalidation: "on index update"
    scope: "exact query match"
  layer_2:
    type: "embedding cache"
    ttl: "24h"
    scope: "query text → vector"
  layer_3:
    type: "application cache (Redis)"
    ttl: "1-15m"
    scope: "user-facing search results"