Index of every page on this site: llms.txt
All skills
github avatar

/diagnosis

@f10fdb8 official
by githubgithub/awesome-copilot41k stars
5,399
Run it yourself

Diagnoses Qdrant search quality issues. Use when someone reports 'results are bad', 'wrong results', 'not relevant results', 'missing matches', 'recall is low', 'approximate search worse than exact', 'which embedding model', or 'quality dropped after quantization'. Also use when search quality degrades without obvious changes.

  • 1 file
  • 3.6 KB
  • Updated 7 months ago
  • GitHub
Use this Skill: https://skilld.dev/gh/github/awesome-copilot/diagnosis

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

SKILL.md

≈85 tokens for metadata: the name and description. ≈830 when used: this file.

Description uses 4.5% of example budget

Before choosing a Skill, your Agent reads its name and description. All available Skills share that space.

  • A shorter description leaves more room for other Skills. This entry exceeds our 1% size suggestion.

In our Claude Code example, all Skill names and descriptions share 8,000 characters. This Skill uses ≈362 characters, or 4.5%.

The 1% threshold is a size suggestion. Longer descriptions can still fit.

Your model, settings, and other Skills decide how much text your Agent can read.

Example settings and source

The example uses a 200k-token context and default Claude Code settings. The count includes the name, description, separators, and when_to_use when present. Codex also counts local file paths.

Skit's source and limits: Codex 0.160.1, Claude Code 2.1.292.

How to Diagnose Bad Search Quality

Before tuning, establish baselines. Use exact KNN as ground truth, compare against approximate HNSW. Target >95% recall@K for production.

Don't Know What's Wrong Yet

Use when: results are irrelevant or missing expected matches and you need to isolate the cause.

  • Test with exact=true to bypass HNSW approximation Search API
  • Exact search bad = model or search pipeline problem. Exact good, approximate bad = tune HNSW.
  • Check if quantization degrades quality (compare with and without)
  • Check if filters are too restrictive (then you might need to use ACORN)
  • If duplicate results from chunked documents, use Grouping API to deduplicate Grouping

Payload filtering and sparse vector search are different things. Metadata (dates, categories, tags) goes in payload for filtering. Text content goes in sparse vectors for search.

Approximate Search Worse Than Exact

Use when: exact search returns good results but HNSW approximation misses them.

Binary quantization requires rescore. Without it, quality loss is severe. Use oversampling (3-5x minimum for binary) to recover recall. Always test quantization impact on your data before production. Quantization

Wrong Embedding Model

Use when: exact search also returns bad results.

Test top 3 MTEB models on 100-1000 sample queries, measure recall@10. Domain-specific models often outperform general models. Hosted inference

Unoptimized Search Pipeline

Use when: exact search also returns bad results and model choice is confirmed by user.

Optimize search according to advanced search-strategies skill.

What NOT to Do

  • Tune Qdrant before verifying the model is right for the task (most quality issues are model issues)
  • Use binary quantization without rescore (severe quality loss)
  • Set hnsw_ef lower than results requested (guaranteed bad recall)
  • Skip payload indexes on filtered fields then blame quality (HNSW can't traverse filtered-out nodes, and filterable HNSW is built only if payload indexes were set up prior)
  • Deploy without baseline recall or other search relevance metrics (no way to measure regressions)
  • Confuse payload filtering with sparse vector search (different things, different config)

Source: SKILL.md on GitHub

No rule matched.

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

No third-party reports yet.

Signed by skilld at f10fdb8. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub yesterday.

Activeupdated 7 months ago
  • Debugging
  • qdrant
  • search-quality
  • vector-search
  • hnsw
  • embedding-models
  • quantization
  • recall
  • retrieval

README badge

README badge for github/awesome-copilot/diagnosis

Diagnoses Qdrant vector search quality issues by testing exact KNN versus approximate HNSW, checking embedding model fit, and tuning HNSW hyperparameters and quantization settings. Targets scenarios where search results are missing expected matches, returning irrelevant results, or degrading after configuration changes.

Generated from the current SKILL.md.

Does this skill help diagnose Qdrant search quality issues?
Yes. Use it when search results are irrelevant, missing expected matches, or recall drops unexpectedly. The skill guides you through isolating whether the problem is the embedding model, HNSW approximation settings, quantization, filtering, or the search pipeline.
How do I know if my embedding model is the problem?
Test exact KNN search first. If exact search also returns bad results, the model is likely the issue. Compare your top 3 MTEB candidates on 100-1000 sample queries and measure recall@10; domain-specific models often outperform general ones.
What's the difference between tuning HNSW and fixing the embedding model?
If exact search is good but approximate HNSW search is bad, tune HNSW parameters (ef_construct, hnsw_ef, m). If exact search is already bad, tuning HNSW won't help—you need a better embedding model or search pipeline.
Can I use binary quantization without losing quality?
Binary quantization requires rescore and oversampling (3-5x minimum) to recover recall. Without rescore, quality loss is severe. Always test quantization impact on your data before production.
What should I measure to avoid regressions?
Establish baseline recall metrics (target >95% recall@K for production) before making changes. Use exact KNN as ground truth to measure approximate search performance and detect regressions.

Generated from the current SKILL.md. These answers refresh after source changes.