Index of every page on this site: llms.txt
All skills
davila7 avatar

/brightdata-local-search

@053feb2
Run it yourself

Set up and run local web searches using Bright Data SERP API with the unfancy-search pipeline (query expansion, SERP retrieval, RRF reranking).

Use this Skill: https://skilld.dev/gh/davila7/claude-code-templates/brightdata-local-search

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

SKILL.md

≈42 tokens for metadata: the name and description. ≈1.2k when used: this file.

Description uses 2.1% of example budget

Before choosing a Skill, your Agent reads its name and description. All available Skills share that space.

  • A shorter description leaves more room for other Skills. This entry exceeds our 1% size suggestion.

In our Claude Code example, all Skill names and descriptions share 8,000 characters. This Skill uses ≈169 characters, or 2.1%.

The 1% threshold is a size suggestion. Longer descriptions can still fit.

Your model, settings, and other Skills decide how much text your Agent can read.

Example settings and source

The example uses a 200k-token context and default Claude Code settings. The count includes the name, description, separators, and when_to_use when present. Codex also counts local file paths.

Skit's source and limits: Codex 0.160.1, Claude Code 2.1.292.

Bright Data Local Search

Run powerful web searches locally using Bright Data's SERP API. This skill sets up the unfancy-search pipeline — a local search engine with query expansion, multi-engine SERP retrieval, RRF reranking, deduplication, and domain clustering.

Important: This skill uses the LOCAL version only. Do not use the hosted endpoint.

Prerequisites

Setup

Step 1: Clone and Configure

git clone https://github.com/yaronbeen/unfancy-search.git
cd unfancy-search
cp .env.example .env

Step 2: Set Environment Variables

Edit .env with your credentials:

BRIGHT_DATA_API_TOKEN=your_brightdata_token
BRIGHT_DATA_SERP_ZONE=serp_api1
ANTHROPIC_API_KEY=your_anthropic_key   # Optional: enables AI query expansion

Get your Bright Data token from: https://brightdata.com (SERP API section)

Step 3: Start the Local Server

Docker (recommended):

docker compose up -d
# Server runs at http://localhost:3000

Node.js:

pnpm install
pnpm dev
# Server runs at http://localhost:3000

API Endpoints

All requests go to http://localhost:3000:

Endpoint Method Description
/api/search POST Start a search job
/api/search-status/{jobId} GET Poll for results
/api/baseline POST Trigger baseline collection
/api/baseline-status/{id} GET Poll baseline progress

Running a Search

Step 1: Submit Search

curl -X POST http://localhost:3000/api/search \
  -H "Content-Type: application/json" \
  -d '{"query": "your search term"}'

The response returns a jobId.

Step 2: Poll for Results

curl http://localhost:3000/api/search-status/{jobId}

Poll every 3 seconds until status is "done".

Search Parameters

Parameter Type Default Description
query string required Search query
expand boolean false Enable AI query expansion via Claude
research boolean false Research mode (12 sub-queries for max coverage)
engines string[] all SERP engines to use
geo string — Geographic region filter
count number 10 Max results (up to 10)
includeDomains string[] — Only include results from these domains
excludeDomains string[] — Exclude results from these domains

Search Modes

  • Basic (expand: false): Single query, fastest, no AI cost
  • Expanded (expand: true): Claude Haiku generates 3 sub-queries for broader coverage
  • Research (research: true): 12 sub-queries for maximum coverage

Usage Examples

Basic Search from an Agent

# Start search
JOB_ID=$(curl -s -X POST http://localhost:3000/api/search \
  -H "Content-Type: application/json" \
  -d '{"query": "best practices for API rate limiting"}' | jq -r '.jobId')

# Poll until done
while true; do
  RESULT=$(curl -s http://localhost:3000/api/search-status/$JOB_ID)
  STATUS=$(echo $RESULT | jq -r '.status')
  if [ "$STATUS" = "done" ]; then
    echo $RESULT | jq '.results'
    break
  fi
  sleep 3
done

Research Mode with Domain Filtering

curl -X POST http://localhost:3000/api/search \
  -H "Content-Type: application/json" \
  -d '{
    "query": "kubernetes scaling strategies",
    "research": true,
    "excludeDomains": ["pinterest.com", "quora.com"]
  }'

Adding Search to an Existing Agent

To give your Claude Code agent search capabilities:

  1. Ensure the local server is running (docker compose up -d in the unfancy-search directory)
  2. Your agent can use curl or fetch to query http://localhost:3000/api/search
  3. Parse the ranked results to ground responses with real web data

Response Format

Results include:

  • Ranked URLs with RRF scores
  • Domain clustering (grouped by source)
  • Cost transparency (per-search expense breakdown)
  • Raw and unique result counts
  • Search duration

Troubleshooting

Issue Solution
Server won't start Verify Docker is running or Node.js 18+ installed
No results returned Check BRIGHT_DATA_API_TOKEN is valid and SERP API zone is active
Query expansion not working Verify ANTHROPIC_API_KEY is set and valid
Slow responses Disable expand mode for faster single-query searches
Port 3000 in use Stop other services or modify the port in docker-compose.yml

Source: SKILL.md on GitHub

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

3 warnings2mo3 checks · Risk MEDIUM
  • Gen Agent Trust Hub2mo

    The skill sets up a local search pipeline by cloning a third-party GitHub repository. It handles sensitive API keys (Bright Data and Anthropic) via standard .env files and interacts with a local API. Security considerations include the download and execution of external code and the processing of untrusted web search results.

  • Socket2mo

    1 alert: gptAnomaly

  • Snyk2mo

    Risk: MEDIUM · 2 issues

Signed by skilld at 053feb2. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub yesterday.

Activeupdated 8 months ago

README badge

README badge for davila7/claude-code-templates/brightdata-local-search