Index of every page on this site: llms.txt
All skills
tavily-ai avatar

/tavily-map

@778122e official
by Tavily AItavily-ai/skills488 stars
42
Run it yourself

Discover and list all URLs on a website without extracting content, via the Tavily CLI. Use this skill when the user wants to find a specific page on a large site, list all URLs, see the site structure, find where something is on a domain, or says "map the site", "find the URL for", "what pages are on", "list all pages", or "site structure". Faster than crawling — returns URLs only. Essential when you know the site but not the exact page. Combine with extract for targeted content retrieval.

Use this Skill: https://skilld.dev/gh/tavily-ai/skills/tavily-map

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

SKILL.md

≈127 tokens for metadata: the name and description. ≈779 when used: this file.

Description uses 6.3% of example budget

Before choosing a Skill, your Agent reads its name and description. All available Skills share that space.

  • A shorter description leaves more room for other Skills. This entry exceeds our 1% size suggestion.

In our Claude Code example, all Skill names and descriptions share 8,000 characters. This Skill uses ≈508 characters, or 6.3%.

The 1% threshold is a size suggestion. Longer descriptions can still fit.

Your model, settings, and other Skills decide how much text your Agent can read.

Example settings and source

The example uses a 200k-token context and default Claude Code settings. The count includes the name, description, separators, and when_to_use when present. Codex also counts local file paths.

Skit's source and limits: Codex 0.160.1, Claude Code 2.1.292.

tavily map

Discover URLs on a website without extracting content. Faster than crawling.

Before running

Map requires authentication. Run the requested command directly when tvly is already authenticated; do not add a status check to every invocation.

If tvly is missing, follow the tavily-cli setup. If an installed CLI reports an authentication error, use tvly login for authentication only, or tvly init --skip-skills when guided verification is also useful. Browser-based OAuth is preferred when an interactive user can complete it. --no-browser prints the sign-in link instead of opening it, but still waits for a localhost callback. In an unattended agent or CI environment, leave authentication to the user or use a securely provided TAVILY_API_KEY. Do not start a second login immediately after guided setup has completed.

When to use

  • You need to find a specific subpage on a large site
  • You want a list of all URLs before deciding what to extract or crawl
  • Step 3 in the workflow: search → extract → map → crawl → research

Quick start

# Discover all URLs
tvly map "https://docs.example.com" --json

# With natural language filtering
tvly map "https://docs.example.com" --instructions "Find API docs and guides" --json

# Filter by path
tvly map "https://example.com" --select-paths "/blog/.*" --limit 500 --json

# Deep map
tvly map "https://example.com" --max-depth 3 --limit 200 --json

Options

Option Description
--max-depth Levels deep (1-5, default: 1)
--max-breadth Links per page (default: 20)
--limit Max URLs to discover (default: 50)
--instructions Natural language guidance for URL filtering
--select-paths Comma-separated regex patterns to include
--exclude-paths Comma-separated regex patterns to exclude
--select-domains Comma-separated regex for domains to include
--exclude-domains Comma-separated regex for domains to exclude
--allow-external / --no-external Include external links
--timeout Max wait (10-150 seconds)
-o, --output Save the JSON response to a file
--json Structured JSON output

Map + Extract pattern

Use map to find the right page, then extract it. This is often more efficient than crawling an entire site:

# Step 1: Find the authentication docs
tvly map "https://docs.example.com" --instructions "authentication" --json

# Step 2: Extract the specific page you found
tvly extract "https://docs.example.com/api/authentication" --json

Tips

  • Map is URL discovery only — no content extraction. Use extract or crawl for content.
  • Map + extract beats crawl when you only need a few specific pages from a large site.
  • Use --instructions for semantic filtering when path patterns aren't enough.

See also

Source: SKILL.md on GitHub

Skill behaviors

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

Third-party checks

1 warning1mo4 checks · Risk SAFE
  • Gen Agent Trust Hub1mo

    The skill is a safe implementation of a website mapping tool using the Tavily CLI. It correctly restricts shell execution to the specific vendor tool and provides clear guidance for secure credential management.

  • Socket1mo

    No alerts

  • Snyk1mo

    Risk: MEDIUM · 1 issue

  • ZeroLeaks6mo

    1 finding · Score: 82/100

Provenance

Signed by skilld at 778122e. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub last week.

Steadyupdated last month

Capability

What it can do
Runs commands
All 1 allowed tools
Bash(tvly *)

Topics

  • CLI
  • tavily
  • web-scraping
  • url-discovery
  • site-mapping
  • bash

README badge

README badge for tavily-ai/skills/tavily-map

What it does

Discovers and lists all URLs on a website via the Tavily CLI without extracting content, returning structured results faster than crawling. Use this when you need to find a specific page on a large site, see the site structure, or filter URLs by path patterns and natural language instructions before targeted extraction.

Generated from the current SKILL.md.

Frequently asked

What does this skill return?
URLs only, without extracting page content. It discovers and lists all URLs on a website up to a configurable depth and breadth.
How is this different from crawling?
Map discovers URLs only and is faster than crawling. Use it to find the right pages first, then combine with extract to get content from specific URLs.
Do I need to install anything?
Yes. The skill requires the Tavily CLI (tvly). If not on PATH, run the install command provided in the skill — do not skip this step.
Can I filter URLs by patterns?
Yes. Use --select-paths and --exclude-paths for regex filtering, --instructions for natural language guidance, or domain filters like --select-domains.
How deep can the map go?
Up to 5 levels deep (--max-depth), with configurable breadth per page (--max-breadth, default 20) and a URL limit (--limit, default 50).

Generated from the current SKILL.md. These answers refresh after source changes.