skilld
All skills
apify avatar

/apify-ultimate-scraper

@97047d3 official
by apifyapify/agent-skills2.4k stars
259

Universal AI-powered web scraper for any platform. Scrape data from Instagram, Facebook, TikTok, YouTube, LinkedIn, X/Twitter, Google Maps, Google Search, Google Trends, Reddit, Airbnb, Yelp, and 15+ more platforms. Use for lead generation, brand monitoring, competitor analysis, influencer discovery, trend research, content analytics, audience analysis, review analysis, SEO intelligence, recruitment, or any data extraction task.

Ask your Agent

Use this Skill: https://skilld.dev/gh/apify/agent-skills/apify-ultimate-scraper

This session only. Nothing lands on disk.

referencesworkflowsecommerce-price-monitoring.md

≈1.1k tokens on demand. Your agent reads this file only when SKILL.md points to it.

E-commerce price monitoring workflows

Competitor product price monitoring with alerts

When: User wants to track competitor prices across product pages and get notified when prices change.

Pipeline

  1. Scrape product pages -> apify/e-commerce-scraping-tool
    • Key input: startUrls (competitor product page URLs), proxyConfiguration
  2. Match products across sites -> tri_angle/e-commerce-product-matching-tool
    • Pipe: results[].url + results[].name -> matching input
    • Key input: source dataset ID from step 1, target product list
  3. Compare vs. baseline (n8n logic: read Google Sheets last_price, compute % change, filter if changed)
  4. Alert via Telegram/Slack node with price delta

Output fields

Step 1: price, currency, name, sku, availability, url Step 2: matched pairs with similarityScore, sourceProduct, targetProduct

Cost estimate

apify/e-commerce-scraping-tool is pay-per-result. For 200 product URLs daily, expect ~$0.50-$1/run depending on site complexity.

Gotcha

Many e-commerce sites use bot protection. If e-commerce-scraping-tool returns empty prices, fall back to apify/camoufox-scraper with residential proxy. Set sessionPoolName to reuse sessions and reduce blocks.

Amazon product and review tracking

When: User wants to monitor own or competitor Amazon listings for price drops or review score changes.

Pipeline

  1. Extract Amazon data -> apify/e-commerce-scraping-tool
    • Key input: startUrls (Amazon product URLs), extractReviews (bool)
  2. Compare vs. stored baseline (n8n: read last values from Sheets or DB)
  3. Alert on new low price or rating drop (n8n: If node + Telegram/Slack send)

Output fields

price, currency, rating, reviewsCount, title, asin, availability

Cost estimate

Flat per-result pricing. 50 ASINs daily ~ $0.10-$0.25/run.

Gotcha

Amazon aggressively rotates prices and sometimes shows regional prices. Always store currency alongside price. For review text (not just counts), search apify actors search "amazon reviews" --user-agent apify-agent-skills/apify-ultimate-scraper for a dedicated Actor.

Supplier catalog extraction to draft products

When: User wants to pull new products from a supplier portal and create draft listings with AI-enriched descriptions.

Pipeline

  1. Crawl supplier catalog -> apify/playwright-scraper (JS-heavy portals) or apify/cheerio-scraper (static HTML)
    • Key input: startUrls (supplier category pages), pseudoUrls (product URL patterns), maxCrawlPages
  2. Extract product content -> apify/website-content-crawler (optional second pass for detail pages)
    • Pipe: results[].url -> startUrls
    • Key input: maxCrawlPages (1 per product), htmlTransformer: "readableText"
  3. AI rewrite (n8n: OpenAI node generates SEO title + bullets from raw specs)
  4. Create draft product (n8n: Shopify node POST /products.json with status: "draft")

Output fields

Step 1/2: text, url, metadata.title, inline image URLs

Cost estimate

Depends on catalog size. playwright-scraper is PPE; 500 product pages ~ $1-3.

Gotcha

Supplier portals often require login. Use apify/playwright-scraper with initialCookies or a pre-login script in preNavigationHooks. Never hardcode credentials - pass via Actor input from n8n credentials store.

Multi-site deal and coupon monitoring

When: User wants to detect when competitors run promotions or publish coupon codes so marketing can respond.

Pipeline

  1. Scrape deals pages -> apify/e-commerce-scraping-tool
    • Key input: startUrls (competitor deal/sale page URLs), proxyConfiguration
  2. Dynamic JS deal pages (fallback) -> apify/camoufox-scraper
    • Pipe: failed URLs from step 1 -> startUrls
  3. AI extract promotion details (n8n: OpenAI node extracts discount %, promo code, expiry from raw text)
  4. Dedup and alert (n8n: compare against stored deals DB, Slack notify on new deals)

Output fields

Raw: price, discountText, url; AI-extracted: promoCode, validUntil, discountPercent, category

Cost estimate

Light scraping - deals pages are few. Expect < $0.20/run for 20 competitor pages.

Gotcha

Promo codes and flash deals may only be visible after login or in geofenced regions. Test each target URL manually first. AI extraction of expiry dates is unreliable - treat as best-effort signal, not exact data.

Source: SKILL.md on GitHub

Third-party checks

1 alert16d5 checks · Risk SAFE
  • Gen Agent Trust Hub16d

    The skill provides comprehensive instructions for orchestrating and interacting with web scrapers (Actors) via the Apify CLI. It provides structured playbooks for different data extraction workflows (B2B lead generation, brand monitoring, social media analytics, etc.) and contains standard guidelines on configuration, parameter passing, and error handling. No malicious behaviors, obfuscation techniques, or hidden actions were detected.

  • Socket16d

    No alerts

  • Snyk16d

    Risk: MEDIUM · 1 issue

  • Runlayer7mo

    2/2 files flagged

  • ZeroLeaks5mo

    Score: 93/100 · 2 sections analyzed

Provenance

Signed by skilld at 97047d3. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub yesterday.

Activeupdated 3 months ago

Topics

  • SEO
  • apify
  • scraping
  • web-scraping
  • instagram
  • facebook
  • tiktok
  • youtube
  • linkedin
  • twitter
  • google-maps
  • reddit
  • lead-generation
  • competitor-analysis
  • brand-monitoring

README badge

README badge for apify/agent-skills/apify-ultimate-scraper

What it does

Scrapes data from 100+ Apify Actors covering Instagram, Facebook, TikTok, YouTube, LinkedIn, X/Twitter, Google Maps, Reddit, Airbnb, Yelp, and 15+ other platforms via the Apify CLI. Use for lead generation, competitor analysis, brand monitoring, influencer discovery, review analysis, or SEO intelligence on any public web platform.

Generated from the current SKILL.md.

Frequently asked

Does this skill work with all platforms?
It covers ~100 Actors across 15+ platforms including Instagram, Facebook, TikTok, YouTube, LinkedIn, X/Twitter, Google Maps, Reddit, Airbnb, Yelp, and more. Not all platforms are equally supported; check the actor-index.md reference for your target platform.
What authentication is required?
You need an Apify account and CLI token. Authenticate via `apify login` (OAuth), set `APIFY_TOKEN` as an environment variable, or source from a .env file.
Do I need to install anything locally?
Yes, Apify CLI v1.5.0 or later must be installed via npm (`npm install -g apify-cli`).
Can I export results in different formats?
Yes. Results can be fetched as JSON or CSV using `apify datasets get-items` with the `--format` flag.
What should I read before running a scraper for the first time?
Check `references/actor-index.md` to find the right Actor for your platform, then read `references/gotchas.md` for common pitfalls specific to that Actor before running.

Generated from the current SKILL.md. These answers refresh after source changes.