All skills
apify avatar

/apify-ultimate-scraper

@97047d3 official
by apifyapify/agent-skills2.4k stars
259

Universal AI-powered web scraper for any platform. Scrape data from Instagram, Facebook, TikTok, YouTube, LinkedIn, X/Twitter, Google Maps, Google Search, Google Trends, Reddit, Airbnb, Yelp, and 15+ more platforms. Use for lead generation, brand monitoring, competitor analysis, influencer discovery, trend research, content analytics, audience analysis, review analysis, SEO intelligence, recruitment, or any data extraction task.

Use this Skill: https://skilld.dev/gh/apify/agent-skills/apify-ultimate-scraper

This session only. Nothing lands on disk.

referencesworkflowsjob-market-and-recruitment.md

≈902 tokens on demand. Your agent reads this file only when SKILL.md points to it.

Job market and recruitment workflows

Job listing research

When: User wants to find and analyze job postings by role, company, or location.

Pipeline

  1. Search jobs -> harvestapi/linkedin-job-search
    • Key input: keyword, location, datePosted, limit
  2. Get job details -> apimaestro/linkedin-job-detail
    • Pipe: results[].jobUrl -> urls
    • Key input: urls

Output fields

Step 1: title, company, location, jobUrl, postedDate, applicantsCount Step 2: description, requirements, seniority, employmentType, salary

Gotcha

Both Actors are PPE. Step 1: ~$0.001/job. Step 2: ~$0.005/job. For 200 jobs, total ~$1.20. Estimate and confirm with user.

Candidate sourcing

When: User wants to find potential candidates matching specific criteria.

Pipeline

  1. Search profiles -> harvestapi/linkedin-profile-search
    • Key input: keyword, title, location, industry, limit
  2. Enrich with details -> apimaestro/linkedin-profile-full-sections-scraper
    • Pipe: results[].profileUrl -> urls
    • Key input: urls

Output fields

Step 1: fullName, headline, location, profileUrl, currentCompany Step 2: experience[], education[], skills[], certifications[], languages[]

Gotcha

Step 2 (apimaestro/linkedin-profile-full-sections-scraper) costs ~$0.01/profile - the most expensive LinkedIn scraper. Use sparingly for shortlisted candidates only.

Sales signal outreach - job posting as buying signal

When: User wants to monitor company job postings as a signal to identify sales opportunities - e.g., a "Head of Data Engineering" hire suggests budget for data tooling.

Pipeline

  1. Monitor target postings -> harvestapi/linkedin-job-search
    • Key input: searchUrl (LinkedIn Jobs URL with company or role filters), keywords
  2. Get company context -> harvestapi/linkedin-company
    • Pipe: results[].companyUrl -> companyUrls

Output fields

Step 1: title, companyName, description, employmentType, seniorityLevel, jobUrl Step 2: name, industry, employeeCount, description, specialties[]

Gotcha

Job descriptions contain implicit buying signals - tech stack mentions, pain points, and headcount growth. Pass description to an LLM to extract inferred tech stack and budget tier before prioritizing outreach. Contact finding (Hunter.io) uses the native n8n node, not an Apify Actor.

Upwork job monitoring for freelancers

When: User wants to continuously monitor Upwork for new jobs matching their skills.

Pipeline

  1. Scrape Upwork search -> apify/playwright-scraper
    • Key input: startUrls (Upwork search URL with skill filters), pseudoUrls, maxCrawledPages

Output fields

Step 1: title, description, budget, clientJobsPosted, clientHireRate, postedAt, url

Gotcha

No dedicated Upwork Actor exists in Apify Store - verify with apify actors search "upwork" --user-agent apify-agent-skills/apify-ultimate-scraper for community options before defaulting to apify/playwright-scraper. Upwork pages are JS-heavy so Playwright is required over basic HTTP scraping. For high-frequency monitoring (every 15 min), store seen job URLs to avoid re-processing duplicates.

GitHub contributor discovery

When: User wants to find developers who contribute to specific open-source projects.

Pipeline

  1. Get contributors -> janbuchar/github-contributors-scraper
    • Key input: repoUrls

Output fields

Step 1: username, contributions, profileUrl, avatarUrl

Source: SKILL.md on GitHub

1 alert16d5 checks · Risk SAFE
  • Gen Agent Trust Hub16d

    The skill provides comprehensive instructions for orchestrating and interacting with web scrapers (Actors) via the Apify CLI. It provides structured playbooks for different data extraction workflows (B2B lead generation, brand monitoring, social media analytics, etc.) and contains standard guidelines on configuration, parameter passing, and error handling. No malicious behaviors, obfuscation techniques, or hidden actions were detected.

  • Socket16d

    No alerts

  • Snyk16d

    Risk: MEDIUM · 1 issue

  • Runlayer7mo

    2/2 files flagged

  • ZeroLeaks5mo

    Score: 93/100 · 2 sections analyzed

Signed by skilld at 97047d3. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub yesterday.

Activeupdated 3 months ago
  • SEO
  • apify
  • scraping
  • web-scraping
  • instagram
  • facebook
  • tiktok
  • youtube
  • linkedin
  • twitter
  • google-maps
  • reddit
  • lead-generation
  • competitor-analysis
  • brand-monitoring

README badge

README badge for apify/agent-skills/apify-ultimate-scraper

Scrapes data from 100+ Apify Actors covering Instagram, Facebook, TikTok, YouTube, LinkedIn, X/Twitter, Google Maps, Reddit, Airbnb, Yelp, and 15+ other platforms via the Apify CLI. Use for lead generation, competitor analysis, brand monitoring, influencer discovery, review analysis, or SEO intelligence on any public web platform.

Generated from the current SKILL.md.

Does this skill work with all platforms?
It covers ~100 Actors across 15+ platforms including Instagram, Facebook, TikTok, YouTube, LinkedIn, X/Twitter, Google Maps, Reddit, Airbnb, Yelp, and more. Not all platforms are equally supported; check the actor-index.md reference for your target platform.
What authentication is required?
You need an Apify account and CLI token. Authenticate via `apify login` (OAuth), set `APIFY_TOKEN` as an environment variable, or source from a .env file.
Do I need to install anything locally?
Yes, Apify CLI v1.5.0 or later must be installed via npm (`npm install -g apify-cli`).
Can I export results in different formats?
Yes. Results can be fetched as JSON or CSV using `apify datasets get-items` with the `--format` flag.
What should I read before running a scraper for the first time?
Check `references/actor-index.md` to find the right Actor for your platform, then read `references/gotchas.md` for common pitfalls specific to that Actor before running.

Generated from the current SKILL.md. These answers refresh after source changes.