Index of every page on this site: llms.txt
All skills
jeremylongshore avatar

/apify-core-workflow-a

@820dfc8
Run it yourself

Build a complete web scraping Actor with Crawlee and deploy to Apify. Use when you need end-to-end web scraping on Apify: defining an input schema, building a router-based Crawlee crawler, extracting structured data, storing results in a dataset, testing locally, and deploying the Actor to the platform. Trigger with "apify scrape website", "build apify actor", "crawlee scraper", "apify main workflow".

Use this Skill: https://skilld.dev/gh/jeremylongshore/tons-of-skills-marketplace/apify-core-workflow-a

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

referencesexamples.md

≈498 tokens on demand. Your agent reads this file only when SKILL.md points to it.

Apify Core Workflow A — Worked Examples

Concrete end-to-end runs that chain the workflow steps. Each example assumes the Actor from implementation.md is in place.

Example 1: Scrape a product catalog locally, then deploy

# 1. Seed a local test input (5 products, one listing URL)
mkdir -p storage/key_value_stores/default
echo '{"startUrls":[{"url":"https://example-store.com/products"}],"maxItems":5}' \
  > storage/key_value_stores/default/INPUT.json

# 2. Run the Actor locally against that input
apify run

# 3. Inspect the scraped dataset and the run summary
ls storage/datasets/default/
cat storage/key_value_stores/default/SUMMARY.json

# 4. Once results look right, deploy to the platform
apify push

Expected SUMMARY.json after a clean run:

{
  "itemCount": 5,
  "finishedAt": "2026-07-17T14:12:03.000Z",
  "startUrls": ["https://example-store.com/products"]
}

Example 2: Run the deployed Actor and export results as CSV

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });

const run = await client.actor('username/my-actor').call({
  startUrls: [{ url: 'https://target-store.com/products' }],
  maxItems: 500,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(`Scraped ${items.length} products`);

const csv = await client.dataset(run.defaultDatasetId).downloadItems('csv');
// Persist `csv` to disk or pipe it into a downstream ETL job.

Example 3: Route through Apify residential proxy for anti-bot sites

When a target blocks datacenter IPs, pass a residential proxy group in the run input — the Actor already reads proxyConfig and wires it into the crawler:

const run = await client.actor('username/my-actor').call({
  startUrls: [{ url: 'https://protected-store.com/products' }],
  maxItems: 200,
  proxyConfig: { useApifyProxy: true, groups: ['RESIDENTIAL'] },
});

Source: SKILL.md on GitHub

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

1 warning2mo3 checks · Risk SAFE
  • Gen Agent Trust Hub2mo

    This skill provides a standard workflow for developing and deploying web scrapers using Crawlee and the Apify platform. It correctly handles sensitive API tokens by recommending environment variables. There is an inherent risk of indirect prompt injection from external web content being scraped, which is common for this type of tool.

  • Socket2mo

    No alerts

  • Snyk2mo

    Risk: MEDIUM · 1 issue

Signed by skilld at 820dfc8. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub yesterday.

Activeupdated 4 months ago
What it can do
Reads files Edits files Runs commands
version
1.5.0
author
Jeremy Longshore <jeremy@intentsolutions.io>
tags
[
  "saas",
  "scraping",
  "automation",
  "apify"
]
All 7 allowed tools
ReadWriteEditBash(npm:*)Bash(npx:*)Bash(apify:*)Grep
Other metadata
compatibility
Designed for Claude Code

README badge

README badge for jeremylongshore/tons-of-skills-marketplace/apify-core-workflow-a