Index of every page on this site: llms.txt
All skills
jeremylongshore avatar

/apify-local-dev-loop

@820dfc8
Run it yourself

Set up local Apify Actor development with the Apify CLI and Crawlee. Use when creating Actors locally, testing with the apify run command, inspecting local storage, or establishing a fast develop-test-deploy cycle before pushing to the platform. Trigger with "apify dev setup", "apify local development", "develop actor locally", "apify run local".

Use this Skill: https://skilld.dev/gh/jeremylongshore/tons-of-skills-marketplace/apify-local-dev-loop

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

SKILL.md

≈93 tokens for metadata: the name and description. ≈1.5k when used: this file. ≈1.5k more on demand in 2 files.

Description uses 4.6% of example budget

Before choosing a Skill, your Agent reads its name and description. All available Skills share that space.

  • A shorter description leaves more room for other Skills. This entry exceeds our 1% size suggestion.

In our Claude Code example, all Skill names and descriptions share 8,000 characters. This Skill uses ≈371 characters, or 4.6%.

The 1% threshold is a size suggestion. Longer descriptions can still fit.

Your model, settings, and other Skills decide how much text your Agent can read.

Example settings and source

The example uses a 200k-token context and default Claude Code settings. The count includes the name, description, separators, and when_to_use when present. Codex also counts local file paths.

Skit's source and limits: Codex 0.160.1, Claude Code 2.1.292.

Apify Local Dev Loop

Overview

Build and test Apify Actors on your local machine before deploying to the platform. The Apify CLI (apify run) emulates the platform environment locally — creating storage directories for datasets, key-value stores, and request queues — giving you a tight edit → run → inspect loop with no cloud round-trip.

Prerequisites

  • npm install -g apify-cli (global CLI)
  • apify login completed with valid token
  • Node.js 18+

Authentication

The CLI authenticates with your Apify API token. Run apify login once (it stores the token under ~/.apify/), or export APIFY_TOKEN in the shell for non-interactive use. Local runs (apify run) do not require auth — only apify push / apify call reach the platform. Never commit the token or a plaintext .env containing it.

Actor Project Structure

my-actor/
├── .actor/
│   ├── actor.json          # Actor metadata and config
│   └── INPUT_SCHEMA.json   # Input schema (auto-generates UI on platform)
├── src/
│   └── main.ts             # Entry point
├── storage/                # Created by apify run (git-ignored)
│   ├── datasets/default/
│   ├── key_value_stores/default/
│   └── request_queues/default/
├── package.json
└── tsconfig.json

Instructions

Full config files and Actor source live in implementation.md; the high-level loop is:

Step 1: Create a New Actor Project

# Create from template (interactive)
apify create my-actor

# Or create from specific template
apify create my-actor --template project_cheerio_crawler_ts
# Templates: project_empty, project_cheerio_crawler_ts,
#   project_playwright_crawler_ts, project_puppeteer_crawler_ts

Step 2: Configure and code

Read and Edit the scaffolded .actor/actor.json (metadata + optional dataset view), define .actor/INPUT_SCHEMA.json (validates input and auto-generates the platform UI), and write your crawler in src/main.ts. See implementation.md for the complete actor.json, input schema, and a Cheerio-based main.ts that reads validated input and pushes structured rows via Actor.pushData().

Step 3: Run Locally

# Run with default input from storage/key_value_stores/default/INPUT.json
apify run

# Run with input from command line
apify run --input='{"startUrls":[{"url":"https://example.com"}],"maxPages":5}'

# View results
cat storage/datasets/default/*.json | jq '.'

Step 4: Provide Local Input

Create storage/key_value_stores/default/INPUT.json so repeated apify run invocations reuse the same input:

{
  "startUrls": [{ "url": "https://example.com" }],
  "maxPages": 5
}

For the fastest inner loop, run the entry point directly with tsx watch instead of apify run — wiring and platform-emulating env vars are in implementation.md § Hot Reload Development. Unit tests that mock the SDK boundary are in that same file.

Local Storage Emulation

apify run creates a storage/ directory that mirrors platform storage:

Platform Storage Local Path Access via SDK
Default dataset storage/datasets/default/ Actor.pushData()
Default KV store storage/key_value_stores/default/ Actor.setValue() / Actor.getValue()
Default request queue storage/request_queues/default/ Managed by crawler

Output

  • A runnable Actor project scaffolded from a template (.actor/, src/, package.json)
  • A typed input schema that validates locally and generates the platform UI
  • Scraped rows written to storage/datasets/default/ as JSON files
  • A local storage/ tree mirroring platform datasets, KV stores, and request queues
  • A watch-mode dev loop (tsx watch) and a Vitest test that mocks the SDK boundary

Error Handling

Error Cause Solution
apify: command not found CLI not installed npm i -g apify-cli
INPUT.json not found No input provided Create storage/key_value_stores/default/INPUT.json
Cannot find module 'apify' SDK not installed npm install apify crawlee
Dockerfile not found Missing actor config Run apify create or create .actor/actor.json

Examples

A quick end-to-end run — seed a local input, run the Actor, and inspect results:

mkdir -p storage/key_value_stores/default
echo '{"startUrls":[{"url":"https://example.com"}],"maxPages":5}' \
  > storage/key_value_stores/default/INPUT.json
apify run
cat storage/datasets/default/*.json | jq '.'

Three fuller worked scenarios live in examples.md:

  • Scaffold a new Actor and run it locally — apify create from a template through the first apify run.
  • Provide a local input file and inspect results — persistent INPUT.json, plus the exact dataset row shape.
  • One-shot run with inline input — throwaway --input runs while iterating on selectors.

Resources

Next Steps

Once the local loop is producing clean data, move on to production-ready Actor code patterns — routing, proxy configuration, retries, and dataset shaping — covered in apify-sdk-patterns.

Source: SKILL.md on GitHub

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

No alerts2mo3 checks · Risk SAFE
  • Gen Agent Trust Hub2mo

    This skill facilitates local development for Apify Actors by setting up the official Apify CLI and environment. It allows developers to scaffold projects, emulate platform storage locally, and test crawlers without cloud dependencies. The skill follows security best practices for credential management and utilizes official tools from the Apify ecosystem.

  • Socket2mo

    No alerts

  • Snyk2mo

    Risk: LOW · No issues

Signed by skilld at 820dfc8. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub 20 hours ago.

Activeupdated 4 months ago
What it can do
Reads files Edits files Runs commands
version
1.5.0
author
Jeremy Longshore <jeremy@intentsolutions.io>
tags
[
  "saas",
  "scraping",
  "automation",
  "apify"
]
All 6 allowed tools
ReadWriteEditBash(npm:*)Bash(npx:*)Bash(apify:*)
Other metadata
compatibility
Designed for Claude Code

README badge

README badge for jeremylongshore/tons-of-skills-marketplace/apify-local-dev-loop