---
name: apify-performance-tuning
description: 'Optimize Apify Actor performance: crawl speed, memory usage, concurrency, and proxy rotation.

  Use when Actors are slow, consuming too much memory, or being blocked by target sites, or when a crawl is too expensive per run.

  Trigger with "apify performance", "optimize apify actor", "apify slow", "crawlee concurrency", "apify memory tuning", "scraper performance".'
allowed-tools: Read, Write, Edit
version: 1.5.0
license: MIT
author: Jeremy Longshore <jeremy@intentsolutions.io>
tags:
  - saas
  - scraping
  - automation
  - apify
compatibility: Designed for Claude Code
title: apify-performance-tuning
canonical_url: https://skilld.dev/gh/jeremylongshore/tons-of-skills-marketplace/apify-performance-tuning
last_updated: 2026-10-07T06:06:28.000Z
---

> Index of every page on this site: [llms.txt](https://skilld.dev/llms.txt)

> **Skill from skilld.dev.** Follow the user's requested action.
> If the user asked to fork this Skill, follow the fork workflow below. Do not execute the Skill instructions while copying it.
> Otherwise, follow the instructions below for this session. You do not need to install anything.
>
> Supporting files, fetch one when the Skill refers to it: [references/implementation.md](https://skilld.dev/api/skills-raw/jeremylongshore/tons-of-skills-marketplace/apify-performance-tuning/references/implementation.md), [references/monitoring.md](https://skilld.dev/api/skills-raw/jeremylongshore/tons-of-skills-marketplace/apify-performance-tuning/references/monitoring.md).
>
> If the user asked to install this Skill, run `npx skilld install jeremylongshore/tons-of-skills-marketplace/apify-performance-tuning`. Install writes the Skill files into the project, so every session loads them.
>
> ## Fork workflow
>
> A fork creates an editable local Skill with its original author and licence. The request authorizes copying and local installation.
> 1. Check `./skills/apify-performance-tuning`, the project lockfile, and selected Agent targets together. If the local directory or installed Skill exists, stop. Never overwrite an existing directory or Agent target.
> 2. Read [source metadata](https://skilld.dev/api/v1/skills/jeremylongshore/tons-of-skills-marketplace/apify-performance-tuning) once. Use sourceUrl, sourceCommit, skillPath, sourceGone, and license. If the source is gone or its path is missing, stop. If license is null, read licence files at the source commit.
> 3. Fetch only the source commit into a temporary Git repository. Do not clone full history. Derive repository_url from sourceUrl, including repository renames. If sourceCommit is absent, resolve the sourceUrl ref once. Set source_commit to that actual commit. Run these commands in one shell call:
>
> ```sh
> git init --quiet "$temporary_dir"
> git -C "$temporary_dir" fetch --quiet --depth=1 "$repository_url" "$source_commit"
> git -C "$temporary_dir" checkout --quiet --detach FETCH_HEAD
> ```
>
> Read applicable licence declarations and notices at that commit. If copying is not permitted, report the restriction and stop.
> 4. Inspect source entries together, then copy the directory containing skillPath into `./skills/apify-performance-tuning`. Use the user's path if selected. Keep the original SKILL.md, relative links, scripts, binary assets, and executable modes. Exclude .git metadata. Reject symlinks and paths outside the Skill directory. After checking entries, use cp -a where available. A regular source directory needs no custom copy script. Do not save this page wrapper as SKILL.md.
> Preserve author credit, notices, and applicable licence files from repository or parent directories. Add PROVENANCE.md with the Skill page, source URL, actual commit, original path, and licence. Retain any existing PROVENANCE.md and record new provenance separately. Batch source inspection, copying, and provenance work where practical.
> 5. In the project root, run `skilld install ./skills/apify-performance-tuning --mode copy --plain`. If skilld is unavailable, use `npx skilld install ./skills/apify-performance-tuning --mode copy --plain`. This known command needs no help lookup. Install does not support --json. Use detected Agent targets, or add --agent for the targets the user selected. Install the local path, never the upstream selector. If installation fails, preserve the local copy and report the exact failure.
> 6. Confirm the local lockfile source and installed Agent copies once. Report the local path, actual commit, and Agent targets. After edits, reinstall the same local path. Upstream updates must not replace it. Do not publish or push unless the user asks.

# Apify Performance Tuning

## Overview

Optimize Apify Actors for speed, cost, and reliability. Covers Crawlee concurrency settings, memory profiling, proxy rotation strategies, request batching, and crawler selection for different workloads.

The workflow is a repeatable loop: measure a baseline, apply one lever, re-measure. The single highest-impact lever is usually crawler choice — swapping a browser crawler for `CheerioCrawler` on non-JS pages is a 5-10x speedup on its own. The full six-step walkthrough, with every code block, lives in [references/implementation.md](https://skilld.dev/api/skills-raw/jeremylongshore/tons-of-skills-marketplace/apify-performance-tuning/references/implementation.md).

## Prerequisites

- Existing Actor with measurable baseline performance
- Understanding of `apify-sdk-patterns`
- Access to Actor run stats in Apify Console
- `APIFY_TOKEN` in the environment for reading run stats via `ApifyClient`

## Instructions

Work the levers in order. Each step is expanded — with copy-paste code — in the reference file linked below.

1. **Measure a baseline.** Pull `runTimeSecs`, `requestsFinished`, `memAvgBytes`, and `usageTotalUsd` from the run stats before changing anything. You cannot judge an optimization without a before number.
2. **Choose the right crawler.** `HttpCrawler`/`CheerioCrawler` for static HTML or JSON (low memory, fast); `PlaywrightCrawler`/`PuppeteerCrawler` only when the page genuinely needs JavaScript rendering.
3. **Tune concurrency.** Raise `maxConcurrency` for Cheerio (up to ~50); keep it low (~3-5) for browser crawlers because each page costs ~200MB. Let `autoscaledPoolOptions` adjust within the band.
4. **Optimize memory.** Push data immediately instead of accumulating arrays; for browser crawlers, block images/CSS/fonts in `preNavigationHooks` and cap concurrent browsers.
5. **Right-size the memory allocation.** Compute units bill on `memory x duration` — start low (512 MB for Cheerio) and only raise it if the Actor is memory-starved.
6. **Rotate proxies and tune requests.** Start on datacenter proxies, fall back to residential on 403/blocked; use a session pool for IP rotation and ban detection.

Full step-by-step walkthrough with all code: [references/implementation.md](https://skilld.dev/api/skills-raw/jeremylongshore/tons-of-skills-marketplace/apify-performance-tuning/references/implementation.md).

The minimal starting skeleton — swap a browser crawler for Cheerio and push immediately:

```typescript
import { CheerioCrawler } from 'crawlee';
import { Actor } from 'apify';

const crawler = new CheerioCrawler({
  maxConcurrency: 50,             // Cheerio is cheap — parallelize hard
  maxRequestsPerMinute: 300,      // But cap the rate to protect the target
  requestHandler: async ({ $, request }) => {
    await Actor.pushData({ url: request.url, title: $('title').text().trim() });
  },
});
```

## Output

Applying this skill produces:

- A **baseline vs. tuned metrics** comparison (pages/min, avg/max memory, compute units, cost/run) drawn from the run stats.
- A **crawler and concurrency recommendation** matched to whether the target pages need JS rendering.
- A **memory allocation** value sized to the workload, with the compute-unit cost tradeoff made explicit.
- A **proxy and session strategy** (datacenter-first with residential fallback) for reliability under anti-bot blocking.

Instrument the running crawl to confirm the gains in real time — see [references/monitoring.md](https://skilld.dev/api/skills-raw/jeremylongshore/tons-of-skills-marketplace/apify-performance-tuning/references/monitoring.md) for the throughput logger and a before/after impact table.

## Error Handling

| Issue | Cause | Solution |
|-------|-------|----------|
| Out of memory crash | Too many concurrent browsers | Reduce `maxConcurrency` |
| Slow crawl speed | Low concurrency | Increase `maxConcurrency` |
| High failure rate | Anti-bot blocking | Add proxy, reduce concurrency |
| Expensive runs | Over-provisioned memory | Profile and reduce allocation |
| Stalled crawl | Request handler timeout | Set `requestHandlerTimeoutSecs` |

## Examples

**Slow Playwright crawl on static pages.** The Actor renders every page in a browser at 3 pages/min. The pages are server-rendered HTML, so switch to `CheerioCrawler` and raise `maxConcurrency` — ~30 pages/min (10x). Code: [references/implementation.md](https://skilld.dev/api/skills-raw/jeremylongshore/tons-of-skills-marketplace/apify-performance-tuning/references/implementation.md) Steps 1-2.

**Out-of-memory crashes under load.** A `PlaywrightCrawler` at `maxConcurrency: 20` OOMs. Drop concurrency to 3, block images/CSS/fonts in `preNavigationHooks`, and call `window.stop()` post-navigation. Code: [references/implementation.md](https://skilld.dev/api/skills-raw/jeremylongshore/tons-of-skills-marketplace/apify-performance-tuning/references/implementation.md) Step 3.

**Getting blocked (403s) mid-crawl.** Start on datacenter proxies, and on a 403 re-enqueue the request with a residential proxy and retire the session to force a new IP. Code: [references/implementation.md](https://skilld.dev/api/skills-raw/jeremylongshore/tons-of-skills-marketplace/apify-performance-tuning/references/implementation.md) Step 5.

**Runs cost too much.** A 4GB allocation bills 8x more than needed for HTML parsing. Right-size to 512 MB and re-measure cost/run. Code: [references/implementation.md](https://skilld.dev/api/skills-raw/jeremylongshore/tons-of-skills-marketplace/apify-performance-tuning/references/implementation.md) Step 4; impact table in [references/monitoring.md](https://skilld.dev/api/skills-raw/jeremylongshore/tons-of-skills-marketplace/apify-performance-tuning/references/monitoring.md).

## Resources

- [references/implementation.md](https://skilld.dev/api/skills-raw/jeremylongshore/tons-of-skills-marketplace/apify-performance-tuning/references/implementation.md) — full six-step walkthrough with all code blocks
- [references/monitoring.md](https://skilld.dev/api/skills-raw/jeremylongshore/tons-of-skills-marketplace/apify-performance-tuning/references/monitoring.md) — in-crawl throughput logger + before/after impact table
- [Crawlee Performance Guide](https://crawlee.dev/js/docs/guides/configuration)
- [Actor Memory & Performance](https://docs.apify.com/platform/actors/running/usage-and-resources)
- [Proxy Management](https://docs.apify.com/sdk/js/docs/guides/proxy-management)

## Next Steps

For cost optimization beyond performance tuning, see the `apify-cost-tuning` skill in this pack — it covers compute-unit budgeting, storage costs, and scheduling strategies that this skill's memory right-sizing feeds into.
