apify_skill

Overview

This skill runs Apify Actors (web scrapers, crawlers, and automation tools) and retrieves their results via the Apify REST API using curl. It guides you to find the right Actor, prepare input using the Actor's input schema, run Actors synchronously or asynchronously, poll status, and fetch dataset or key-value results. Designed for practical, command-line usage with examples and error-handling tips.

How this skill works

Use your APIFY_TOKEN as a Bearer token against https://api.apify.com. Search the Apify Store for an Actor, inspect its default build to get README and inputSchema, then start a run with a POST request. For short runs use sync endpoints to get dataset items directly; for longer runs start async runs, poll the run status, and then fetch dataset items, key-value records, or logs with simple curl commands.

When to use it

• You need to scrape or crawl a website and return structured results quickly from the command line.
• You want to run an off-the-shelf Actor from the Apify Store without writing Node/Python code.
• You must automate periodic scraping tasks or integrate Actor runs into pipelines using shell scripts.
• You need screenshots, HTML, or other large artifacts stored in Apify key-value stores.
• You want to debug runs by fetching run logs or inspect input schema before running an Actor.

Best practices

• Always set APIFY_TOKEN as an environment variable and send it as Authorization: Bearer $APIFY_TOKEN.
• Fetch the Actor's default build and parse inputSchema to construct valid JSON input before running.
• Use run-sync endpoints only for short tasks (under 300s); otherwise start async runs and poll status.
• Use ?waitForFinish for small waits (0–60s) to reduce polling frequency for short runs.
• Handle HTTP errors explicitly (401, 402/403, 404, 408, 429) and implement exponential backoff on 429.

Example use cases

• Quickly scrape a single site and return JSON using run-sync-get-dataset-items with a timeout.
• Run a Google search scraper Actor to collect SERP results for a set of queries.
• Start a large crawl asynchronously, poll until SUCCEEDED, and then download dataset items.
• Retrieve screenshots or HTML from the key-value store OUTPUT record for visual verification.
• Abort a runaway Actor run via the actor-runs/RUN_ID/abort endpoint.