← Tool directory
TOOL PROFILE

Crawlee

Crawlee fits JavaScript teams building crawlers that need request queues, stored output and browser automation.

Open source

Community support

Upvotes come from SearchEngineAnswer members. GitHub stars and reader reviews are counted separately.

Claim this tool if you maintain it.

Best for
Crawling
Setup required
Node.js; optional browser engine
Costs to check
Local compute or optional paid Apify hosting
License in catalog
Apache-2.0

GitHub repository

GitHub stars
26,394
Forks
1,747
Last push
2026-11-09
Primary language
TypeScript

GitHub snapshot from . Stars count GitHub bookmarks. Reader ratings appear separately below.

How to install Crawlee

  1. In a Node.js project, install Crawlee and Playwright if you plan to use the browser crawler.
    npm install crawlee playwright
  2. Follow the browser installation instructions for your environment. Start with a handler that stores the URL and the field you want to extract.

First tutorial for Crawlee

  1. Compare extraction from a static page with extraction from a page where JavaScript inserts the target field.
  2. Log which crawler mode produced each result. Verify that retries do not add duplicate rows.

Official installation and usage documentation ↗

Where Crawlee fits

Build a JavaScript or TypeScript crawler that needs to handle both raw HTML and browser-rendered pages.

When to choose another tool
Use a simpler HTML crawler for fields present in the initial response. Use Playwright directly when you only need a handful of scripted browser checks.
First evaluation
Choose one static page and one page with a field added after rendering. Check which crawler mode can extract each field, then log the URL, extraction result and any retry.
How to interpret the result
Budget for browser overhead. Check that retries do not create duplicate records or turn a missing field into a silent success.

Reviews and discussion

Reader reviews

Be the first to share your experience with this tool.

Discussion

Ask a question or share a setup tip.

Write a review or join the discussion

Reviews should describe your own experience. All submissions are moderated.

Flag an outdated detail

Include the field that needs updating and a source link. Copy your draft, then use the corrections policy to contact the editorial team.