Best JavaScript Web Scraping Libraries

How JavaScript Scraping Libraries Are Organized

JavaScript scraping libraries fall into three functional layers. HTTP clients like Axios and Got handle fetching web pages by sending requests and receiving responses. HTML parsers like Cheerio and JSDOM take the raw HTML from those responses and provide an API for searching through the document and extracting data. Browser automation tools like Puppeteer and Playwright launch real browser instances that render JavaScript, execute AJAX calls, and give you access to the fully rendered page. Crawling frameworks like Crawlee combine all three layers into a complete system with request queuing, retry logic, data storage, and proxy management.

Most scraping projects use one tool from each layer. A typical lightweight setup pairs Axios (HTTP client) with Cheerio (parser). A browser-based setup uses Puppeteer or Playwright, which handle both fetching and rendering. A production-scale setup uses Crawlee, which wraps Cheerio, Puppeteer, or Playwright with infrastructure for managing large crawls. Understanding which layer each library occupies helps you assemble the right stack without redundant tools.

Cheerio

Cheerio is the most popular HTML parsing library for Node.js, averaging over 4 million weekly downloads on npm. It implements a subset of jQuery's API specifically designed for server-side use. You load an HTML string into Cheerio with cheerio.load(html), then use jQuery-style selectors and methods like $(selector), .find(), .text(), .attr(), .each(), and .map() to navigate the DOM and extract data.

Cheerio's primary advantage is speed. Because it parses HTML into a lightweight data structure rather than a full browser DOM, it processes pages in milliseconds with minimal memory overhead. A single Node.js process running Cheerio can parse hundreds of pages per second, making it the fastest option for static page scraping. Cheerio also handles malformed HTML gracefully, which is important for scraping real-world websites where markup quality varies widely.

The limitation is that Cheerio does not execute JavaScript. If the target page loads data through AJAX calls, renders content with React or Vue, or requires user interaction to reveal data, Cheerio will only see the initial HTML skeleton, not the fully rendered content. For these cases, you need Puppeteer or Playwright instead.

Cheerio is the right choice for scraping blog posts, news articles, documentation pages, product listings on traditional e-commerce sites, government databases, Wikipedia, and any other website that delivers its content in the initial HTML response. It pairs naturally with any HTTP client. For a hands-on guide, see Web Scraping with Cheerio.

Puppeteer

Puppeteer is Google's official Node.js library for controlling headless Chrome and Chromium browsers. It communicates with the browser through the Chrome DevTools Protocol, giving you programmatic access to everything a real Chrome user can do: navigating to URLs, clicking elements, filling forms, scrolling pages, taking screenshots, generating PDFs, and extracting data from the fully rendered DOM.

Puppeteer's key strength is rendering dynamic content. When you navigate to a page with Puppeteer, it executes all JavaScript, completes all AJAX calls, and renders the DOM exactly as a human user would see it. This makes it essential for scraping single-page applications built with React, Angular, or Vue, as well as any site that loads data asynchronously after the initial page load.

Puppeteer includes useful scraping features beyond basic rendering. Network interception lets you capture API responses directly, often providing cleaner data than parsing rendered HTML. Request blocking lets you prevent images, stylesheets, and fonts from loading, which speeds up page rendering when you only need the text content. Device emulation lets you scrape mobile versions of websites. The page.evaluate() method lets you run arbitrary JavaScript in the browser context, giving you access to the full browser DOM API.

The trade-off is resource consumption. Each Puppeteer instance runs a full Chromium process that consumes 150 to 500 MB of RAM depending on page complexity. Rendering a page takes one to five seconds compared to the milliseconds Cheerio needs for static HTML parsing. For high-volume scraping, you need careful concurrency management and significantly more server resources than a Cheerio-based approach.

Puppeteer is the right choice when you need to scrape JavaScript-rendered content, interact with page elements (clicking, scrolling, form submission), take screenshots, or bypass bot detection that requires a real browser. For comparisons with alternatives, see our guides on Puppeteer vs Playwright and Puppeteer overview.

Playwright

Playwright is Microsoft's browser automation library and the most direct alternative to Puppeteer. It supports three browser engines: Chromium (Chrome and Edge), Firefox, and WebKit (Safari). This cross-browser support is valuable for scraping sites that render differently across browsers, and for accessing content that is only available on specific browser platforms.

Playwright's most significant advantage over Puppeteer is its auto-waiting system. When you call methods like page.click(selector) or page.fill(selector, value), Playwright automatically waits for the element to appear in the DOM, become visible, become enabled, and become stable before performing the action. This eliminates one of the most common sources of flaky scraper behavior: timing issues where your code tries to interact with elements that have not finished loading yet. With Puppeteer, you often need explicit page.waitForSelector() calls, while Playwright handles the waiting implicitly.

Playwright also supports browser contexts, which are lightweight, isolated browser sessions within a single browser instance. You can create multiple contexts, each with its own cookies, local storage, and session state, without the overhead of launching separate browser processes. This is useful for scraping with multiple user sessions or for parallelizing scraping tasks within a single browser instance.

The API is largely similar to Puppeteer's but with some improvements. Playwright's locator API provides a more reliable way to reference page elements with automatic retries and built-in assertions. Its trace viewer lets you record and replay browser sessions for debugging. Its codegen tool generates scraper code by recording your manual browser interactions.

Playwright has overtaken Puppeteer as the recommended choice for new browser automation projects. Its broader browser support, more reliable waiting behavior, better concurrency model, and faster development pace make it the stronger option. Unless you have a specific reason to prefer Puppeteer (such as existing Puppeteer infrastructure), Playwright is the library to choose for browser-based scraping. See our Playwright guide for comprehensive coverage.

Axios

Axios is the most downloaded HTTP client for Node.js, with over 50 million weekly installations from npm. It provides a promise-based API for making HTTP requests with features that are directly useful for scraping: automatic JSON parsing, request and response interceptors, configurable timeouts, cookie management through a cookie jar, and request cancellation via AbortController.

For scraping, Axios handles the fetching step of the pipeline. You send a GET request to a URL and receive the HTML response, which you then pass to Cheerio or another parser for data extraction. Axios's interceptor system is particularly valuable for scraping because it lets you inject custom logic into the request and response lifecycle without modifying your core scraping code. For example, you can use a request interceptor to rotate User-Agent strings automatically, and a response interceptor to add retry logic for failed requests.

Axios supports custom headers, query parameters, proxy configuration, and HTTP basic authentication out of the box. It follows redirects automatically and provides detailed error information that distinguishes between network errors, timeout errors, and HTTP error responses. Combined with Cheerio, Axios forms the most popular static scraping stack in the JavaScript ecosystem. See Scraping with Axios and Cheerio for a detailed tutorial.

Got

Got is a lightweight HTTP client that positions itself as a more modern, feature-rich alternative to Axios with a smaller dependency tree. It includes several features that make it particularly well-suited for scraping workflows.

Got's built-in retry system is its standout feature. It automatically retries failed requests with configurable backoff strategies, maximum retry counts, and status code filters. For scraping, where transient errors from server overload, network issues, and rate limiting are routine, having retry logic built into the HTTP client eliminates a significant amount of custom code. You configure retry behavior once and every request inherits it.

Got also includes native HTTP/2 support, which can improve throughput when scraping sites that support HTTP/2. Its pagination helper simplifies fetching paginated API responses by automatically following next-page links or cursor tokens. Got's hook system provides similar functionality to Axios's interceptors, letting you inject custom logic at various stages of the request lifecycle.

The choice between Axios and Got is largely one of preference. Axios has a larger community and more tutorials. Got has more built-in features relevant to scraping (retries, pagination helpers) and a smaller dependency tree. Both work equally well with Cheerio for static page scraping.

Crawlee

Crawlee is a full web crawling and scraping framework built by Apify. It is the most complete, production-ready scraping framework available in the Node.js ecosystem, comparable to Python's Scrapy in scope and ambition. Crawlee is not a single library but a framework that wraps Cheerio, Puppeteer, and Playwright with infrastructure for building scrapers that run reliably at scale.

Crawlee provides three crawler classes that share a common API: CheerioCrawler for fast static page scraping, PuppeteerCrawler for dynamic pages using headless Chrome, and PlaywrightCrawler for cross-browser dynamic scraping. Regardless of which crawler you choose, you get the same request queuing system that handles URL deduplication, priority ordering, and automatic retries for failed requests. You get the same data storage API that persists scraped items to JSON, CSV, or a custom backend. And you get the same proxy rotation and session management system that distributes requests across proxy pools and maintains separate browser sessions.

Crawlee's request queue is particularly powerful. It persists to disk, which means your crawl survives process restarts. If your scraper crashes after processing 50,000 out of 100,000 URLs, you can restart it and it picks up where it left off without re-scraping already-processed pages. The queue also handles automatic URL normalization, preventing duplicate requests to URLs that differ only in trailing slashes, query parameter order, or fragment identifiers.

For stealth scraping, Crawlee includes built-in fingerprint randomization that varies browser fingerprints across requests, making it harder for target sites to identify your scraper. Its session rotation system automatically associates each request with a session that includes a specific proxy, set of cookies, and browser fingerprint, and retires sessions that start receiving errors.

Crawlee is the right choice when you are building a scraper that needs to handle thousands or millions of URLs, run reliably with automatic error recovery, rotate proxies and fingerprints, or integrate with the Apify cloud platform for managed execution and data storage. For smaller projects, the Axios-Cheerio or Puppeteer stack is simpler and requires less setup. See our Crawlee framework guide for a detailed walkthrough.

JSDOM

JSDOM is a pure JavaScript implementation of the W3C DOM and HTML standards. While Cheerio implements a subset of jQuery's API, JSDOM provides a full browser-like DOM environment, including document.querySelector(), document.querySelectorAll(), element.addEventListener(), and other standard DOM APIs. JSDOM can even execute some inline JavaScript, making it capable of handling pages with simple scripts that modify the DOM after load.

JSDOM occupies a middle ground between Cheerio and Puppeteer. It is significantly slower and heavier than Cheerio because it implements a complete DOM specification rather than just a CSS selector engine. But it is much lighter than Puppeteer because it does not launch a real browser process. It cannot render CSS, execute complex JavaScript applications, or make network requests triggered by page scripts.

In practice, JSDOM is rarely the best choice for scraping. Cheerio handles static HTML parsing faster and with less memory. Puppeteer and Playwright handle dynamic content more completely and reliably. JSDOM's niche is pages with simple inline scripts that modify the DOM, where Cheerio cannot execute the scripts and Puppeteer would be overkill. For most projects, start with Cheerio and move to Puppeteer/Playwright if needed, bypassing JSDOM entirely.

x-ray

x-ray is a declarative scraping library that lets you define what data to extract using a schema object rather than imperative code. Instead of writing loops and extraction logic, you pass x-ray a URL and a mapping object that describes which CSS selectors map to which output fields. x-ray handles fetching, parsing, pagination, and data extraction in a single declarative call.

The declarative approach makes x-ray code extremely concise for simple scraping tasks. A scraper that would take 20 lines with Axios and Cheerio might take 5 lines with x-ray. However, x-ray's simplicity becomes a limitation for complex scraping tasks that require custom logic, error handling, or conditional processing. Its development has also slowed compared to actively maintained alternatives like Crawlee.

x-ray is worth considering for quick, one-off scraping tasks where you want minimum code. For production systems or complex scraping logic, Cheerio or Crawlee provides more control and reliability.

Choosing the Right Library Stack

The right combination of libraries depends on three factors: whether the target site is static or dynamic, the scale of your scraping project, and whether you need a simple script or a production system.

For static pages at small scale (tens to hundreds of pages), use Axios and Cheerio. This combination is fast, lightweight, and easy to set up. It handles the vast majority of basic scraping tasks with minimal code.

For dynamic pages at small scale, use Playwright (preferred) or Puppeteer. These tools launch real browsers that render JavaScript, handle AJAX calls, and interact with page elements. Playwright is the better choice for new projects due to its auto-waiting, cross-browser support, and active development.

For any scraping project at large scale (thousands to millions of pages), use Crawlee. It provides request queuing, automatic retries, proxy rotation, data storage, and concurrency management out of the box. You choose the underlying engine (Cheerio, Puppeteer, or Playwright) based on whether your targets are static or dynamic, and Crawlee handles all the infrastructure around it.

For API-based data extraction, where the target site loads data from JSON endpoints, Axios or Got alone is sufficient. You do not need an HTML parser if the data comes as structured JSON. This is often the fastest and most reliable scraping approach when it is available.