Getting Started with ScrapingBee's NodeJS SDK

If you’re looking for a web scraping API for Node.js, you can use the ScrapingBee SDK, which acts as a wrapper around ScrapingBee’s API. In this tutorial, we’ll show you how to install it and use it for web scraping with Node.js. To follow along, you will need a ScrapingBee API key (get one here). These snippets assume that this key is set as the SCRAPINGBEE_KEY environment variable. For security, don’t hardcode your API key, as you could accidentally commit it to version control.

1. Install the ScrapingBee SDK

To use the ScrapingBee SDK, you first need to install it. You can do that using npm or any other package manager for Node.js using the package name scrapingbee:

npm install scrapingbee

Note: The latest version of our SDK is 2.0.0, and it requires Node.js 24 or later.

2. A simple web scraping API request

To get started, you can try the following snippet, which shows how to scrape a URL and extract some data based on CSS selectors with just an API call:

const { ScrapingBeeClient } = require('scrapingbee');
async function scrape() {
  const scrapingClient = new ScrapingBeeClient(process.env.SCRAPINGBEE_KEY);
  try {
    const response = await scrapingClient.htmlApi({
      url: 'https://www.scrapingbee.com/', // url to scrape
      params: {
        extract_rules: {
          "page_heading": "h1" // CSS selector to extract
        }
      },
    });

    console.log(response.status);

    const decoder = new TextDecoder();
    const text = decoder.decode(response.data);
    console.log(text);
  } catch(e) {
    console.log(`An error occurred: ${e}`);
  }
}
scrape();

The above code scrapes the ScrapingBee home page and extracts the content of its h1 using the CSS selector. It outputs the following:

200
{"page_heading": "The Best Web Scraping API to Avoid Getting Blocked"}

The ScrapingBee SDK provides premium proxies, stealth proxies, and other features out of the box. These can be helpful for scraping websites when your default IP gets blocked. They can be easily used by adding additional parameters (e.g., premium_proxy: true) to the API call:

const response = await scrapingClient.htmlApi({
  url: 'https://www.scrapingbee.com/', // url to scrape
  params: {
    premium_proxy: true, // for hard to scrape websites
    extract_rules: {
      "page_heading": "h1" // CSS selector to extract
    }
  },
});

If premium proxies are still blocked, you can enable ScrapingBee’s stealth proxy pool with stealth_proxy: true. Keep in mind that premium and stealth proxies consume more API credits than the default proxy.

3. Scraping Markdown with Node.js

If you’re looking to scrape web pages as Markdown for LLMs, ScrapingBee handles that on its backend. You only need to pass a single parameter, return_page_markdown: true.

const { ScrapingBeeClient } = require('scrapingbee');
async function scrape() {
  const scrapingClient = new ScrapingBeeClient(process.env.SCRAPINGBEE_KEY);
  try {
    const response = await scrapingClient.htmlApi({
      url: 'https://www.scrapingbee.com/', // url to scrape
      params: {
        return_page_markdown: true
      },
    });
    console.log(response.status);
    const decoder = new TextDecoder();
    const text = decoder.decode(response.data);
    console.log(text);
  } catch(e) {
    console.log(`An error occurred: ${e}`);
  }
}
scrape();

The code above prints the main content of the ScrapingBee home page parsed into Markdown. Below is an example output, abbreviated:

200
The Best Web Scraping API to Avoid Getting Blocked
==================================================

Scrape websites with our powerful web scraping API, without managing proxies, browsers, or anti-bot defenses. Fetch pages, leverage AI extraction, or integrate with dedicated APIs for LLM, RAG, and analytics pipelines.

* Easy to integrate
* 1,000 free scraper API credits
* No credit card required
…

Conclusion

In this tutorial, we briefly covered how to get started with the ScrapingBee Node.js SDK. Features such as premium proxies, JS rendering, and data extraction with CSS selectors or AI LLM prompts are available through API parameters. All of these parameters are detailed in our documentation. To try out the tutorial or get started, you can sign up for a free trial with 1,000 credits.