ScrapingBee logo

The ScrapingBee Blog

We help you get better at web-scraping: detailed tutorials, case studies and writings by industry experts.

Don't know where to begin?

Check out our most popular articles.

Web scraping without getting blocked Web scraping without getting blocked Web scraping in Python Web scraping in Python Web scraping in NodeJS Web scraping in NodeJS Web scraping in R Web scraping in R Web scraping in C++ Web scraping in C++ Web scraping in C# Web scraping in C# Web scraping in Php Web scraping in Php Web scraping in Scala Web scraping in Scala Java web scraping Book Java web scraping Book Web scraping in Ruby Web scraping in Ruby Web scraping in Go Web scraping in Go Web scraping in Elixir Web scraping in Elixir Web scraping in Rust Web scraping in Rust Web scraping in Perl Web scraping in Perl No code web scraping No code web scraping No code competitor monitoring No code competitor monitoring
cURL JavaScript Guide: How to convert commands to JS blog post cover
11 min read

cURL JavaScript Guide: How to convert commands to JS

Learn how to convert cURL JavaScript commands into clean Fetch or Axios code, extract cURL from your browser, and run native cURL in Node.js when needed.

Photo of Arek Nawo
Arek Nawo
How to use a proxy with HttpClient in C# blog post cover
18 min read

How to use a proxy with HttpClient in C#

Learn how to configure a C# HttpClient proxy for web scraping and API requests, including authenticated proxies, rotation, and ScrapingBee integration.

Photo of Yasoob Khalid
Yasoob Khalid
HTML Parsing in Java with JSoup blog post cover
11 min read

HTML Parsing in Java with JSoup

This tutorial will teach you how to parse HTML with Java using the Jsoup library. Jsoup is an open source Java library to extract an manipulate HTML using CSS selectors.

Photo of Aniket Bhattacharyea
Aniket Bhattacharyea
Scrapy vs Selenium: Which one to choose blog post cover
17 min read

Scrapy vs Selenium: Which one to choose

Explore the strengths and weaknesses of Scrapy and Selenium to determine which web scraping tool best fits your needs. Read the full article for insights.

Photo of Kevin Sahin
Kevin Sahin
The Best Ruby HTTP clients blog post cover
11 min read

The Best Ruby HTTP clients

This article will discuss the best HTTP clients in Ruby. Faraday, HTTParty, Rest-client...it can be hard to choose the best one.

Photo of Sylwia vargas
Sylwia vargas
Using Watir to automate web browsers with Ruby blog post cover
9 min read

Using Watir to automate web browsers with Ruby

In this tutorial, you will learn about browser automation and how to do it using Watir and Ruby.

Photo of Jude Ero
Jude Ero
5 Best Free Web Scraping Tools for 2026 blog post cover
12 min read

5 Best Free Web Scraping Tools for 2026

Looking for a free web scraper in 2026? Compare the best free web scraping tools, key limits, and use cases - see when a free-to-start API makes more sense.-

Photo of Kevin Sahin
Kevin Sahin
BeautifulSoup tutorial: Scraping web pages with Python blog post cover
20 min read

BeautifulSoup tutorial: Scraping web pages with Python

In this tutorial, we will learn how to scrape the web using BeautifulSoup and CSS selectors with step-by-step instructions.

Photo of Satyam Tripathi
Satyam Tripathi
Charles proxy for web scraping blog post cover
10 min read

Charles proxy for web scraping

This post will show you how to use Charles proxy to reverse-engineer hidden and private APIs on websites and mobile applications.

Photo of Kevin Sahin
Kevin Sahin
Getting Started with MechanicalSoup blog post cover
12 min read

Getting Started with MechanicalSoup

MechanicalSoup is an excellent tool that can be used to scrape websites in Python. In this guide, you'll give a high-level overview of how to use it and explore some of its advanced capabilities.

Photo of Kumar Harsh
Kumar Harsh
Previous
Next