ScrapingBee logo

The ScrapingBee Blog

We help you get better at web-scraping: detailed tutorials, case studies and writings by industry experts.

Don't know where to begin?

Check out our most popular articles.

Python Web scraping without getting blocked Python Web scraping in Python Python Web scraping in NodeJS Python Web scraping in R Python Web scraping in C++ Python Web scraping in C# Python Web scraping in Php Python Web scraping in Scala Python Java web scraping Book Python Web scraping in Ruby Python Web scraping in Go Python Web scraping in Elixir Python Web scraping in Rust Python Web scraping in Perl Python No code web scraping Python No code competitor monitoring
How to build a job board with web scraping and ChatGPT
10 min read

How to build a job board with web scraping and ChatGPT

Learn how to collect job openings by scraping Google and recruiting software sites, and then extract useful information such as salary and benefits using ChatGPT.

Lior Neu-ner
6 min read

The 11 best web scraping subreddits

Learn about the 11 best web scraping subreddits that can help you on your web scraping journey

Pierre de Wulf
12 min read

Web Scraping with Scala

This tutorial cove the main tools and techniques for web scraping in Scala. You will learn how to use the Jsoup library to parse HTML with Scala, and renders Javascript with Selenium.

Raul Estrada
10 min read

How to read and parse JSON data with Python

Learn how to effectively read, parse, and dump JSON data with Python.

Yasoob Khalid
6 min read

What are the 6 characteristics of a REST API?

Learn about the six characteristics that each API has to conform to in order to be considered a REST API.

Yasoob Khalid
13 min read

API for dummies: learning the basics of API

Learn about the basics of APIs and the different kinds of APIs that are available to use.

Yasoob Khalid
12 min read

How to Parse HTML with Regex

Web scraping has a number of applications in the modern day software industry. This tutorial shows you how to parse HTML and extract data from the content using regular expressions (regex).

Rajkumar Venkatasamy
14 min read

Web crawling with Python

This post will show you how to crawl the web using Python. Web crawling is a powerful technique to collect data from the web by finding all the URLs for one or multiple domains

Ari Bajo
14 min read

How to scrape channel data from YouTube

Learn how to easily scrape channel data from youtube.com.

Yasoob Khalid
7 min read

Ruby HTML and XML Parsers

Web scraping comes in handy when collecting large amounts of data from the internet. This roundup shares a list of popular Ruby HTML and XML parsers that you can use to simplify web scraping.

Tahanima Chowdhury
17 min read

Easy web scraping with Scrapy

Scrapy is the most popular Python web scraping framework. In this tutorial we will see how to scrape an E-commerce website with Scrapy from scratch.

Kevin Sahin
12 min read

XPath vs CSS selectors

Looking at the differences between XPath expressions and CSS selectors. When it is best to use an XPath expression and when a CSS selector. Advantages and drawbacks of each.

Alexander M
6 min read

How to use a Proxy with Ruby and Faraday

Learn how to use a proxy with Ruby and Faraday and prevent your IP from being blacklisted.

Yasoob Khalid
10 min read

How to scrape data from realtor.com

Learn how to use selenium to scrape data from realtor.com and evade bot detection.

Yasoob Khalid
15 min read

Using CSS Selectors for Web Scraping

We are taking a look at how CSS selectors can help us in web scraping, what their syntax is, how we build an ideal selector string, and how they are supported in mainstream languages.

Alexander M
Next