ScrapingBee logo
  • Login
  • Sign Up
  • Pricing
  • FAQ
  • Blog
  • Other Features
    • Screenshots
    • Google search API
    • Data extraction
    • JavaScript scenario
    • No code web scraping
  • Developers
    • Tutorials
    • Documentation
    • Knowledge Base

The ScrapingBee Blog

We help you get better at web-scraping: detailed tutorials, case studies and writings by industry experts.

Don't know where to begin?

Check out our most popular articles.

Python Web scraping without getting blocked Python Web scraping in Python Python Web scraping in NodeJS Python Web scraping in R Python Web scraping in C++ Python Web scraping in C# Python Web scraping in Php Python Web scraping in Scala Python Java web scraping Book Python Web scraping in Ruby Python Web scraping in Go Python Web scraping in Elixir Python Web scraping in Rust Python Web scraping in Perl Python No code web scraping Python No code competitor monitoring
Web Scraping with Html Agility Pack
11 min read

Web Scraping with Html Agility Pack

Html Agility Pack is the standard for parsing HTML pages in C#. The HTML Agility pack has everything you need to parse, manipulate and extract data from any HTML document.

Jennifer Marsh 22 July 2021
8 min read

Using cURL with a proxy

Using a proxy with cURL is easy. This step-by-step tutorial will show you the three different ways to set up a proxy server with cURL command line tool.

Kevin Sahin 14 July 2021
5 min read

How to send a POST with Python Requests?

Python POST data using requests package. This article will teach you how to POST JSON data with Python Requests library.

Ian Wootten 05 July 2021
10 min read

What is data parsing?

Data parsing is the process of taking data in one format and transforming it to another format. This is particulary interesting for web scraping.

Kevin Sahin 07 June 2021
5 min read

Block ressources with Puppeteer

This article will show you how to intercept and block requests with Puppeteer using the request interception API and the puppeteer extra plugin.

Shadid Haque 25 May 2021
10 min read

Web Scraping vs Web Crawling: Ultimate Guide

What is the difference between web scraping and web crawling? That's exactly what we will discover in this article, and the different tools you can use.

Kevin Sahin 18 May 2021
6 min read

How to submit a form with Puppeteer?

Submitting a form with Puppeteer can be tricky. In this article, we will show the different way to submit a simple form, but also with attachments.

Shadid Haque 13 May 2021
11 min read

The Best Ruby HTTP clients for 2021

This article will discuss the best HTTP clients in Ruby. Faraday, HTTParty, Rest-client...it can be hard to choose the best one.

Sylwia vargas 28 April 2021
7 min read

How to download a file with Puppeteer?

There are different ways to download a file with Puppeteer. Unfortunately, all of the different techniques are not well documented. We will go through several practical examples of file dowloading with Puppeteer.

Kevin Sahin 23 April 2021
11 min read

Scraping the web with Playwright

Learn how to scrape the web with Playwright. We will see the different ways to scrape the web with Playwright through lots of example.

Kevin Sahin 07 April 2021
12 min read

Top 5 Rotating and Residential Proxies for Web Scraping

In this post you will find the 5 best rotating and residential proxies for Web Scraping. 1) ScrapingBee 2) Luminati 3) Oxylabs 4) Smartproxy 5) Crawlera

Kevin Sahin 23 March 2021
15 min read

The 5 Best Free Proxy Lists for Web Scraping

This post will show you the top 5 free proxy lists available on the web: ScrapingBee, ProxyScrape, FreeproxylistCZ and more.

Kevin Sahin 08 March 2021
11 min read

How to extract data from a website? Ultimate guide to pull data from any website

This article will teach you to extract data from the web easily. There are many ways to pull data from websites, from web scraping tools to in-house pipeline. This post will show you exactly what's the best solution to your web scraping problem.

Kevin Sahin 04 March 2021
7 min read

HTTP headers with axios

This post will show you how to send HTTP headers with Axios. We will see different examples with GET and POST requests on how to set your headers with Axios.

Kevin Sahin 15 January 2021
8 min read

Web Scraping with node-fetch

In this tutorial we will see how to use the node-fetch package for web scraping. We will learn what the fetch API is and the different ways to use the package.

Kevin Sahin 03 December 2020
Previous
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
Next

Tired of getting blocked while scraping the web?

ScrapingBee API handles headless browsers and rotates proxies for you.

Get access to 1,000 free API credits, no credit card required!

Try ScrapingBee for Free
ScrapingBee

ScrapingBee API handles headless browsers and rotates proxies for you.

Company

  • Team
  • Company's journey
  • Blog
  • Rebranding
  • Affiliate Program

Tools

  • Curl converter

Legal

  • Terms of Service
  • Privacy Policy
  • GDPR Compliance
  • Data Processing Agreement

Product

  • Features
  • Pricing
  • Status

How we compare

  • Alternative to Crawlera
  • Alternative to Luminati
  • Alternative to Smartproxy
  • Alternative to NetNut
  • Alternative to ScraperAPI
  • Alternatives to ScrapingBee

No code web scraping

  • No code web scraping
  • No code competitor monitoring
  • How to put scraped website data into Google Sheets
  • Send stock prices update to Slack
  • Scrape Amazon products' price with no code
  • Scrape Amazon products' price with no code
  • Extract job listings, details and salaries

Learning Web Scraping

  • Web scraping questions
  • A guide to Web Scraping without getting blocked
  • Web Scraping Tools
  • Best Free Proxies
  • Best Mobile proxies
  • Web Scraping vs Web Crawling
  • Rotating and residential proxies
  • Web Scraping with Python
  • Web Scraping with PHP
  • Web Scraping with Java
  • Web Scraping with Ruby
  • Web Scraping with NodeJS
  • Web Scraping with R
  • Web Scraping with C#
  • Web Scraping with C++
  • Web Scraping with Elixir
  • Web Scraping with Perl
  • Web Scraping with Rust
  • Web Scraping with Go

Copyright © 2023

Made in France