Data Scraping API

# pip install scrapingbee

from scrapingbee import ScrapingBeeClient

client = ScrapingBeeClient(api_key='YOUR-API-KEY')
response = client.get(
    'https://www.scrapingbee.com/blog',
    params={
        'extract_rules':{
            "title" : "h1",
            "subtitle" : "#subtitle",
            "articles": {
                "selector": ".card",
                "type": "list",
                "output": {
                    "title": ".post-title",
                    "link": ".post-title@href",
                    "description": ".post-description"
                }
            }
        },
    },
)
print('Response HTTP Status Code: ', response.status_code)
print('Response HTTP Response Body: ', response.json())

----------

Response HTTP Status Code: 200

{
  "title": "The ScrapingBee Blog",
  "subtitle": " We help you get better at web-scraping: detailed tutorial, case studies and writing by industry experts",
  "articles": [
    {
      "title": "  Block ressources with Puppeteer - (5min)",
      "link": "https://www.scrapingbee.com/blog/block-requests-puppeteer/",
      "description": "This article will show you how to intercept and block requests with Puppeteer using the request interception API and the puppeteer extra plugin."
    },
    ...
    {
      "title": "  Web Scraping vs Web Crawling: Ultimate Guide - (10min)",
      "link": "https://www.scrapingbee.com/blog/scraping-vs-crawling/",
      "description": "What is the difference between web scraping and web crawling? That's exactly what we will discover in this article, and the different tools you can use."
    },
  ]
}