Getting started with ScrapingBee and PHP

In this tutorial, we will see how you can use ScrapingBee’s API with PHP, and use it to scrape web pages. As such, we will cover these topics:

• General structure of an API request
• Create your first API request.

Let’s get started!

1. General structure of an API request

The general structure of an API request made in PHP will always look like this:

<?php

// Get cURL resource
$ch = curl_init();

// Set base url & API key
$BASE_URL = "https://app.scrapingbee.com/api/v1/?";
$API_KEY = "YOUR-API-KEY";

// Set parameters
$parameters = array(
    'api_key' => $API_KEY,
    'url' => 'YOUR-URL' // The URL to scrape
);
// Building the URL query
$query = http_build_query($parameters);

// Set the URL for cURL
curl_setopt($ch, CURLOPT_URL, $BASE_URL.$query);

// Set method
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET');

// Return the transfer as a string
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

// Send the request and save response to $response
$response = curl_exec($ch);

// Stop if fails
if (!$response) {
    die('Error: "' . curl_error($ch) . '" - Code: ' . curl_errno($ch));
}

// Do what you want with the response here

// Close curl resource to free up system resources
curl_close($ch);
?>

And you can do whatever you want with the response variable! For example:

echo 'HTTP Status Code: ' . curl_getinfo($ch, CURLINFO_HTTP_CODE) . PHP_EOL;
echo 'Response Body: ' . $response . PHP_EOL;

2. Create your first API request:

Let’s create a tool that saves the HTML code of ScrapingBee’s blog:

<?php

// Get cURL resource
$ch = curl_init();

// Set base url & API key
$BASE_URL = "https://app.scrapingbee.com/api/v1/?";
$API_KEY = "YOUR-API-KEY";

// Set parameters
$parameters = array(
    'api_key' => $API_KEY,
    'url' => 'https://scrapingbee.com/blog' // The URL to scrape
);
// Building the URL query
$query = http_build_query($parameters);

// Set the URL for cURL
curl_setopt($ch, CURLOPT_URL, $BASE_URL.$query);

// Set method
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET');

// Return the transfer as a string
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

// Send the request and save response to $response
$response = curl_exec($ch);

// Stop if fails
if (!$response) {
    die('Error: "' . curl_error($ch) . '" - Code: ' . curl_errno($ch));
}

echo 'HTTP Status Code: ' . curl_getinfo($ch, CURLINFO_HTTP_CODE) . PHP_EOL;

$file = fopen("scrapingbee.html", "w");
fwrite($file, $response);
fclose($file);

// Close curl resource to free up system resources
curl_close($ch);
?>