Remote MCP

What is MCP?

The Model Context Protocol (MCP) is an open standard that lets AI clients call external tools. Think of it as a USB port for AI — any client that speaks MCP can plug into any MCP server, no custom integration needed. ScrapingBee's Remote MCP server is one such tool provider, giving your AI live web access for scraping, searching, and extracting structured data on demand.

Overview

The ScrapingBee Remote MCP server exposes the ScrapingBee API as tools any MCP client can call. Connect a client like Claude Desktop, Cursor, or ChatGPT and your AI assistant gains web scraping, screenshotting, Google/Amazon/Walmart/YouTube search, and ChatGPT prompting — all backed by your ScrapingBee account.

The server is hosted by ScrapingBee at https://mcp.scrapingbee.com/mcp. There is nothing to install or run on your machine — your client connects directly to the hosted endpoint over HTTPS. (Your API key is appended as a query parameter when you configure a client; see Authentication.)

What you can build

Hooking an MCP-capable client up to ScrapingBee turns the AI into a web-aware agent. A couple of starting points:

Research agent. Ask: "Find the three best Python libraries for PDF parsing in 2026 and summarize the trade-offs." The AI calls fast_search to find candidates, then get_page_text on each result to read the docs and GitHub READMEs — no copy-pasted URLs.

Competitor price monitoring. Drop a list of ASINs into the chat: "Check the current Amazon price for each and flag anything that dropped more than 10% this week." The AI calls get_amazon_product_details per ASIN, compares against your earlier numbers, and returns a summary.

The AI decides which tools to call from the descriptions it sees at handshake — combine them however the task demands.

Prerequisites

You need:

• A ScrapingBee account and an API key (available at app.scrapingbee.com/account/manage/api_key)
• An MCP-compatible AI client — Claude Desktop, Cursor, ChatGPT (Plus/Pro/Business/Enterprise), or any client that speaks MCP over streamable HTTP
• Node.js installed locally if your client connects through the mcp-remote wrapper (Claude Desktop, Cursor — see below)

Connecting your AI client

The server URL is the same for every client — only the configuration format differs.

Claude Desktop

Open claude_desktop_config.json (Claude → Settings → Developer → Edit Config) and add the ScrapingBee entry under mcpServers:

{
  "mcpServers": {
    "scrapingbee": {
      "command": "npx",
      "args": [
        "mcp-remote",
        "https://mcp.scrapingbee.com/mcp?api_key=YOUR_API_KEY"
      ]
    }
  }
}

Replace YOUR_API_KEY with the key from your ScrapingBee dashboard. Fully quit and reopen Claude Desktop. ScrapingBee appears under the + menu next to the prompt input → Connectors → toggle scrapingbee on. From then on it's available in any conversation. To make sure the model routes to the MCP server (rather than any built-in web tools or other installed alternatives), include the phrase "use scrapingbee mcp" in your prompt — e.g. "use scrapingbee mcp to find the top 5 Python PDF parsing libraries."

Cursor

Open Cursor's settings, find the MCP section, and click + New MCP Server ("Add a Custom MCP Server"). Cursor opens its mcp.json config file — paste this entry, using your real API key:

{
  "mcpServers": {
    "scrapingbee": {
      "command": "npx",
      "args": [
        "mcp-remote",
        "https://mcp.scrapingbee.com/mcp?api_key=YOUR_API_KEY"
      ]
    }
  }
}

The same mcp-remote wrapper used by Claude Desktop works here. Save the file. Cursor's MCP settings view doesn't refresh automatically — click into any other settings section, then back into MCP, and scrapingbee will be listed. Toggle it on; the green status indicator appears and the tools become available to Cursor's agent. To make sure the agent routes to the MCP server (and not built-in web tools, another MCP server, or the ScrapingBee CLI skill if you have it installed), include the phrase "use scrapingbee mcp" in your prompt — e.g. "use scrapingbee mcp to extract product prices from this Amazon URL."

By default this writes to your global config (~/.cursor/mcp.json, available in every project). For a project-only install, put the same JSON in .cursor/mcp.json at your project root instead.

ChatGPT

ChatGPT supports custom MCP servers as apps. MCP custom apps are available on:

• Plus and Pro — read/fetch tools only
• Business, Enterprise, and Edu — full MCP including write actions (beta)

To connect ScrapingBee:

1. Enable developer mode: Settings → Apps → Advanced Settings → Developer mode. On Business/Enterprise workspaces an admin may need to enable it first under Workspace Settings → Permissions & Roles → Connected Data.
2. Go to Settings → Apps → Create. The New App dialog opens.
3. Name the app ScrapingBee — this is what you'll reference in prompts.
4. In the Connection field, keep the toggle on Server URL and paste the MCP endpoint: https://mcp.scrapingbee.com/mcp?api_key=YOUR_API_KEY (replace YOUR_API_KEY with your real key).
5. Leave Authentication set to No Auth — the API key is already embedded in the URL.
6. Check the I understand and want to continue box (this acknowledges the security warning ChatGPT shows for any custom MCP server), then click Create. Creation takes a few seconds.
7. An Add ScrapingBee to ChatGPT dialog appears with permission and data-use notes. Click Connect to finish the setup.
8. The new app appears in Settings → Apps → Enabled Apps with a Dev label. On Business/Enterprise an admin then publishes it for the workspace.

Once connected, invoke ScrapingBee in any chat by typing @ScrapingBee in your prompt — this routes the request straight to the MCP app, e.g. "@ScrapingBee get the current price of ASIN B00P7VW2H6". Alternative: click the + button next to the prompt input → More → select the app from the list.

Custom MCP app setup has to be done from chatgpt.com on the web.

Other MCP clients

Any client that speaks MCP over streamable HTTP can use the same endpoint. For clients that don't have a native HTTP transport, the mcp-remote npx wrapper bridges from stdio to the remote URL (the same pattern shown above for Claude Desktop and Cursor).

If you'd rather talk to the server directly without a desktop client, a minimal Python example using httpx (MCP handshake + tool listing + tool call) is available at mcp.scrapingbee.com under the Custom Python Client tab.

Available MCP tools

The server exposes 16 tools, grouped below by capability.

HTML API

All five accept premium_proxy, custom_google, and country_code for geo-targeted and anti-bot scraping. The first four also accept stealth_proxy; get_file doesn't, because stealth_proxy requires JavaScript rendering and get_file runs without it (raw file downloads don't need a browser).

When to use premium_proxy and stealth_proxy: Start without either. If the site blocks you (CAPTCHA page, empty result, or error response), retry with premium_proxy: true. If you're still blocked, escalate to stealth_proxy: true. The ladder goes cheap → expensive, so only step up when you need to. See credit costs for exact pricing per option.

Search and AI

E-commerce

YouTube

Each tool's full parameter list is exposed to your client through the MCP tools/list handshake — most AI assistants surface the parameters inline when you ask them to call a tool.

Authentication

The Remote MCP server reads your ScrapingBee API key from the api_key query parameter on the server URL itself — e.g. https://mcp.scrapingbee.com/mcp?api_key=YOUR_API_KEY. There is no header-based or OAuth flow. The key is part of the URL your client connects to, and the server forwards it to the underlying ScrapingBee API on each tool call.

Keep the URL out of source control. Most client config files (Claude Desktop, Cursor) live in your home directory or workspace, so anyone with read access to those files sees the key. Treat the full URL like any other secret: use environment variables in scripted setups, and rotate the key from your account dashboard if it leaks.

Credits and limits

Each MCP tool call consumes ScrapingBee credits at exactly the same rate as the underlying API endpoint it wraps. Using the MCP server adds no surcharge — calling get_page_html via MCP costs the same as calling the HTML API directly. Credit costs per endpoint are documented in the relevant API reference (HTML API, Google API, Amazon API, etc.).

The hosted server enforces a global concurrency limit shared across all customers; under sustained heavy load some requests may receive HTTP 429 Too Many Requests. Retry with exponential backoff if your client surfaces these. The per-account concurrency limit on your ScrapingBee plan still applies as usual.

Troubleshooting

"ScrapingBee API key is missing in the remote mcp url." The ?api_key= query parameter is missing or empty in the URL your client is using. Edit the client config so the URL contains the full key (?api_key=YOUR_KEY) and reload the client.

Tools appear to load but every call returns Error 401: Invalid api key The API key is present but invalid, revoked, or out of credits. Check the key at app.scrapingbee.com/account/manage/api_key and your credit balance.

Client can't find the server / handshake never completes For clients using the mcp-remote wrapper (Claude Desktop, Cursor), make sure Node.js is installed and npx is on your PATH. Run npx mcp-remote "https://mcp.scrapingbee.com/mcp?api_key=YOUR_KEY" in a terminal — a healthy run prints Connected to remote server using StreamableHTTPClientTransport followed by Proxy established successfully. If you don't see those lines, the issue is local (Node.js missing, outbound network blocked); if you do see them but your client still can't connect, the issue is in the client's MCP config.

Resources

• Model Context Protocol specification
• ScrapingBee dashboard — manage your API key, view usage, upgrade your plan
• Knowledge Base — searchable troubleshooting articles