In this guide we’ll dive into Google Shopping scraping techniques that actually work in 2025. If you’ve ever needed to extract product data, prices, or seller information from Google Shopping, you’re in the right place. Google Shopping scraping has become essential for businesses that need competitive pricing data. I’ve spent years refining these methods, and today I’ll show you how to use ScrapingBee to make this process straightforward and reliable.
Quick Answer (TL;DR)
To scrape Google Shopping effectively, our customizable tools meet all requirements for building an effective Google Shopping Scraper API. We can handle JavaScript rendering, bypasss anti-bot measures, and deliver structured data without managing proxies or browser instances. With just a few lines of Python code, you can extract product titles, prices, descriptions, and seller information.
Quickstart: Full Parser Code
If you want to skip the installation and work with a pre-built script, copy this parser code and make adjustments to match your use cases:
from scrapingbee import ScrapingBeeClient
import pandas as pd
import asyncio
import functools
client = ScrapingBeeClient(api_key='YOUR_API_KEY')
def google_shopping_search(search_query, country_code='us', start=0, filename="google_shopping_output.csv"):
set_1 = {
"products": {
"selector": ".MtXiu.mZ9c3d.wYFOId.M919M.W5CKGc.wTrwWd",
"type": "list",
"output": {
"name": ".rRCm8.sg5mJb .UC8ZCe.QS8Cxb .gkQHve.SsM98d.RmEs5b",
"price": ".rRCm8.sg5mJb .UC8ZCe.QS8Cxb .lmQWe",
"rating": ".rRCm8.sg5mJb .UC8ZCe.QS8Cxb .yi40Hd",
"reviews": ".rRCm8.sg5mJb .UC8ZCe.QS8Cxb .RDApEe",
"store": ".rRCm8.sg5mJb .UC8ZCe.QS8Cxb .WJMUdc.rw5ecc",
"delivery": ".rRCm8.sg5mJb .UC8ZCe.QS8Cxb .ybnj7e",
"rank": ".MtXiu.mZ9c3d.wYFOId.M919M.W5CKGc.wTrwWd@data-index", # rank starts from 0
"product_link": ".product_link@href"
}
}
}
set_2 = {
"products": {
"selector": ".sh-dgr__gr-auto.sh-dgr__grid-result",
"type": "list",
"output": {
"name": ".tAxDx",
"price": ".shntl .QIrs8",
"rating": ".Rsc7Yb",
"reviews": ".reviews",
"store": ".aULzUe.IuHnof",
"delivery": ".bONr3b > .vEjMR",
"features": ".F7Kwhf.bzCQNe.t5OWfd",
"link": "a.shntl@href",
"rank": ".sh-dgr__gr-auto.sh-dgr__grid-result@rank", # rank starts from 0
"product_id": ".sh-dgr__gr-auto.sh-dgr__grid-result@data-docid"
}
}
}
# Some regions have a different structure, so we have to use a different set of rule
extract_rules = set_1 #use set_2 is set_1 doesn't work
# JavaScript code to make data extraction easier and to make some value clean.
# Since the structure is different for us page, we are using 2 different codes enclosed in try catch block
# The try catch block will prevent our code from erroring out
js_scenario = {
"instructions": [
{"evaluate": rf"""
try {{
document.querySelector('.aULzUe.IuHnof > style')?.remove();
document.querySelectorAll('.qSSQfd.uqAnbd').forEach((e) => {{
const n_elm = document.createElement('p');
n_elm.className = 'reviews';
n_elm.textContent = e.nextSibling?.textContent || '';
e.append(n_elm);
}});
document.querySelectorAll('.sh-dgr__gr-auto.sh-dgr__grid-result').forEach((e, index) => {{
e.setAttribute('rank', index + {start});
}});
document.querySelectorAll('.sh-dgr__gr-auto.sh-dgr__grid-result a.shntl:first-child').forEach((e) => {{
e.href = e.href.replace(/.+\/url\?url=(.+)/g, '$1');
}});
}} catch(e) {{}}
try {{
(() => {{
const baseURL = 'https://www.google.com/search?start=0&q=laptop&gl={country_code}&udm=28#oshop=apv&oshopproduct=';
document.querySelectorAll('.MtXiu.mZ9c3d.wYFOId.M919M.W5CKGc.wTrwWd').forEach(item => {{
const gid = item.getAttribute('data-gid'),
mid = item.getAttribute('data-mid'),
oid = item.getAttribute('data-oid'),
iid = item.getAttribute('data-iid'),
rds = encodeURIComponent(btoa(item.getAttribute('data-rds') || ''));
if (gid && mid && oid && iid && rds) {{
let linkElement = item.querySelector('.product_link');
if (!linkElement) {{
linkElement = document.createElement('a');
linkElement.className = 'product_link';
linkElement.textContent = 'View Product';
linkElement.target = '_blank';
item.appendChild(linkElement);
}}
linkElement.href = `${{baseURL}}gid:${{gid}},mid:${{mid}},oid:${{oid}},iid:${{iid}},rds=${{rds}}&pvs=0`;
}}
}});
}})();
document.querySelectorAll('.rRCm8.sg5mJb .UC8ZCe.QS8Cxb .RDApEe').forEach((e) => {{
e.textContent = e.textContent.replace(/\((.+)\)/g, '$1');
}});
}} catch(e) {{}}
"""}
]
}
q = search_query.replace(' ', '+')
response = client.get(
f'https://www.google.com/search?start={start}&q={q}&gl={country_code}&tbm=shop',
params={
"custom_google": "true",
"wait_browser": "networkidle2",
"premium_proxy": "true",
"extract_rules": extract_rules,
"js_scenario": js_scenario,
'country_code': 'us' # Ideally we can put the same country as the argument. However, using European country code will result in a cookie popup; which may fail our code.
},
retries=3
)
products = response.json().get('products', [])
if products:
df = pd.DataFrame(products)
df.to_csv(filename, index=False)
print(f"✅ Exported {len(df)} products to {filename}")
return {
'start': start,
'products': products,
'info': f"{response.status_code} SUCCESS" if products else f"{response.status_code} NO PRODUCTS",
'next_start': start + len(products)
}
# Run 3 queries in parallel using asyncio
import concurrent.futures
async def main():
loop = asyncio.get_event_loop()
with concurrent.futures.ThreadPoolExecutor() as executor:
tasks = [
loop.run_in_executor(None, functools.partial(google_shopping_search, 'full frame camera', filename='camera.csv')),
loop.run_in_executor(None, functools.partial(google_shopping_search, 'gaming laptop', filename='laptop.csv')),
loop.run_in_executor(None, functools.partial(google_shopping_search, 'smartwatch', filename='watch.csv')),
]
await asyncio.gather(*tasks)
if __name__ == "__main__":
asyncio.run(main())
Setting Up Your Python Environment
Let’s get your environment ready for Google Shopping scraping. I’ll walk you through the essential setup steps to ensure everything runs smoothly. If you want more information, check out our blog on Web scraping with Python.
Let's begin by setting up Python and required libraries for data extraction. Make make sure you have Python 3.6+ installed on your system with this line in Command Prompt:
python --version
Next, install the necessary libraries. For our Google Shopping scraper Python implementation, we’ll need:
pip install scrapingbee
pip install pandas
The ScrapingBee client library makes it easy to structure GET API requests, while pandas helps us organize and export the scraped data. In my experience, this combination provides the perfect balance of simplicity and power for Google Shopping data extraction.
Once everything is installed, make sure you have the right libraries with a "pip list" command:
Create your project folder
I always recommend creating a dedicated project folder to keep your scraping scripts organized. This structure also makes it easier to add more functionality later, such as data processing or visualization components.
Get your API key
To use ScrapingBee, you’ll need an API key:
Sign up at ScrapingBee.com
Navigate to your dashboard
Copy your API key from the account section
Now let’s build our Google Shopping price scraper using our powerful API. This approach handles all the complex parts of web scraping, like JavaScript rendering and proxy management, so we can focus on extracting the data we need.
Step-by-Step Guide to Scrape Google Shopping
1. Build your payload with query parameters
First, start your scraping script with integration of downloaded libraries and initialize the our client with the provided API key.
from scrapingbee import ScrapingBeeClient
import pandas as pd
client = ScrapingBeeClient(api_key='YOUR_API_KEY')
Now we gonna define the rest of our scraper in the google_shopping_search function.
def google_shopping_search(search_query, country_code='us', start=0):
Let's take a closer look at these parameters:
search_query: the product to search for.
country_code: 2-letter country code to localize search.
start: pagination offset (e.g., 0 for page 1, 20 for page 2).
In this function, we define two python dictionaries that will store parsing guidelines ant attach them to a GET API call. After inspecting product data and CSS selectors, these sets should work on most extractions with set_1 usually matching the page structure. Each product has a defined selector – a product container which holds all relevant product data within its boundaries.
By defining selectors into lists of product details, and picking their appropriate HTML classes, we end up with parsing instructions that look like this:
set_1 = {
"products": {
"selector": ".MtXiu.mZ9c3d.wYFOId.M919M.W5CKGc.wTrwWd",
"type": "list",
"output": {
"name": ".rRCm8.sg5mJb .UC8ZCe.QS8Cxb .gkQHve.SsM98d.RmEs5b",
"price": ".rRCm8.sg5mJb .UC8ZCe.QS8Cxb .lmQWe",
"rating": ".rRCm8.sg5mJb .UC8ZCe.QS8Cxb .yi40Hd",
"reviews": ".rRCm8.sg5mJb .UC8ZCe.QS8Cxb .RDApEe",
"store": ".rRCm8.sg5mJb .UC8ZCe.QS8Cxb .WJMUdc.rw5ecc",
"delivery": ".rRCm8.sg5mJb .UC8ZCe.QS8Cxb .ybnj7e",
"rank": ".MtXiu.mZ9c3d.wYFOId.M919M.W5CKGc.wTrwWd@data-index", # rank starts from 0
"product_link": ".product_link@href"
}
}
}
set_2 = {
"products": {
"selector": ".sh-dgr__gr-auto.sh-dgr__grid-result",
"type": "list",
"output": {
"name": ".tAxDx",
"price": ".shntl .QIrs8",
"rating": ".Rsc7Yb",
"reviews": ".reviews",
"store": ".aULzUe.IuHnof",
"delivery": ".bONr3b > .vEjMR",
"features": ".F7Kwhf.bzCQNe.t5OWfd",
"link": "a.shntl@href",
"rank": ".sh-dgr__gr-auto.sh-dgr__grid-result@rank", # rank starts from 0
"product_id": ".sh-dgr__gr-auto.sh-dgr__grid-result@data-docid"
}
}
}
# Some regions have a different structure, so we have to use a different set of rule
extract_rules = set_1 #use set_2 is set_1 doesn't work
Now let's add the definition for the ScrabingBee's js_scenario parameter, that will prepare Google Shopping pages for scraping by cleaning styles, adding missing data like reviews and ranks, and generating clean product links. It ensures consistent structure across different layouts for accurate extraction.
js_scenario = {
"instructions": [
{"evaluate": rf"""
try {{
document.querySelector('.aULzUe.IuHnof > style')?.remove();
document.querySelectorAll('.qSSQfd.uqAnbd').forEach((e) => {{
const n_elm = document.createElement('p');
n_elm.className = 'reviews';
n_elm.textContent = e.nextSibling?.textContent || '';
e.append(n_elm);
}});
document.querySelectorAll('.sh-dgr__gr-auto.sh-dgr__grid-result').forEach((e, index) => {{
e.setAttribute('rank', index + {start});
}});
document.querySelectorAll('.sh-dgr__gr-auto.sh-dgr__grid-result a.shntl:first-child').forEach((e) => {{
e.href = e.href.replace(/.+\/url\?url=(.+)/g, '$1');
}});
}} catch(e) {{}}
try {{
(() => {{
const baseURL = 'https://www.google.com/search?start=0&q=laptop&gl={country_code}&udm=28#oshop=apv&oshopproduct=';
document.querySelectorAll('.MtXiu.mZ9c3d.wYFOId.M919M.W5CKGc.wTrwWd').forEach(item => {{
const gid = item.getAttribute('data-gid'),
mid = item.getAttribute('data-mid'),
oid = item.getAttribute('data-oid'),
iid = item.getAttribute('data-iid'),
rds = encodeURIComponent(btoa(item.getAttribute('data-rds') || ''));
if (gid && mid && oid && iid && rds) {{
let linkElement = item.querySelector('.product_link');
if (!linkElement) {{
linkElement = document.createElement('a');
linkElement.className = 'product_link';
linkElement.textContent = 'View Product';
linkElement.target = '_blank';
item.appendChild(linkElement);
}}
linkElement.href = `${{baseURL}}gid:${{gid}},mid:${{mid}},oid:${{oid}},iid:${{iid}},rds=${{rds}}&pvs=0`;
}}
}});
}})();
document.querySelectorAll('.rRCm8.sg5mJb .UC8ZCe.QS8Cxb .RDApEe').forEach((e) => {{
e.textContent = e.textContent.replace(/\((.+)\)/g, '$1');
}});
}} catch(e) {{}}
"""}
]
}
To integrate the desired product search into the webpage URL, this variable definition replaces empty spaces with a "+" sign:
q = search_query.replace(' ', '+')
Now it is time to define the API client within the "response" variable:
response = client.get(
f'https://www.google.com/search?start={start}&q={q}&gl={country_code}&tbm=shop',
params={
"custom_google": "true",
"wait_browser": "networkidle2",
"premium_proxy": "true",
"extract_rules": extract_rules,
"js_scenario": js_scenario,
'country_code': 'us'
},
retries=3
)
Note: The &tbm=shop parameter in a Google search URL stands for "to be matched" = shop, and it tells Google to return results specifically from Google Shopping.
Let's take a closer look at the parameters that define the GET API request:
custom_google: "true" – Instructs our API to use optimized settings for scraping Google pages.
wait_browser: "networkidle2" – Waits until the page has finished loading and no network activity is detected for at least 500 ms.
premium_proxy: "true" – Routes the request through a high-quality proxy for consistent access to the platform.
extract_rules – Defines what data to extract and how, using CSS selectors in a structured format.
js_scenario – Executes custom JavaScript on the page before scraping,
country_code: 'us' – Sets the location of the proxy to the United States to match U.S. search results.
retries=3 – If the request fails, ScrapingBee will retry up to 3 times automatically to avoid a hard failure.
2. Send a request using Python
Now, let’s send our request and get the response:
if response.text.startswith('{"message":"Invalid api key:'):
return f"Oops! It seems you may have missed adding your API KEY or you are using an incorrect key.\nGet your free API KEY and 1000 free scraping credits by signing up to our platform here: https://app.scrapingbee.com/account/register"
else:
return {
'start': start,
'products': response.json()['products'],
'info': f"{response.status_code} SUCCESS" if len(response.json()['products']) else f"{response.status_code} FAILED TO RETRIEVE PRODUCTS",
'next_start': len(response.json()['products']) + start
}
print(google_shopping_search(search_query='full frame camera', country_code='us', start=0))
When implementing a Google Shopping scraper Python solution, I always include proper error handling. Google Shopping’s structure can change, and having robust error handling helps identify and address issues quickly. This section closes the function definition and invokes it through the print function.
If everything is done correctly, your result should look like this:
The extraction is successful, but there is not much we can do without this data, so it is time to restructure into a readable and understandable CSV file.
3. Parse the JSON response
After successful parsing, let's adjust the function definition that builds a data frame thanks to pandas library, and exports it into a neatly organized data set.
products = response.json().get('products', [])
# Export to CSV
if products:
df = pd.DataFrame(products)
df.to_csv("google_shopping_output.csv", index=False)
print(f" Exported {len(df)} products to google_shopping_output.csv")
return {
'start': start,
'products': products,
'info': f"{response.status_code} SUCCESS" if products else f"{response.status_code} NO PRODUCTS",
'next_start': start + len(products)
}
# Example call
google_shopping_search(search_query='full frame camera')
Note: Google Shopping’s HTML structure can be quite complex. The selectors I’ve provided work as of 2025, but Google occasionally updates their structure. If you find that certain selectors no longer work, you may need to inspect the page again and update them.
4. Extract product title, price, and store
With this approach to Google Shopping scraping, we’ve extracted the essential product information and saved it in a structured format. This data is now ready for analysis, comparison, or integration with other systems.
Understanding Google Shopping Structure
Google Shopping is a powerful tool that collects product listings across multiple retailers. Its interface is visually rich and dynamically generated, with many data points like prices, ratings, delivery details, and product variants being loaded via JavaScript. However, these elements complicate automated extraction, because not all product details are loaded without interaction with JavaScript elements.
Thanks to our intuitive API, we can customize this process by offering built-in JavaScript rendering. You don’t need to configure a headless browser or deal with delays caused by dynamic loading. It loads the page as a browser would, executes the necessary JavaScript through the js_scenario parameter, and delivers the final rendered HTML ready for extraction.
Products are grouped into containers with distinct CSS classes. Each container holds multiple pieces of information like the product name, pricing, seller info, and delivery terms. ScrapingBee’s extract_rules lets you target these elements directly.
Search Results Page Overview
The Google Shopping search results page presents a grid of product cards. Each card includes:
Product name
Price
Seller name
Rating and review count
Delivery details
These cards are wrapped in containers with specific CSS classes like .sh-dgr__grid-result or .MtXiu. We already took care of these extractions through the definition of parsing parameters:
"selector": ".MtXiu.mZ9c3d.wYFOId.M919M.W5CKGc.wTrwWd",
Product Page Elements
When users click on a product, they’re taken to a detailed product page. The js_scenario parameter allows automated interaction, such as clicking to load all sellers or scrolling to trigger lazy-loaded elements.
Pricing Page Breakdown
On the product detail page, pricing is presented in tiers from different sellers. Each seller entry typically shows:
Seller name
Price
Shipping fee or delivery time
Return policies or ratings
These entries are often loaded dynamically as users scroll or interact with the page. We handle these cases through JavaScript rendering and wait conditions, enabling full visibility into all pricing options.
Scraping Product and Pricing Pages
To extract product and pricing data from Google Shopping, you need to handle two types of pages: search results and individual product detail pages. Both require JavaScript rendering due to dynamic loading and DOM updates.
ScrapingBee supports this natively. By setting render_js=true and defining js_scenario, the scraper scrolls the page, waits for content, and ensures full rendering before extraction. CSS selectors used in extract_rules help target the exact product, price, rating, and link elements.
Scrape Detailed Product Information
To extract specific fields like the product name, seller, price, rating, and delivery details, define a precise extract_rules structure. For example:
"output": {
"name": ".tAxDx",
"price": ".QIrs8",
"rating": ".Rsc7Yb",
"store": ".aULzUe.IuHnof",
"delivery": ".bONr3b > .vEjMR"
}
These fields are nested inside each product container. With our HTML API, these selectors are executed on the fully rendered DOM, ensuring you get accurate results.
Extract Pricing from Multiple Sellers
Google Shopping often lists the same product from different retailers. Each seller may offer a different price, delivery speed, or promo. Using JavaScript manipulation in js_scenario, you can loop through all seller blocks, assign a rank to each offer, and extract individual prices and seller details.
This allows you to build a structured table with multiple rows for the same product, differentiated by the seller and price, which is useful for market comparison or pricing intelligence.
"rank": ".MtXiu.mZ9c3d.wYFOId.M919M.W5CKGc.wTrwWd@data-index", # rank starts from 0
Handle JavaScript-rendered Content
Without JavaScript rendering, much of Google Shopping’s content won’t load. We handle it by executing scripts in the background, simulating a real browser, and ensuring that all products are captured in the final HTML.
Scaling Your Scraper for Multiple Keywords
Now that we have our basic Google Shopping scraper Python implementation working, let’s scale it to handle multiple search terms and organize the results efficiently.
Loop through multiple search terms
To target multiple products at the same time, let's use a simple example that will use two additional Python libraries – "asyncio" and "functools" . First, let's import them at the beginning of your script:
import asyncio
import functools
import concurrent.futures
By adding this chunk of code, we can run 3 concurrent extractions at the same time, split into different files. This way, it is very easy to scale your scraper, either by adding a prompt on how many URLS to target, or duplicating lines and defining different queries. How convenient!
async def main():
loop = asyncio.get_event_loop()
with concurrent.futures.ThreadPoolExecutor() as executor:
tasks = [
loop.run_in_executor(None, functools.partial(google_shopping_search, 'full frame camera', filename='camera.csv')),
loop.run_in_executor(None, functools.partial(google_shopping_search, 'gaming laptop', filename='laptop.csv')),
loop.run_in_executor(None, functools.partial(google_shopping_search, 'smartwatch', filename='watch.csv')),
]
await asyncio.gather(*tasks)
if __name__ == "__main__":
asyncio.run(main())
Create separate CSVs for each keyword
By looping through a desired number of keywords, our code organizes a new CSV file for each keyword by using functools.partial to call the same google_shopping_search function with a unique filename argument for each search term. Each task runs in parallel and saves its results into its own CSV file (e.g. camera.csv, laptop.csv, watch.csv), keeping the data separated by keyword.
Avoid getting blocked with proxies
Using proxies is essential in web scraping to avoid getting blocked by websites that detect and limit repeated requests from the same IP address. Proxies rotate IPs and simulate traffic from different locations, helping scrapers appear more like real users.
Through integration of parameters within our documentation, you can enable built-in proxy support, automatically managing IP rotation and geolocation, allowing you to focus data extraction with consistent and anonymous connections.
Get Started with ScrapingBee Today
After developing numerous Google Shopping scraper Python solutions, I can confidently say that ScrapingBee offers the most reliable and efficient approach. Here’s how to get started:
Sign up for an account at ScrapingBee.com
Get your API key from the dashboard
Install the Python client with pip install scrapingbee
Start scraping using the code examples in this guide
ScrapingBee’s Google Shopping Scraper API eliminates the common headaches of web scraping:
No need to manage proxy rotations
No JavaScript rendering issues
No CAPTCHA solving required
No need to maintain browser instances
Our API is designed to handle all the complexity of Google Shopping scraping, so you can focus on using the data rather than struggling to extract it. With competitive pricing plans starting at just $49/month for 1,000,000 API credits, you’ll find a solution that fits your needs whether you’re a small business or an enterprise.
Frequently Asked Questions (FAQs)
Is it legal to scrape data from Google?
Web scraping itself is not illegal, but how you use the data matters. Always respect Google’s Terms of Service and robots.txt file. Generally, scraping publicly available data for non-commercial, research, or personal use is considered fair use. However, for commercial applications, consult with a legal professional regarding your specific use case. Our API provides the tools, but users are responsible for using the data legally and ethically.
How do I get Google Shopping data?
The most reliable method to extract Google Shopping data is through ScrapingBee’s API. Our approach handles the technical challenges of JavaScript rendering, proxy management, and CAPTCHA avoidance. Simply follow the code examples in this guide, which demonstrate how to search for products, extract pricing information, and collect detailed product data from Google Shopping.
Does Google have a web scraper?
Google does not provide an official web scraper or API for Google Shopping data. While they offer APIs for other services like Search Console or Analytics, Google Shopping data must be accessed through web scraping techniques, and our HTML API provides a dedicated solution for this purpose.
Kevin worked in the web scraping industry for 10 years before co-founding ScrapingBee. He is also the author of the Java Web Scraping Handbook.
