---
title: "How to Parse HTML in Ruby with Nokogiri?"
description: "Learn to parse HTML with Ruby and Nokogiri with this step-by-step tutorial. We will see the different ways to scrape the web in Ruby through lots of example with the Nokogiri library."
url: "https://www.scrapingbee.com/blog/parse-html-nokogiri/"
canonical: "https://www.scrapingbee.com/blog/parse-html-nokogiri/"
date: 2026-01-16
lastmod: 2026-01-16
author: "Aniket Bhattacharyea"
publisher: "ScrapingBee"
---

# How to Parse HTML in Ruby with Nokogiri?



APIs are the cornerstone of the modern internet as they enable different services to communicate with each other. With APIs, you can gather information from different sources and use different services. However, not all services provide an API for you to consume. Even if an API is offered, it might be limited in comparison to a service’s web application(s). Thankfully, you can use web scraping to overcome these limitations. *Web scraping* refers to the practice of extracting data from the HTML source of the web page. That is, instead of communicating with a server through APIs, web scraping lets you extract information directly from the web page itself.

Web scraping is often more accessible than using an API because with an API, you have to complete several additional steps: you need to get an API key, set up your script to make a request to the API with the correctly formatted request body, and finally, parse the response body to get the data you want. Then add in the time spent to read the API documentation and figure out the necessary details about the API, and you can see how this quickly becomes a hassle. With web scraping, however, you simply need to look at the HTML structure of the web page and determine which elements you want to extract the data from.

In this article, you’ll learn how to parse HTML from a web page in Ruby using the Nokogiri gem. You can find the code for this article in this [GitHub repo](https://github.com/heraldofsolace/Nokogiri-demo).

![cover image](https://www.scrapingbee.com/blog/parse-html-nokogiri/cover.png)


## What Is Nokogiri?

[Nokogiri (鋸)](https://nokogiri.org/index.html) is a lightweight Ruby gem that makes it easy to parse, build, and search HTML and XML documents. Nokogiri provides a DOM parser, SAX parser, and push parser, as well as the ability to search documents using Xpath and CSS3 selectors.

## How to Extract Text from a Web Page Using Nokogiri

In this tutorial, you’ll scrape [this web page](https://www.scrapethissite.com/pages/forms/), which provides a database of NHL team stats since 1990. You’ll build a web scraper that will let the user enter a search term and then display the stats for all teams matching the search.

Before you get your hands dirty with coding, spend some time on the web page. Specifically, you need to know how to find the elements you want in the HTML source code using CSS selectors.

Your first task is to get familiar with the way the web page works. Since you want to perform a search, you need to figure out what happens on the web page when you do so. Go ahead and enter a search. You’ll see that it redirects to a URL that looks like this:

```
https://www.scrapethissite.com/pages/forms/?q=<your_search_term>
```

This tells you the URL you need to scrape. Let’s get started!

To follow along with the tutorial, create a directory:

```shell
mkdir nokogiri-demo
cd nokogiri demo
```

Then, install Nokogiri:

```shell
gem install nokogiri
```

Create a file called **sscrape.rb**. Import the `nokogiri` and `open-uri` modules. The `open-uri` module is a convenience wrapper around the `net/http` module.

```ruby
require 'nokogiri'
require 'open-uri'
```

Add the following lines to input the search term from the user:

```ruby
puts "Enter a search term"
search_term = gets.chomp
```

Once you have the search term, you need to make a request to `https://www.scrapethissite.com/pages/forms/?q=<search_term>`. The `URI.open` method makes the request and returns the HTML of the page, which is then passed to the [`Nokogiri::HTML5`](https://nokogiri.org/rdoc/Nokogiri/HTML5.html) class that returns a [`Document`](https://nokogiri.org/rdoc/Nokogiri/HTML4/Document.html):

```ruby
doc = Nokogiri::HTML5(URI.open("https://www.scrapethissite.com/pages/forms/?q=#{search_term}"))
```

To extract text from a particular element on the page, you first need a way to select the element. Using a CSS selector is the easiest way to go about this.

To look at the source code of the page, press `CTRL+Shift+I` to open up the developer console and click on the tiny mouse pointer:

![Developer console](https://www.scrapingbee.com/blog/parse-html-nokogiri/0XIQouX.png)


![Developer console unannotated](https://www.scrapingbee.com/blog/parse-html-nokogiri/0XIQouX.png)


You can now hover your mouse over any element, and it will be highlighted in the HTML source. Hover over a single team to find out its HTML structure.

You can see that the data is inside a `<table>` tag, where each team is in a separate `<tr>` tag. Each team also has a class `team`.

![The team element](https://www.scrapingbee.com/blog/parse-html-nokogiri/f5MzCFX.png)



From this, you can infer the CSS selector required to select the data: `tr.team`. The `css` method takes the CSS selector and returns a [`NodeSet`](https://nokogiri.org/rdoc/Nokogiri/XML/NodeSet) of elements matching the query. Then you can simply iterate over the elements using the `each` method.

For every team, their name is stored in a `<td>` tag with a class `name`. Since there is only one element with class `name` inside each team, you can use the `at` method to get the element. Like the `css` method, the `at` method finds elements with a given CSS selector but returns only the first element.

Once you have the element, you can extract its text using the `content` method:

```ruby
doc.css("tr.team").each do |team|
  puts team.at(".name").content.strip
end
```

Run the code using `ruby scrape.rb`, and enter a search term when prompted. You should see the names of the teams matching the search query:

![Search results](https://www.scrapingbee.com/blog/parse-html-nokogiri/9liCNHx.png)



Let’s go one step further and extract more information. How about the total wins of each team?

Going back to the page source, the number of wins for each team is stored in a `<td>` element with a class `wins`. Like the name, you can use the `at` method to extract it. Here’s the code to do so:

```ruby
require 'nokogiri'
require 'open-uri'

puts "Enter a search term"
search_term = gets.chomp

doc = Nokogiri::HTML5(URI.open("https://www.scrapethissite.com/pages/forms/?q=#{search_term}"))

teams = {}

doc.css("tr.team").each do |team|
  team_name = team.at(".name").content.strip
  team_wins = team.at(".wins").content.to_i

  teams[team_name] ||= 0
  teams[team_name] += team_wins
end

teams.each do |team_name, team_wins|
  puts "#{team_name} => #{team_wins}"
end

```

Here, a hash named `teams` has been defined to store the scores. First, the code checks if the team name is already in the hash. If it is, the number of wins is added to the existing score. If it isn’t, the score is initialized to 0.

Run the script and enter a search term. You will see the total wins of each team:

![Output of the script](https://www.scrapingbee.com/blog/parse-html-nokogiri/N1iQAtx.png)



### Handling Pagination

Your code can now successfully extract text from the web page. But this is just the first page. Let’s take the script to the next level by adding the ability to handle pagination.

The code starts similarly—by importing the required modules, prompting the user for a search term, and making a request to the first page:

```ruby
require 'nokogiri'
require 'open-uri'

puts "Enter a search term"
search_term = gets.chomp

doc = Nokogiri::HTML5(URI.open("https://www.scrapethissite.com/pages/forms/?q=#{search_term}"))
```

To handle pagination, you need to know the total number of pages. A quick peek to the page source reveals that the links to the pages are stored as `<a>` tags inside `<li>` tags inside a `<ul>` tag with a class `pagination`.

The number of pages is the last number that occurs in this list, which is the second to last `<li>` tag. (The last `<li>` is the link to the last page but doesn’t contain the page number.)

The following code extracts the second to last element as well as the page number:

```ruby
pages = doc.css(".pagination a")[-2].content.to_i
```

As before, create a `teams` hash:

```ruby
teams = {}
```

Then create a loop that will run from `1` to `pages`.

```ruby
(1..pages).each do |page_num|
  puts "Scraping page #{page_num}"
end
```

This time, the URL will be slightly different. If you visit any page (other than the first one) in your browser, you’ll see a `page_num` query parameter added to the URL. This parameter is the page number you want to view. You’ll land on the first page if you set `page_num` to `1`. This is a huge relief, as you don’t need to handle the first page differently.

Add the following code to the `each` loop. Note that this code is the same as what you did previously. The only difference is that this time, it is executed for each page.

```ruby
(1..pages).each do |page_num|
  puts "Scraping page #{page_num}"
  doc = Nokogiri::HTML5(URI.open("https://www.scrapethissite.com/pages/forms/?q=#{search_term}&page_num=#{page_num}"))
  doc.css("tr.team").each do |team|
    team_name = team.at(".name").content.strip
    team_wins = team.at(".wins").content.to_i

    teams[team_name] ||= 0
    teams[team_name] += team_wins
  end
end
```

Finally, print the result:

```ruby
teams.each do |team_name, team_wins|
  puts "#{team_name} => #{team_wins}"
end
```

Here’s the complete code:

```ruby
require 'nokogiri'
require 'open-uri'

puts "Enter a search term"
search_term = gets.chomp

doc = Nokogiri::HTML(URI.open("https://www.scrapethissite.com/pages/forms/?q=#{search_term}"))

pages = doc.css(".pagination a")[-2].content.to_i

teams = {}

(1..pages).each do |page_num|
  puts "Scraping page #{page_num}"
  doc = Nokogiri::HTML(URI.open("https://www.scrapethissite.com/pages/forms/?q=#{search_term}&page_num=#{page_num}"))
  doc.css("tr.team").each do |team|
    team_name = team.at(".name").content.strip
    team_wins = team.at(".wins").content.to_i

    teams[team_name] ||= 0
    teams[team_name] += team_wins
  end
end

teams.each do |team_name, team_wins|
  puts "#{team_name} => #{team_wins}"
end
```

Run the code, and you’ll see that it now scrapes each page and provides each team’s total number of wins:

![Screenshot_20220523_221806](https://www.scrapingbee.com/blog/parse-html-nokogiri/o1GyDkD.png)



To check out the final code, you can access this [GitHub repository](https://github.com/heraldofsolace/Nokogiri-demo).

## Conclusion

As you’ve seen, web scraping is a powerful tool that you can use to extract text from a web page. You can use it in the absence of an API, or you can use it to overcome an API’s limitations. In this tutorial, you learned how to use the Nokogiri gem to scrape a web page and extract text.

Even though Nokogiri is lightweight and simple, there are some use cases that are outside its scope. For example, if you’re scraping a dynamic web page that runs JavaScript, Nokogiri will not be able to do the job because it can’t execute JavaScript. In that case, a sophisticated tool like [Watir](/blog/scraping-watir-ruby/) may be more suitable.

If you’re looking for a fast, powerful, and easy-to-use web scraping tool, be sure to check out our [scraper API](https://www.scrapingbee.com/). It handles headless browsers at scale and rotates proxies, giving you an unmatched and uninterrupted web scraping experience.

Before you go, check out these related reads:
- [Getting Started with HtmlUnit](https://www.scrapingbee.com/blog/getting-started-with-htmlunit/)
- [10 Tips on How to make Python's Beautiful Soup faster when scraping](https://www.scrapingbee.com/blog/how-to-make-pythons-beautiful-soup-faster-performance/)

