Custom Python scripts are much more customizable than Excel spreadsheets. This is good news for SEOs — this can lead to optimization opportunities and low-hanging fruit. One way you can use Python to uncover these opportunities is by pairing it with natural language processing. This way, you can match how your audience searches with your...READ POST
Whether it’s used on an e-commerce site or a forum, pagination is the process of dividing category pages into several smaller sets in order to display information in a more organized manner. While pagination is a great and efficient technique, SEO issues can often arise within Content Management Systems (CMS), specifically with canonicalizing these paginated pages to the first page in the set.
This issue is listed by Google as the most common canonical mistake seen in practice. This can lead to the loss of important content being de-indexed (or never indexed) from a search engine.
It sounds bad but no need to worry! This tutorial is meant to help webmasters control these issues. With the RankSense app, as well as a site crawler, this problem can be easily fixed. After it has been fixed, you can then use the RankSense app to track the effectiveness of the solution.
Table of Contents:
- Why does the issue matter?
- “View All” Canonical Tags
- Using RankSense and a Crawler to Fix the Issue
- RankSense Insights
Why does the issue matter?
Content Management Systems oftentimes are not optimized for SEO. As a result, many of them will often canonicalize all of their paginated pages to the first page in the set. This can lead to bigger problems. During a crawl of a website, the search engine looks at different signals in order to make a decision on whether a page should be indexed or not. These signals include canonical tags on the pages themselves as well as the internal links used within the site.
If all of the canonical tags on the paginated pages on the site point to the first page in the set and the search engine decides to follow these canonical tags, eventually these paginated pages will be “folded” into their first page. This means that all of the content on these pages is essentially lost since that content was dropped from the index because the canonical tags told the search engine that the first page contained the same information. Additionally, if there are no other internal links pointing to the products/blog posts/etc on these paginated pages, it will be difficult for this content to be indexed and show up in search results as there will be no internal links pointing to it.
“View All” Canonical Tags
This issue can be fixed quickly using a “view-all” canonical tag. A “view-all” page is meant to display all the information from each paginated page on one page.
We can then replace all of the canonical tags on paginated pages that point to the first page in the set with canonical tags that point to the ‘“View-all” page. Search crawlers will now be able to crawl one page and access all of the content that should be indexed for the category.
However, the view-all page may not be beneficial for every site. For example, if an e-commerce site has a category with 20 products a page with 50 total pages, 1,000 items will be displayed on the view all page. This may slow down loading time, and in this case, self-referential canonical tags may be a better solution instead. Read more on that here.
(Note that many CMS systems include View All page options. If A View All option is not available for your CMS, the self-referential canonical solution may be better. Remember that the load time of the page is more important than the amount of products displayed on the View-All. If the load time of the View-All page is not reasonable, do not implement this View All solution.)
Here is the great part about RankSense: you will be able to track the effectiveness of your chosen solution and decide whether or not it was the best option for the site.
Using RankSense and a Crawler to Fix the Issue
It’s a good idea to use a website crawler in order to find the paginated URLs on the site, as manually reviewing the website for every URL can be a lengthy process. For the purpose of this article, we will be using Screaming Frog.
Step 1: Crawl the Website
At the top of Screaming Frog, input your homepage URL as indicated by the first arrow. Then click “Start.”
Step 2: Find and Interpret the pages’ Canonical Tags
On the top bar, click on “Canonicals”. After “Canonicals” is selected, look for “Canonical Link Element 1.” This element includes the URL that the canonicals tags on the page are pointing to.
Step 3: Find Canonicals Needing To Be Updated
From this list, check the URLs of paginated pages and check where their canonicals are pointing to. If they are pointing to the first page of their category, you can copy and paste those URLs into a Google spreadsheet.
Step 4: Implementing Changes Using RankSense
Previously, you should have copy and pasted all of the URLs needing this fix into a Google spreadsheet. RankSense implements rules using these spreadsheets. Check this template here to see how your spreadsheet should be formatted.
With the URLs you wish to fix in the first column, you should place your view-all URLs in the second column. This way, all of your paginated URLs will have a canonical tag pointing to their equivalent view-all page. With your rules sheet complete, it’s now time to tag and track changes using RankSense Insights.
Note that since paginated categories can often grow or shrink, depending on the category having new products added or old products removed, these paginated URLs can become outdated.
Thankfully RankSense supports wildcards to help with this. Instead of having to create a rule for every single paginated URL, you can create one rule that fits the URL structure of the paginated page. For example, www.ranksense.com/categogry?page=* will cover every single paginated page for the category. Learn more about wild cards here.
A key feature of the RankSense app is the ability to tag and track the performance of your pages using RankSense Insights. This information can help you determine whether or not the changes made were beneficial.
To begin, head over to dash.ranksense.com and select the website you have RankSense installed on. Then navigate over to “Settings,” where there will be a tab for “SEO Rules.”
Next, click on “+ File,” and input your spreadsheet URL. Beside that you will see the associated tags. These are crucial for tracking changes. For this specific case, use the following tags:
- Issue – Paginated pages not indexed
- Solution – Add Correct Canonicals
- Affected – Category pages
Click on “Import”, and then publish to your desired branch (it is recommended to publish to staging first).
If you have never published SEO rules through RankSense before, we recommend checking this article first.
Tracking The Results
After your sheet has been published, navigate to the “Insights” tab at the top.
Here, there are plenty of configurations to create a custom report specific to the rules you are interested in. In the bottom left, you can select the tags you wish to track or even the entire spreadsheet. Located above the graph are options to change the search engine the data is coming from as well as to select a time period.
(URLs have been blurred for privacy concerns)
You can see the last time these URLs were crawled under the “Latest pages crawled” section of the “Dashboard” tab. Once you see that your preferred search engine has crawled the pages you fixed, you can begin to study the effects your changes had on the site.
(Note that for these Insights results, we are actually expecting the traffic for all of 1st page paginated pages to drop while the View-All pages’ traffic should greatly increase. This should also help the products/blog posts/etc. on these paginated pages perform better, which will not be tracked in this report.)
- View-all canonical tags can be more efficient for paginated pages as they allow search engines to see all the content each category has to offer on one page.
- A website crawler can be used to gather the paginated URLs needing to be fixed, shortening the time needed to implement changes.
- The RankSense app allows for a simple way to implement, tag, and track changes.