Power Up Your Marketing Workflows with Python & JavaScript

What SERP features does my competitor have that I don’t?

There are various tools we can use to analyze this, such as SEMrush reports to view each feature at a time, but what if we have to repeat this process every month, or say for multiple competitors? 

We’ll learn how to leverage the SEMrush API with Python and JavaScript to:

1. Pull data programmatically from SEMrush
2. Slice and dice the data
3. Export the results to a CSV for further analysis

But why would you want to learn that yourself? Why should SEO’s learn to code other than HTML?

• Do more exciting work
• Increase productivity
• Gain credibility
• Greater job security

What is the minimum Python knowledge you need?

Typically, Hamlet would recommend tutorials and courses such as Coursera or DataCamp. However, from the feedback he’s received, he has learned that these resources are more designed for engineers or someone looking to go into a programming career. For marketers, that’s not usually the case. 

Instead, Hamlet came up with a resource crafted specifically for marketers—An Introduction to Python for SEO Pros Using Spreadsheets.

In this article, Hamlet focuses on a familiar task for marketers, using spreadsheets. This foundation in Google Sheets can be easily adapted with Python. Hamlet’s approach was to use code from a Python notebook written by John Mueller of Google, which determines what type of content performs better on mobile vs. desktop. Line-by-line, Hamlet walks through the code in detail to explain how everything works, while taking the opportunity to introduce the building blocks of Python.

SERP features 101

Using the Position Tracking tool in SEMrush, you can see a list of SERP features, such as featured snippets, local packs, reviews, sitelinks, etc. that create a useful experience for end-users when completing a search. 

Mastering and keeping track of which features are used by your competitors is crucial. 

Competitive comparison

In this example, we can compare the SERP features of AutoZone and Advanced Auto Parts. 

After comparing the two reports, we can see that AutoZone has 3,000 more featured snippets, 162,000 more reviews, 34,000 more sitelinks, and performs better for most features except for top stories where they are both tied.

This type of comparison is not a standard report. 

You may be able to find capabilities and workflows that you don’t have immediately and turn it into a competitive advantage. If you can figure out a process that no one else knows and automate it with programming, you can create a very valuable asset.

How can we repeat this valuable comparison over time or across different competitors?

Hamlet created a Colab notebook that takes this workflow and puts it all together. Using previous code that he has written, Hamlet makes small modifications until it can successfully perform what he wants it to do. 

Here’s an example. 

In the notebook, the code is hidden and replaced by a form to make it easier and less complicated for users. 

After you input the information and press run, it will go line-by-line through each step to reach the output.

The output is a CSV file that was downloaded to the computer. 

The CSV files can then be imported into Google Sheets. Below are the sheets for Advanced Auto Parts and AutoZone. 

This workflow can be completed for any number of domains and can even accept a list of domains as well.

The code behind the scenes

Now let’s analyze the code a bit to see how we accomplished this. 

From the article, Hamlet copy and pasted the code he had already developed and made a small change. The original code pulled a number of columns from SEMrush, but in our case, we only needed one, the SERP features.

You can learn more about the columns from the API documentation.

When we tested the changes made, we found something interesting. The SERP features we get back from the API are not names, but instead numbers that identify each feature.

Since we may not know what the numbers stand for, we had to translate these features into their names. We accomplished this automatically using a JavaScript snippet from another one of Hamlet’s previous articles. 

This code can be adapted for this specific use case. All we have to do is copy a selector from the page that allows us to address an element on it.

Here we have to modify the selector so that it captures the whole column instead of just the single element.

While the page is open, copy and paste the code into the Developers Tools Console and then call the function we created.

And that’s it. Now we have all the elements!

Watch the full video from Hamlet: 

Resources to learn more

Automate repetitive tasks

Understand and solve complex problems

Write compelling stories from hard-to-find insights 

Invent new creative and effective processes

Future proof your SEO job