Finding new keyword ideas is easy. There are plenty of tools, ours included, that can generate thousands of topic-specific keywords at the click of a button. The problem is, you can't afford to create a page for each of these keywords — you have to decide which few are worth your effort.
Keyword difficulty is here to solve this problem and help you prioritize your list of keywords by how likely they are to succeed.
What is keyword difficulty?
Keyword difficulty is an estimate of how much effort it would take to rank your page among the top ten search results. The estimate is made by evaluating each of the top-ranking pages for a given query and seeing whether they have any weaknesses that you can exploit. The weaker the pages, the higher your chances of ranking for this query, the sooner you should create a page for the keyword.
How to calculate keyword difficulty?
The first thing to understand about calculating keyword difficulty is that it's not an exact science. While we have a pretty good idea of what makes a strong or a weak page, we don't have access to the actual search engine algorithm. So it's up to us to decide which factors to consider when calculating keyword difficulty, but there is no guarantee that a search engine uses the exact same factors.
The second thing to understand about calculating keyword difficulty is that there is no single approach. Most keyword research tools use their own formulas to calculate a difficulty score. Likewise, most SEO professionals have their own routines to do the same. Ultimately, none of the methods are better or worse — choose a method you identify most with and see if it works.
In this article, we will cover a two-stage process for evaluating keyword difficulty. In stage one, we will use Rank Tracker's difficulty score to filter massive keyword lists. In stage two, we will establish the keyword's difficulty by looking at those pages that already rank for it in SERP.
Stage 1. Filter keyword lists using a difficulty score
For the purpose of this article, let's assume that you've already used Rank Tracker to generate a list of keywords and now you need to evaluate them for ranking potential:
Select target keywords
Though Rank Tracker has the capacity to discover thousands of keywords on any topic, only a fraction of those keywords is actually worth pursuing. So, before you begin evaluating each keyword for its ranking difficulty, it's a good idea to rid the list of those keywords that you are not interested in.
For this example, let's say you've used Rank Tracker to find keyword ideas around the topic of coffee and ended up with almost 7,000 keywords. Now, this is entirely too many and you can already see that the list includes some irrelevant keywords, so you have to do some cleaning:
- Go to Keyword Research > Keyword Sandbox — this is where we store all of your keyword ideas.
- Click on the filter icon in the top right corner.
- Create a filter to remove keywords with a low search volume, let's say anything under 250 searches per month (feel free to use your own benchmarks).
- Create a filter to remove keywords with a difficulty score of over 40 — go higher if you are an established brand and lower if you are new.
- Create a filter to remove keywords that contain only one or two words — those are usually too generic to deliver a quality audience.
- Create a filter to remove keywords that contain irrelevant words — I always remove keywords that contain competitor brands and irrelevant locations.
With these filters alone, you would be able to change the number of keywords from several thousand to several hundred.
And now that your list is down to a manageable size, you can go over it and pick keywords for an in-depth analysis of ranking difficulty. Right-click selected keywords and choose to move them to rank tracking, which is where we keep those keywords that you are actively pursuing.
Evaluate keyword difficulty
Once you've selected those keywords you find promising, you can take a much closer look at their ranking difficulty. Go to Target Keywords > Rank Tracking and find a list of target keywords in the upper half of the screen:
Select any keyword, click Keyword Difficulty, and you will get detailed keyword difficulty metrics in the lower part of the screen:
Metrics include difficulty scores for each of the top ten ranking pages as well as the information we use to calculate those scores: InLink Rank, Domain InLink Rank, Sites Linking to Page, and Sites Linking to Domain. Let's go over a few examples of how you can use these metrics to decide whether to pursue a keyword.
A keyword with one of the lowest difficulty scores on the list is best coffee beans for espresso. It has a search volume of almost a thousand and an overall difficulty score of 17.8, which is promising. Let's look at the other metrics to see if it's actually so:
We can see from the table that ranking pages are indeed quite weak, although some of them come from above-average domains. In particular, positions #3, #6, and #10 all have a solid domain strength, defined by the number and diversity of backlinks.
We can also see that some of the results have rich features, like reviews and featured snippets. It means that they have applied schema markup on their pages and it would be advisable for you to do the same if you want to beat them.
Other than that, this particular SERP looks pretty approachable. Most of the pages have no means of defending their positions, and even those that do are not very convincing. All you have to do to beat them is have an okay domain strength and apply schema markup. And if you were to open some of the ranking pages, you would see that they are written in a generic manner and usually include a listicle of coffee products and some background information — not much effort content-wise as well.
Here is another keyword with a relatively low difficulty score of 34.4 and a search volume of one thousand. Let's see whether it's also worth your effort.
The first thing that's immediately noticeable is that nine out of ten search results have rich snippets, which means that you must use schema markup in order to compete for this type of query:
The second thing that's immediately noticeable is that most of the search results come from powerful websites with thousands of backlinks. Some of those websites, like YouTube, Starbucks, and AllRecipes are easily recognized by mainstream users to be highly authoritative:
In fact, the only reason this keyword has a low difficulty score is that the pages themselves do not have that many backlinks. But, looking at domain strengths and the amount of markup involved, it would probably take a tremendous amount of effort to rank for this keyword. You should only attempt it if you have a powerful website.
Stage 2. Evaluate keyword difficulty using search results
Once you've used Rank Tracker to shorten your list of keywords, you can make your own estimate of keyword difficulty by looking at search results. All you have to do is google your query and see whether you are likely to rank among the top ten pages. Here are some of the things to pay attention to:
The best way to dethrone a top-ranking search result is to create better content. Take a look at ranking pages and see whether they provide a full answer, whether the facts are up to date, whether they have good structure, whether there are some extras that could enhance the information, whether you have unique expertise on the subject. The more ways you find to create better content, the lower the keyword difficulty.
In one of the examples above, I was looking for best coffee beans for espresso. All of the top-ranking pages for this query are listicles of coffee products, but some of them also have guides on how to make the right choice. Not surprisingly, the pages that have guides rank higher than those that don't — it's better content.
Don't quote me on this, but, all things being equal, an optimized snippet will always win over a clumsy one. So, whenever you see a snippet that doesn't follow best practices, its position is yours for the taking.
Above is one of the top search results for best italian bikes and its snippet is fairly poorly optimized: weak title structure, low impact words in the title, not a single exact keyword match, and the description seems to be autogenerated. And when I look at the actual meta description in the page's source, I can see why Google decided not to use it — it's twice as long as it should be, first two sentences are off-topic, and it contains an old date.
Now, results like this are a great opportunity. If you can create a page of similar quality and have a domain of comparable strength, then all it would take to rank higher is a properly optimized snippet.
It is practically useless to rank for the types of queries that return a lot of SERP features. Even if you manage to rank in the first position, the majority of your traffic is going to be stolen by Google panels placed just above your snippet. Some queries even return several panels placed throughout the SERP, making it even harder for an organic result to get noticed.
Google panels are almost guaranteed to appear whenever you search for famous people, paintings, books, and movies, but also whenever you explore varieties of generic things, like coffee types, bird species, or popular national dishes. Which is why it is better to move away from generic keywords and pursue the ones with a longer tail.
Google uses a number of tells to figure out whether we want to find, learn, or buy something — this is called search intent. For example, the query best full frame camera has the word best in it, so the intent is probably to choose a camera from several top models and the best type of article to match this intent is a listicle. Also, since we are talking about a rapidly evolving technology, you would probably want the latest information, which is why most search results have the current year in the title:
In the image above, you can see that one of the results stands out from the rest — it is neither a listicle nor does it contain any indication of recency, and, despite what it says in the title, there are no top picks in the article. So, this result will probably be on its way out as soon as something better comes along.
The idea is somewhat similar to search intent, except you look at the intent of the entire website, not just a single page. If most ranking pages come from one type of website, say an online retailer, then other types of websites are less likely to rank for this query.
Let's say you sell bikes, but a part of your catalog is also dedicated to bike maintenance, things like spare parts and tools. Naturally, you'd want to rank for these types of keywords. Now, if you were to google bicycle maintenance, you'd see that almost all of the top results come from bike reviewers and biking enthusiasts. But, if you were to google bicycle spare parts, all of the top results come from retailers like yourself. The latter would be much easier to rank for and would deliver a better crowd.
How to apply keyword difficulty?
Whether you go by keyword difficulty score or by your own judgment, it's important to remember that keyword difficulty is relative.
If you have a powerful website, with tons of quality backlinks, then there is no reason for you not to pursue even the most difficult of keywords. If your website is not as strong, then perhaps start with easier keywords, but do not disregard difficult ones completely. Create pages for difficult keywords, keep them updated, keep adding new information, and, eventually, you will catch up to your competitors.
Think of keyword difficulty as a way to prioritize your list of target keywords, perhaps even a way to find low-hanging fruit — if you are an SEO agency looking to kick the project off with some quick wins. But, long term, if a keyword is important to your website, then its difficulty score should not discourage you from using it.
Assessing keyword difficulty is one of the few parts of SEO where you can still get your hands dirty. I mean, yes, you can use Rank Tracker to calculate a difficulty score and apply it to filter your keyword list, but there is no substitute for your own judgment when it comes to making a final selection of target keywords. And that's the fun part, isn't it?
By: Andrei Prakharevich