Advanced SEO's Guide to Keyword Research
A framework for smarter keyword selection & targeting
Keyword research is so often talked about it's almost a cliche. But is it really the simple 5-step process so many blog posts claim it to be? For a tiny site with some 10 target terms, it may be — and that's about it.
Before you start reading, please have a look at the keyword research report below.
If you are totally comfortable with it, this article is not for you. You'll probably have more fun if you grab a coffee and have a look at the other posts on our SEO blog.
What's wrong with this kind of keyword research reports?
Sincerely — nothing, as long as you don't go beyond 20-30 keywords and your website is fairly small. The more entries you add, the less meaningful such reports become. They can't answer any of the questions you should be asking yourself when you do keyword research:
- What are your priority keywords? Why?
- How hard is it to get to page 1 in Google for any given term on the list?
- Why are some keywords losing rankings?
- What keywords should go in the title tag of your home page?
You name it.
But it's not all lost. Data segmentation will help you answer these questions, make bulky keyword lists manageable, and the entire process productive. In this guide, I'll share my strategy for reworking the traditional keyword research routine to make it meaningful, and the data you get actionable. If you follow all the steps below, you'll end up with a long list of profitable terms mapped to your landing pages, and a clear understanding of the traffic and conversions these keywords will bring.
1. Collect the data
At this stage, our goal is to collect as much data as possible. Sure thing, we also want to do it fast.
Before we start, grab a piece of paper and jot down the phrases and keywords that best describe the products or services that you offer. If you run an ecommerce website, fetch product names and product category names from your catalogue.
Done? We're good to go!
Start with the terms you already rank for
If your website is not brand new, then, most likely, it ranks for some keywords already. You can get the list of these keywords from Google Search Console. Log in to your account and go to Search Traffic > Search Analytics. Check the Impressions checkbox. Set the Dates filter to show 90 days of historical data. Then click Download to export the report in CSV.
Next, start Rank Tracker and create a project. Click on the Keyword Research icon in the left hand navigation pane. Then hit the Suggest Keywords button and select Google Search Console to import the CSV file you just downloaded.
Get suggestions from the good ol' Keyword Planner and other tools
You'll see your workspace populate with the keywords you already rank for. Now, hit the Suggest Keywords button once more, this time selecting Google AdWords Keyword Planner. Specify your Google AdWords login and password, and enter the seed terms you jotted down at the prep stage. Try to use terms related to the same topic at a time to make sure you get relevant suggestions.
Note: In Rank Tracker, AdWords targeting settings are set to USA/English by default. If you need to change those, go to Preferences > Google Keyword Planner Settings in the app and specify a different target country and language.
Feel free to repeat the process using one of Rank Tracker's twenty keyword research methods (or all of them). I would recommend you to start with Google Autocomplete. Use your initial keyword list and the keywords you pulled from Google Search Console as the seed terms.
For each keyword research task, Rank Tracker will show the suggestions along with search volume, competition, expected visits and KEI. At the final step, you may sieve out keywords that don't meet your goals. However, for now I'd recommend to keep all of them — we'll do lots of filtering later on.
2. Sanitize the data
When you're done with the research, you'll probably have thousands of potential keywords in your Rank Tracker workspace. Not all of these keywords may be a good match for your website. Our next goal is to refine the raw keyword list and reduce the amount of manual work at later stages.
There are two quantitative - and therefore handy - criteria that one may use to filter out useless keywords. These are the number of words in your key phrase and the search volume.
Short-tail vs. long-tail
Any classic keyword research guide "for dummies" would tell you something like that: 'initially target long-tail keywords and then gradually switch to short-tails', because 'it's easier to rank for long-tail keywords' and 'long-tail keywords convert better'. However, they will never tell how long is long enough and why.
How about we throw some science in?
First, let me share two insightful research papers on the topic: Web Searcher Interaction With the Dogpile.com Metasearch Engine (published as long ago as 2007 but equally useful today), and External to internal search: Associating searching on search engines with searching on sites (a more recent publication from 2015).
Along with other valuable information, both studies show that nearly 88% of all search queries used by English speaking people are 1 to 4 words long. Two-word queries are the most frequent.
Thus, keyword phrases that include over four words are likely a little bit too long, because ppl r lazy & h8 2 type. Bottom line? Very long keyword phrases won't bring you lots of traffic.
Nearly every Internet marketer trusts Google Keyword Planner when it comes to keywords' search volume. And although AdWords' search volume data is notoriously imprecise, we hardly have any better alternatives. I recommend using Google Keyword Planner to make relative judgments about the keywords on our list. E.g., if keyword A has 3,600 searches per month and keyword B has 1,800 searches per month, we can't be sure about the accuracy of these numbers, but that shows us that the first keyword is likely to bring twice as much traffic as the second one.
Furthermore, from what I observe, keywords with very low search volume (below 50) don't bring any noticeable traffic and you may safely exclude them from your list.
This leaves us with two filtering conditions. We are going to filter out the keyword phrases that consist of more than four words and have the search volume below 50.
We'll do this in Rank Tracker's Keyword Research module. Think of this module as a sandbox or a draft keyword list.
1. In Rank Tracker's Keyword Research module, click on the funnel icon at the top right corner of your workspace. A modal window with filtering options will pop up.
2. Under Show results matching… select all; this will ensure you'll only see the records that match all of your filtering conditions once the filter has been applied.
3. Click Add filter and select Length > More than > 4 (four words).
4. Click Add filter once more and select # of Searches > Less than > 50 (fifty searches per month per keyword phrase).
5. Then click OK. Your workspace will be updated and show only those keyword phrases that are over four words long and have the search volume below 50 searches. That's right — these are the keywords we won't be targeting.
6. Now you can either delete these low traffic keywords or tag them as low search volume terms. First, press Ctrl + A to select all the keywords in your filtered view. If you'd like to delete them from your project completely, right click the selection, and hit Remove Keywords. If you'd rather keep these terms and tag them, hit Add tags to selected records and label them with a meaningful tag (like "Low Search Volume").
7. Remove your filtering conditions by hitting the funnel icon once more and clicking the X button next to every filter you've added. This will reveal the unfiltered keyword list.
You can use the Tags column to sort the results, or the Quick Filter bar in the top right corner to display the records that have or do not have certain tags.
By now, we've managed to sieve out hundreds of keywords that are not likely to bring us traffic — and it only took a few minutes. You can apply other filters to quickly remove or tag keywords that match any specific criteria. Once ready, select the keywords that will become the semantic core of your website and click the Move to Target Keywords Module button in the top options pane.
3. Segment your keywords.
I'm not a fan of raw data. My eyes get sore quickly and I get irritated when the information in front of me doesn't answer the 'what?' and 'why?' sort of questions. Segmentation is a good way to turn chaos into order. There is no limit to the number of segments and their combinations that you can use for your keyword data. For now, we are going to work only with the following dimensions that I've found to be universal for all niches: semantics, searcher intent, search volume, and competition.
Switch to the Target Keywords module in Rank Tracker, and go to the Keyword Map submodule. Here, you'll see a list of semantic keyword groups, or 'buckets', that Rank Tracker has automatically created for your target terms.
Have a look at the screenshot below. Rank Tracker put all the keywords related to 'Bozeman Montana real estate' into one bucket. We can see the aggregate number of searches, the competition level, and the expected visits for the entire keyword group. Hmm, if I add a page dedicated to Bozeman real estate on my real estate website and it ranks well, then I might get an additional 129 visits per month. And if I already have a page dedicated to Bozeman real estate I can assign the whole group to this page. This initial semantic segmentation is a real time-saver.
Note: You can manually create new semantic groups, merge the existing ones, or move your keywords between groups in order to improve the automatically created segments.
To merge groups, select several of them, right-click the selection, and click Merge groups.
To move keywords to a different group, select the keywords that need to be moved, right-click the selections, and hit Move to keyword group. A modal window will pop up. You can choose one of the existing groups or type in the name of a new group to create.
Segmentation by intent
Let's have a closer look at the keywords included in my 'Bozeman Montana real estate' segment. We've got 'Bozeman Montana real estate conditions', 'Bozeman Montana real estate trends', and 'Bozeman Montana real estate for sale' here. Although all these phrases are related to the common concept, they are obviously characterized by different intent. These keywords are not likely to be a good fit for the same page.
We need to introduce another criterion for further keyword segmentation — searcher intent. People query search engines within a certain pragmatic intention in mind. For example, the customers of an online store typically go through five stages when making a purchase; and they tend to use particular keywords at each stage. See the illustration below.
Let's make things a little simpler and decide on the three major types of searcher intent. These are:
- Information (e.g., 'quadrocopter'),
- Investigation (e.g., 'best quadrocopters'),
- Transaction (e.g., 'buy Parrot AR Drone online').
Not surprisingly, keyword search volume depends largely on which type the term belongs to. We can plot the relationship on a chart.
Informational queries have the highest search volume, but at this stage the users are not ready for a transaction (the purchase of your product or service). Queries made at the investigation stage have a lower search volume, but imply that the user is more ready for a transaction. Finally, transactional keywords have the lowest search volume, but the searchers are willing to carry out a transaction. Note that queries become longer-tailed along the user intent curve, with transactional keywords being the longest.
The chart above gives you the recipe for the perfect money keyword:
- Maximal search volume,
- Minimal competition.
But is there a way to tell between informational, investigational, and transactional queries just by looking at them? Yes there is. People construct search queries using somewhat steady patterns for each type. All we need to do is find these patterns in our keyword list.
In the table below, I collected lexical indicators typical of each query type. Keep in mind that it's better to make up a list of such indicators for your own niche on a case basis, because there's jargon and specific set expressions in every industry.
best ways to
i need to
how do i
how do you
how to … with [product name]
how to build
how to get rid of
how to make
where can i buy
where to buy
where to shop for
with credit card
Now we are going to improve our keyword list back in Rank Tracker's Keyword Map submodule.
1. Click on funnel icon in the top right corner. The advanced filtering dialogue will pop up.
2. Select to show results matching any of the following conditions.
3. Add the following filters for all your transactional 'indicators' (feel free to use the table above for reference):
- Keyword > Contains > [transactional indicator 1]
- Keyword > Contains > [transactional indicator 2]
Et cetera. See the screenshot below.
4. Rank Tracker will return the list of keyword phrases containing any of the words that you have provided. Press Ctrl + A to select all the keywords and right click the selection.
5. Feel free to tag these terms with a meaningful tag like 'Transaction'. To make things even quicker and easier, you can color mark the keywords according to the intention type by choosing Set color marker in the context menu. I like to use blue for informational, green for investigational, and orange for transactional.
6. Loop back to Step 1 to filter and mark investigational and informational keywords in a similar manner.
Segmentation by search volume and competition
By now, each keyword on your list has two dimensions (semantic and searcher intent) and two metrics (competition and search volume). You can use these to generate a list of easy-to-get money keywords right away. Simply create a new filtering condition, include all transactional keywords with low and medium competition, sort the keywords in descending order by the search volume — and voila! You've created an insightful keyword list full of efficient buy-now terms.
4. Build a keyword map
Keyword mapping is the process where you assign your researched keywords to specific landing pages on your site. But does everyone need a keyword map?
The short answer is "yes". Have you heard about keyword cannibalization? To put it short, if several pages of your website are relevant to the same search query, these pages will compete with each other in SERPs for rankings. The page that is less suitable or important from your standpoint may be ranked higher than the landing page you are specifically targeting this keyword with.
On the one hand, a keyword map helps you avoid cannibalization and ensure that the keywords are used consistently, and all your landing pages rank for the right keywords. On the other hand, it also makes it easier to detect page-specific issues. For example, if you observe that a group of keywords assigned to a single page loses rankings, you may assert that there is an issue with this page. You may further dig into its on-page factors or backlinks to find the causes of the drop.
You can create a keyword map and bring a new dimension to your keyword list right in Rank Tracker by assigning keywords to relevant landing pages.
Apart from common sense and relevance, there's one more important factor you should take into account when you assign your keywords to your pages. That's Keyword Difficulty.
Keyword Difficulty is a metric that shows you how strong the top ranking pages for any given keyword are, and hence how difficult it's going to be for you to beat them. In Rank Tracker, the Keyword Difficulty formula takes all the important SEO factors into account: backlinks, social signals, on-page, domain age, etc. The tool calculates the difficulty score for each page ranking in top 10 for the keyword. Then, the average difficulty score for the 10 competitors is calculated — and that's the keyword's overall Keyword Difficulty.
Keyword Difficulty is ultra-important for building a keyword map as it largely determines how likely each of your landing pages is to rank well in the SERPs. The trick is to use Keyword Difficulty in combination with the page's SEO authority so you can target the more 'difficult' terms with the more authoritative pages.
Here's how you can check the Keyword Difficulty of your target keywords:
1. In Rank Tracker's Keyword Map submodule, select the terms you'll be analyzing. You can go to All keywords and hit Ctrl + A to update the factor for all your terms, but mind that it may really take a while if you've got a few hundred of keywords in this module.
2. Switch to the Keyword Difficulty view in the top right corner of the dashboard.
3. Hit the Update Keyword Difficulty button on the top menu pane.
When the analysis is complete, you'll see a difficulty score calculated for each of your keywords. Click on any term for a difficulty breakdown across your top ranking competitors.
Like I said, you'll want to target the terms with a higher difficulty score with your most important pages. You probably already know what these are: the pages that have the most traffic and backlinks pointing to them. E.g., if you have an ecommerce site, they are probably your home page and category pages. (If you're not sure about your site's strongest pages, go on and use SEO SpyGlass to figure out how many links point to each page of your site).
Now, let's assign keywords to landing pages in your Rank Tracker project and build a keyword map for your site.
1. Still in the Keyword Map submodule, select the keywords or group(s) that you'll be targeting with a single landing page.
2. Right-click the selection. In the context menu, hit Assign Selected Keywords to Landing Page.
3. In the modal window that pops us, enter the URL of the landing page. When in doubt, choose the page with more traffic and authority.
4. Repeat the 3 steps above for each of your landing pages.
5. When you are done, click on Landing Pages at the top of the list of your keyword groups. Here, you'll find your keyword map — the list of landing pages, and the terms you'll be targeting with each page.
In this view, you'll see the average Keyword Difficulty and aggregate expected visits for all keywords that have been mapped to any given page. At this point, you can do a little final keyword moving, and re-assign some keywords to other pages.
So that's my keyword research routine that's been working out extremely well so far. The process is universal for literally any niche, but the criteria you may want to segment your keywords by may be different. Other common keyword segments include:
- Branded vs. non-branded keywords,
- 'Low-hanging fruit' keywords (or 'quick wins'),
- Top converters,
- Keywords split by geographic locations.
And so on. You can further combine these dimensions or slice them up according to the terms' search volume, competition, and rankings. The approach won't change a lot: collect the data, sanitize it, create and apply filters, tag or mark your shortlisted terms accordingly, and finally, map them to your pages.
Do you have more ideas for insightful keyword research reports? Please share them in the comments below.
Head of SEO at SEO PowerSuite