First things first

When people talk about your SME being indexed for search engines what they really mean is a three step process which includes indexing.

What search engines do

Step 1: Discover

A search engine first needs to discover your website. You can help this process by providing a sitemap which enables you to make sure the search engine is going to get the right impression of your what, where, when and for whom.

Step 2: Crawl

The search engine crawler follows the URLs of your site and gathers all the information within your pages. If you have a main shop page with links to product category pages and they have links to individual products, crawlers will be happy following this well trodden, organised and user friendly way of getting to your products.

Step 3: Index

Once back at base the indexing process starts to make sense of all those URLs, structure and content they hold. At the end of this process the information produced is used to search against when a user types in a search term and hopefully you appear on the Search Engine Result Pages (SERPS).

1. Search for your product or service in incognito mode

Let’s start with the bleedin’ obvious but add some value to it.

• Search engines love to know all about you so they can give you more relevant results. So open your browser and select incognito or private mode to make you more anonymous and search for your product and service like your potential customer would i.e. “bespoke window maker”, “Fitness classes near me”, “live-in care agency in Oxford”.
• If you appear in the first 3 result pages have a look at which of your pages it takes you to and ask yourself if that is the page you intended. If it is great! If not you need to look at improving the page you wanted and incorporating the indexing “love” the search engine has for the one that features.
• If you don’t appear, go to the page you wanted to appear with, copy a nice long sentence and paste that in. If you appear then you need to improve this page (and other SEO pointers) to get it featured in results. If you don’t appear then you probably have some wider issues that need addressing.

2. Use a Search Operator

That’s what I am calling it anyway. For Google, in your Chrome or Safari browser type in “Site:” followed by your domain name.

For example: site:castleminibus.co.uk

Here you will see what Google is picking up. This is useful for new websites and for sites that have been added to, butchered, extended sympathetically or multiple people being responsible for updating over time with varying degrees of competence.

• Have a look at the first 20 results (Google SERPS page 1 and 2) and ask yourself if these represent the important aspects of your business.
• If they really represent you down to a T, great! If there are some glaring anomalies, then you need to do some optimisation work and even some re categorisation if things are not looking good.
• If you don’t appear then you have some issues and would need to contact a professional to sort you out.

3. Look at your sitemap

A website can have two types of sitemap: An HTML version and an XML version. Simply put, the former is a user friendly one for people and the latter is one for search engines.

Go and find your XML one to see what it is asking Search Engines to discover (Please see the “First Things First” section above for more info on “discover”)

For most of us, the sitemap file is called sitemap.xml or sitemap_index.xml

Example: https://castleminibus.co.uk/sitemap_index.xml

• Regardless of the software used to generate the sitemap, your pages are represented in a logical, organised way. For example, this entry is telling search engines to follow this link to see all the posts
  
• Once followed the search engine is presented with the posts
• If you are seeing a healthy amount of your pages then Search Engines are probably seeing them to.
• If you are only seeing a few of your pages and posts then you need to look at how your site map is being generated.
• If you can’t find your sitemap, please see next point.

4. Look at your robots.txt file

A robots.txt file tells search engines whether they can discover and crawl parts of your site in order to index it. This file is placed in the root directory of your site.

Example: https://castleminibus.co.uk/robots.txt

The instructions within this file tells search engines different bits of information but essentially to not index some files:

Disallow:/wp-content/uploads/page-with-my-bank-details.html

and to index others:

Allow: /wp-content/images/my-cat-falling-over.jpg

You should also see this entry telling search engines where your site map is.

Sitemap: https://castleminibus.co.uk/sitemap_index.xml

• If you see your sitemap here it means search engines will also.
• If you don’t see a reference to your sitemap it doesn’t mean you don’t have one. However, make it easy for them by adding it in this file.
• If you have the “Disallow” operator followed by your sitemap reference it means someone has instructed search engines to not index you. Whoops. You need to change that.

5. Google Search Console

It would be amiss of me to leave this out. Your site should be associated with a Google Analytics and Google Search Console account. I tend to find if you don’t have the former you probably won’t have the latter.

You can quickly check by right clicking any of your pages and selecting View Source (if that option isn’t there just search in Google for “how to show view source”) and search for  Google Analytics.

• If that’s there then you have Google Analytics so you may well have the Search Console. You just need to contact the person who set it up.
• If it’s not there you need to get both accounts set up.

To see if Google is picking you up (and if Google is the others will be also unless you have done something that prevents this), pick any of your URLs from your website and paste into the inspection field in the Search Console header..

• I would test the important ones such as your homepage and your products and services pages.
• If they are not there, Google will give you an indication as to why they are not.
• If you can’t get any success check the page is appearing on the sitemap that has been submitted in Search console..
  
• If it is not there, please go back to the previous sitemap point above.

Even if you do nothing, search engines like Google will try and index your website. As the SME business owner, you need to make sure you are making that process as easy as possible and that they are indexing the right things and in the right order of importance. That’s what SEO is all about of course. Without having to spend too much time you can periodically carry out some of these quick checks to make sure everything is ok with your website and to nip in the bud anything that isn’t.