There is little doubt that SEO is taking longer to deliver results than ever before (assuming you comply with Google’s guidelines as we do at Totally).
At the same time however, the value of being top of organic search has never been greater, B2B marketers say that SEO generates more leads than any other marketing initiative, which is why the SEO/Search industry has grown at such a rapid rate in recent years.
As more and more websites are developed and compete for attention, search engines have had to keep up with this growth while still focusing on delivering the most relevant content for a user. As a result, it is important to have a firm grounding in how search works in order to ensure your site is as search engine friendly as possible.
At a basic level, there are three key processes in delivering search results I am going to cover today; crawling, indexing and ranking.
Crawling
Crawling is the process by which search engines discover updated content on the web, such as new sites or pages, changes to existing sites, and dead links.
To do this, a search engine uses a program that can be referred to as a ‘crawler’, ‘bot’ or ‘spider’ (each search engine has its own type) which follows an algorithmic process to determine which sites to crawl and how often.
As a search engine’s crawler moves through your site it will also detect and record any links it finds on these pages and add them to a list that will be crawled later. This is how new content is discovered.
Indexing
Once a search engine processes each of the pages it crawls, it compiles a massive index of all the words it sees and their location on each page. It is essentially a database of billions of web pages.
This extracted content is then stored, with the information then organised and interpreted by the search engine’s algorithm to measure its importance compared to similar pages.
Servers based all around the world allow users to access these pages almost instantaneously. Storing and sorting this information requires significant space and both Microsoft and Google have over a million servers each.
Ranking
As SEO’s this is the area we are most concerned with and the part that allows us to show clients tangible progress.
Once a keyword is entered into a search box, search engines will check for pages within their index that are a closest match; a score will be assigned to these pages based on an algorithm consisting of hundreds of different ranking signals.
These pages (or images & videos) will then be displayed to the user in order of score.
So in order for your site to rank well in search results pages, it’s important to make sure search engines can crawl and index your site correctly – otherwise they will be unable to appropriately rank your website’s content in search results.
To help give you even more of a basic introduction to this process, here is a useful video from Google which explains it quite well. Each search engine follows a similar methodology to this.
