By analysing the logs, we can identify which pages search engines like Google or Baidu scanned. We compare this data to the crawl in order to understand what search engines can access and what is available from your site’s structure.
Log analysis help to better understand what search engines do on your website in order to provide advanced SEO recommendations that will help your pages to rank higher.
In order to run a log analysis, we need to get the logs from your server so that we will have all the hits from bots and users for a specific period of time. We will then crawl your website in order to understand what is currently available in terms of content as well as to get the site’s structure. Once we have these two data, we will merge them, create reports and issue recommendations about three main types of URLS.
URLS known by search engines but out of the site’s structure. These URLS will not perform as they should. The goal is to put them back in the site's structure, if they are relevant, or to apply specific recommendations so that search engine stop crawling them.
URLS that are in the structure but that search engines haven't crawled. Since these pages are unknown for search engines, they won't rank them. The goal is to understand why they don't crawl them and to make sure that they will do it after having identified solutions.
URLS that search engines crawl and that are available in the site's structure. These pages will perform in terms of rankings. However, we can still identify if some of these pages are irrelevant so that we can block them in order to improve the crawling process.
The log analysis will provide you advanced SEO recommendations that you can’t have if you just run a crawl. Additional files will be delivered to fix issues as well.
Get Clicks can provide on-off log analysis. In that case, we will need to get the logs from a specific period of time. We will use Screaming Frog Log Analyser to run the survey.
We can also run analysis on daily basis and we will mainly use Botify. Logs will have to be delivered on daily basis. This will fit large e-commerce websites.
Log analysis is important for eCommerce, newspapers, HR platforms and any other websites that manage high volume of pages. Search engines need help to crawl, index, and log analysis is the perfect way to follow exactly what do they do on your pages. By merging the logs to other data like your on-page SEO elements or your organic traffic and revenues, we can highlight key recommendations that will be tailored for search engines.
Logs are stored on your web-server and your IT team or hosting provider can help you to access them.
GDPR (General Data Protection Regulation) is a European law which protects users and their privacy. While they need to consent to share their data with a company and that this law only apply to Europe at the moment, it is highly possible that such initiative will spread across the globe in the future.
However, it is not an issue for SEO since log analysis aims at analysing Googlebot, not humans.