As a website owner, you probably have a lot on your plate. Some of the issues may be getting in the way of you doing good business. Some problems that come by as a result of having a poorly designed website and failure to regularly back up your sites can be avoided. The solution is to invest in a professional web designer and have it backed up now and then. Then there are problems that you may face in the long run: like being unable to connect with your audience.
For beginners, crawl errors occur when the efforts of search engines to reach a page on your website become unsuccessful. What is crawling? A search engine interacts with your site through a program which is referred to as a ‘spider.’ The search engines use the spider to scan your website. This scan is to ensure that if someone interested in the content on your site does not find it, you get to know about it. That is how you get a crawl error.
Google Search Console or Google Webmasters Tools, on the other hand, is a free web service that allows website owners to optimize the visibility of their websites. It is a set of resources and tools that help webmasters to oversee the performance of their sites. Among their functions include the provision of information such as technical status updates, search appearance, crawl data, search traffic and educational resources.
Types of crawl errors
Before you get a guide on how to fix the crawl errors that make your website vulnerable to penalties by Google’s ranking algorithm, you need to have an idea of the errors you are likely to encounter.
Crawl errors can be segmented into two:
- URL errors which are specific to a single or particular URL.
- Site errors which suggest that your whole website cannot be crawled, and are more complicated to fix as compared to URL errors.
Crawl errors broken down into:
- Server error (500)
Server error (500) is returned by the ‘spider’ if it fails to get your URL or traverse through your website.
- Not Found (404 or 410)
It is an error that you have probably come across multiple times. So you know how annoying it is, especially if the page you just came by seems promising in comparison to the ones you have been getting for the past 1 hour. It means that a Googlebot cannot locate the page.
- Soft 404
It seems the bot does some discernment too! Well, “soft” 404 errors occur when the bot feels that a 200 error should have been a 400 status code.
- Moved permanently (303)
You are likely to get this error if the Googlebot is redirected to URLs that are no longer functional.
- Access Denied (403)
You are probably trying to gain access to particular content, but you have not been authorized. There is a likelihood of you getting 403 error.
- Other (404, 405, 406)
What these errors mean is that Google cannot particularly determine the cause and therefore no specific name for them. The errors are mostly treated like the 404 errors.
You may have blocked some web pages on your smartphone. Usually, the bot would have gained passage to the web pages, but it has not been authorized to. The reason as to why you are always getting frustrated by this error.
- Flash Content
Another error prevalent among smartphones is the Flash content. You probably have enabled way too many flash content on your phone as you do your searches. It is important to note that it is incompatible with your smartphone.
How to fix them
You have so much to take care of that you do not have a minute to spare to check for crawl errors. Wondering of what importance it is? Note that the accumulation of crawl errors will obstruct the appearance of your website on a search results page. Crawl errors will keep you updated on the URL’s that the Googlebot was unable to find. The information is provided so that you may find ways of fixing it. However, before you get anything started, you should see first all the errors as corrected so that it is easier for you to access any future errors that may be reported with urgency. Remember not to let them accumulate again!
- 404 error
404 errors are so far the easiest to fix: you might need your tea break to get it done. You might get different types of 404 error reports: some might be specific while others mysterious, that is of unknown cause.
If the cause is a link from another website which does not work: best described as a broken link, what you are required is to execute a redirect. By employing a 301 redirect, your initial request will be overridden, and you will be sent to the intended content. There is a lengthy process that can also get you around this: requesting the website owner to make adjustments.
For broken internal links and sitemap entries, all you will be required to do is treat them like other false URLs. How do you do this? By accessing the back end of your platform and setting up 301 redirects. That is all you will do to have it fixed. Simple, right?
Do not let an unknown 404 error vex you. It may even be coming from a non-existent URL. A simple 301 redirect will ward off the nuisance.
- “Soft” 404 errors
You will mostly get a soft 404 error if the URL lands you on an empty page or a page with little or no content. A 301 redirect is all you need to combat a soft 404 error that results from a faulty link. However, if the URL is of suitable shape and you still want users to be led to the page, you should consider adding content so that the bot does not pick it the next time you do a scan.
- 400, 500 and 503 error codes
These are not fixed the same way the other 404 errors are fixed. They can be a little complex, but you can get rid of them by yourself with some help from Google’s Advice or contacting your web developer.
- Site errors
Site errors are a lot more complicated. Why so? They hinder the Googlebot from accessing any given page that you have on your website. That is why you should eliminate them as soon as you can: consider giving them a priority. This tells you that you should frequent the Google Search Console to make sure none of these are lingering. There are three categories of site errors:
- Website errors, which may include typos, non-friendly URLs, and broken links.
- DNS errors. DNS is an abbreviation for Domain Name System which is a naming system for computers, resources or services connected to a private network or the internet. A DNS error is a tell-tale sign that you are losing internet connection, which indicates a failure to look up an IP address for a domain that you are trying to get hold of.
- Server errors, which mean that the server has failed to implement your request and your server has been incorrectly configured.
Now that you know how site errors come by, knowledge on how to go about them will be beneficial to the health of your website.
Being penalized in your website’s ranking is the lasting you want as a webmaster. If you are confident about your DIY skills, you could try fixing them personally with some help from guides on how to fix site errors. However, getting assistance from your web developer would be the best way around it.
Whichever method of curbing the errors you choose: whether it is getting professional help or using guides to do it yourself, ensure that it is fast and efficient. However, with a lot of practice and patience, while fixing the simple ones, you will get better with time, and they won’t find them as annoying as you do now.
Sometimes you may need to have your website relaunched, expect to get a dozen of crawl errors after that. Do not allow yourself to get maddened by this. All you need to is to adopt a few practices that will be very effective in getting rid of them.
In case you do a relaunch, and you get hit by a wave of errors, the first thing you will do, as earlier mentioned, is to mark all the errors as fixed. The next step will be to ensure that you visit your Google Search Console on a weekly basis and fixing the 404 errors: since they are the most common. Call your web developer to help you correct site errors and make use of Google’s resources to help you fix any other errors you may encounter.