How To Fix “Not Found (404)” In Google Search Console

0:00Why 404 Errors Are Not Inherently Bad

Hi, it's WP Speed Fix back. In this video we're going to talk about more Google Search Console, specifically the 404 or "Not found (404)" error under the indexing Pages section. We actually get a lot of questions about 404s, and there are a lot of misconceptions about them, so let's run through the highlights first and talk about that for a minute before we dig into Google Search Console and what to do.

404s are not inherently bad. There's this idea that 404 errors are horrible and should be avoided at all costs, and that's not true. If a page shouldn't exist on a site, it should actually 404 out. A 404 "not found" is the correct message the server should be returning. Just to be clear, it's not just the page that you see as a visitor in the browser that should say 404. The server actually returns a 404 status code to the browser instead of a 200 OK code. So your hosting or your website actually needs to return that 404.

There are some plugins that allow you to stop 404s happening entirely. They can redirect all 404s to a particular page. Broadly speaking, that's not a good idea. It's bad for SEO overall, especially over time, and it can open you up to negative SEO attacks very easily, because you're redirecting all these things that you should be telling Google don't exist, and instead you are redirecting them somewhere. There are also situations where CMS systems are broken and they're not returning a 404 status. They actually show a 404 page but return a 200. We want to avoid the scenarios where we're totally avoiding a 404 message by redirecting, and we want to make sure our hosting generates the technically correct 404 message.

There's an easy way to check this, and I'll show you on our website. There's a free SEO audit tool that takes 60 to 90 seconds, and there's no opt-in required. I've just searched for 404, and there's a section about 404s. The custom 404 on this website found a 404 page and it responded with the correct message. You want the server to respond with a 404, so that's probably a good first check to make sure you have technically correct 404s coming from your hosting or your website. That's an easy one, and there's no opt-in required, so go and run that test.

2:19Where 404s Hurt Performance and SEO

In Google Search Console there are really two places where 404s appear. Typically you'll see them in the indexing Pages section, but they're also under the Crawl section, and the Crawl Stats data is arguably more important than the indexing section. We'll talk about both of those in a second.

Some other things about 404s, especially in WordPress: if there are a lot of them, they can actually hurt performance quite a bit. If you have a lot of 404s at high frequency in WordPress, they're processed at the PHP level. In theory you can flood a site with 404s, generate a 404 error on the server end and technically knock it over. If you hammer a site hard enough, because of the way WordPress handles a 404 message, you can actually crash the site. So if you're a site owner you want to avoid too many 404s, because they can knock the site over and make it crash.

I have here the common types of 404s that we see. The most common scenario, and the one that is a huge performance problem for site speed, is a typo in the website code, or something in the website template that is referring to a file or resource that doesn't exist. Commonly it's an image that's missing and has been deleted. It might be a CSS file or a JavaScript file, and I'll show you an example in a second. These are the worst type of 404s because they are bad for speed.

Anytime the browser loads a page and gets a 404 message, it needs to figure out what to do. That's especially true if it's something to do with a template or theme, which means it's sitewide, and the browser is also trying to render the page and gets this error message. If you have a custom 404 page, it's sending the browser off to get that, which can be a couple of seconds of lag, and then the browser has to process and work it out. So if you have a 404 in the template, something erroring out, it can make the website load feel like it lags or stalls halfway through. It's bad for speed, and it's bad for SEO as well, because Google doesn't like error messages. Like we've talked about in some of the other videos, when you give Google errors to deal with or conflicting signals, Google has to figure out what to do with it, and it's generally bad for SEO and rankings.

We want to avoid 404s in the template of that nature in particular, because they will hurt performance. Typically we see these errors in the footer. If something in your footer is generating a 404 message and you do a mailout and have 5 or 10,000 people hitting the site in a short period of time, that can knock over the hosting very easily. That's a very common scenario.

5:04How to Find 404s Proactively

Before we get into Google Search Console, you can be proactive about your site, and you should be if you care about SEO or marketing. There are a few different ways to find 404s in advance. One is in Google Search Console, but there are other tools you can use as well. First, try our site speed test tool. Again it's free, takes 90 seconds and there's no opt-in required, and it's a good way of picking these up. I'll show you in this one. This is just a test I was poking around with. It will find 404 messages, so we have them in the critical section. This is a good example of a template that has 404s. We put that in critical because it's bad for SEO and bad for rankings.

If you scroll down, the ones with the red next to them are 404ing out. There are actually a few on this one. It has some sort of cookie, and there's a JavaScript file. I think this will expand out, no, that's a JavaScript file, so that's going to be bad for the site render. Then it's actually got a favicon that is 404ing out. This is a very common issue: you want to make sure you have a favicon.ico file. That is another type of 404 we see, what we would call a technical SEO 404. These are like favicon files and ads.txt files, which we'll talk about in a second. We want to avoid those because they're super simple and easy to fix. This one also obviously has some live chat, and it looks like maybe even two versions of the live chat are embedded, and that's erroring out too. So there are actually three 404s on this page, and this needs to be fixed on this site. This will be really bad for speed. Just one is bad, but when you have multiples like this, it's going to really slow down the site render and make it problematic.

That's SiteSpeedBot. You can also run a sitewide SEO tool. We like Screaming Frog SEO Spider, but there are others, like Ahrefs and Semrush, and they're all good. Basically any SEO tool that will crawl the site will find 404s. Then there are SEO plugins. Most SEO plugins monitor for 404 errors, and we like Rank Math. There's a link in the description, and the pro version is well worth it. Most SEO plugins will track 404 errors and allow you to deal with them, create a redirect or whatever. It would be worth running through this exercise just to clean up your site.

7:20Using Crawl Stats in Google Search Console

Let's look at Search Console. I'm in this site, and we want to be in the domain property. Remember, we talked about it in some of the other videos: in Google Search Console you can have a domain-wide level property, or you can have one for each type of URL or each URL variation, so HTTP, HTTPS, www, no www. You want to be in the domain level one, because that's going to show you all the data. The individual hostnames won't show you all the data. So when you're working with this, use the domain property.

I'll just move my little picture over the way. We want to be over here under Settings and then Crawl stats. We actually want to start here, because these are the more impactful 404s that it will pick up. Click Open report and you'll get this crawl report, which will give you a whole bunch of data. There is a 404 section here, but before we even get into that, there's a very common issue you'll see if you have an old site. This site is more than 15 years old, so it's been around a while. It actually has two different hostnames that have errors, crm.dojo and store.dojo, and those no longer exist. They're old, so they must still have DNS records. Those should just be deleted from DNS, because they don't exist anymore. If I click on them, this one's generating 301s, so that's generating errors. They're not generating 404s, but it doesn't exist anymore. The CRM one, let's look at that. This one is generating 404s on 52% of the loads, so that's not good. That needs to be cleaned up and deleted. I put a note down in here about old URLs. We commonly see this with old sites: old hostnames that should be deleted.

Down here there's a 404 section. This one's not too bad. We see some of the CRM stuff that we just saw, but a very common one is Google crawling for the ads.txt file. It doesn't stand for advertising. There's an article here I've linked you to: it stands for Authorized Digital Sellers. If you're not selling ads on your site, you don't really need this file, but we want to avoid the 404, so it's probably wise to just create a blank ads.txt, and then that will go away. It also gives you a little bit of protection against dodginess, because this file technically tells Google who's allowed to run ads or sell ad inventory on your site. If the file's empty, nobody's allowed, so it is in a sense a security precaution to have a blank version of that file that just doesn't generate a 404. You see Google's trying to crawl it a lot, multiple times, on these different subdomains. Every couple of days it's trying to crawl it, so that's not ideal. We don't want Google getting any errors at all, so we want to get rid of that.

What else do we have? These .well-known files, I believe, are for Android or something related to it. We don't have an Android or Apple iPhone app for this site, so those are fine to 404 out. Here we have one that should be a 404: it's just a bad link, something Google is crawling that doesn't exist. These old CSS files that don't exist are not really an issue either. Generally this is actually pretty clean. Usually we see a lot of garbage here, but deleting this CRM hostname is going to help a lot, and this stuff will go away. So that's that one.

9:21Old URLs, E-commerce and Negative SEO in Crawl Stats

Let's have a look at a couple of other sites. This one has a similar ads.txt issue, which is very common. This is an old site again, it's been around 15 or 20 years in some form or another, and it has gone through a lot of versions. You'll see there's stuff here from old themes, old versions of the site and different sorts of URLs. Some of these we might want to pick out and 301 redirect to the right page on the site, the current version of the page. Some of them, like theme images, might be from an old version of the site, an old design. One useful thing to do is export it to Google Sheets, and then you can manipulate the data, filter it, sort it and see what's going on.

A lot of this stuff, you'll see there are landing pages here for different locations, so those may be important. If something's getting crawled a lot with a lot of frequency, it might have links pointing to it. If something has a lot of links pointing to it, then we might want to 301 redirect that to another page. A common problem we see is with e-commerce sites, and I think I made a note here about old URLs. This is very common in e-commerce, where products come and go. Those products will pick up links or authority over time. If you have an e-commerce site where products come and go, or get replaced or made obsolete, then we probably want to redirect those URLs to the appropriate product that has replaced it, or the appropriate category. That way you're not losing link juice, and if someone clicks on that link that's around the web, they'll still get directed to the site or the appropriate category. If you have a WooCommerce site or e-commerce site, this is a really important one.

I think there was a note we had in there about negative SEO. This is another common one we're seeing, and I think this one had some garbage going on. Here we go. Just remember with any of these links you can click into them and get some more information. You can see what's happening and where the referral is coming from. This one we couldn't find the referral for, but there were some other garbage URLs here somewhere. Sometimes it's a negative SEO attack. If that is the case, we do want them to 404 out. We don't want them to resolve to proper pages or redirect. There was a site that was referring these dodgy links, and in that case we probably want to disavow that domain and make that go away.

Broadly, because this one is a bigger site and there's a lot more in here, I would export it to a Google Sheet. Go through this process, export to Google Sheets, and then manage it there. Decide what I'm going to do, filter it. Some of these are going to be genuine URLs from an old version of the site, some of them are garbage, and some of them we want to redirect to the page on the current site. So that's that one. I think that's enough for Crawl Stats.

14:05Not Found (404) in the Indexing Pages Report

Let's look at the indexing section. Under Pages, under indexing, you'll see "Not found". This is the same site, and let's have a look at all the URLs. Same thing here, we can export it to Google Sheets and manage it in there. Here we have some PDFs. These PDFs may be important, and they may be worth redirecting. Some of these are like theme locations, and this is not ideal, so let's see where that's coming from. That may actually be a typo in the code somewhere. Looking at this, that's one that we want to investigate. We shouldn't be linking just to this folder on the theme, so that one we probably want to do some work on and find out what's going on.

Here's default.htm. This would be an old version of the site, and it may still have links pointing at it, so again we probably want to 301 redirect that. Some of these are, again, location pages that may not exist anymore. Some of these are garbage links again, and these definitely look like a negative SEO attack. Let's look at this one. Interesting, it's coming from an internal link, which is not ideal. That's probably one that needs to be investigated as well. If the site has been hacked, we may see a lot of this stuff too. Let's have a look and see what's going on here. Okay, so this is one that we probably want to disavow. This is where these garbage links are coming from, and that one is probably worth disavowing. There's a lot of that stuff going on here. This one is pretty messy, so we definitely need some Google Sheets here.

You're probably not aware that you can also see links inside Google Search Console. I would do some more work on this one for this site. Let's have a look. Okay, that's a Russian site. If you go to Links here, you can see where Google is seeing the links coming from. So you may be able to do some troubleshooting here: look at these top linking sites and go in there and clean them up that way, or maybe disavow some of the garbage links that are pointing at the site, if they are genuinely spammy or look like negative SEO. These actually look like negative SEO because there's so much of it. There's a lot of garbage there, so that one actually probably needs some work. That's not good.

Now let's look at this one. This is WP Speed Fix, our website, and this is actually pretty clean. I don't really care about any of these. They're probably just genuine 404s that Google has picked up along the way, and we don't really need to redirect them. This one is orders, which is our order management system, so we don't really care about those. These 404s are pretty clean, so I wouldn't redirect any of them. I would just let them 404 out. Those are just natural 404s, pretty standard, so not much to do there.

17:09A Magento E-commerce Site With Review and Old Product URLs

Let's look at this next site. This is a big e-commerce site on Magento. I talked about this in one of the other videos: Magento, particularly the old version, Magento 1, has a lot of problems with canonical URLs. You can see there are a lot of these review URLs in here that probably shouldn't exist. They're probably from old products, but Google probably shouldn't be trying to index these review lists. I'd probably do some work here and I'd disallow that review string. I'd add a disallow for something like /review, like that, because that's an area of the site we don't want crawled. I'd probably go and find that URL for a genuine product that still exists and have a look at what it looks like. If we don't want those pages indexed, we would just block them with the robots.txt file, because there's a lot of that stuff here.

From the look of it, there are a lot of old products here, so we definitely want to look at redirecting some of these. I'd say this site has a problem where products come and go and are just allowed to 404 out, which we probably don't want to do. You can see here this site has hundreds, maybe thousands of products, so we don't want this to happen. This is very common with e-commerce. That's pretty much all the same.

18:17Summary of 404 Types and Free Tools

That's pretty much it. There are lots of different ways, and I'll put these notes under the video. We talked about typos in templates, old hostnames, technical 404s, negative SEO attacks, old URLs, which are common in e-commerce, and just general typos in the code. If you run a sitewide tool like Screaming Frog, you'll pick those up and it'll give you a list of 404s. Likewise with old images. That one's hard to fix if you don't have a copy of those old images, but it can be common, particularly if you are in an industry that relies a lot on images or image SEO. Then there are areas of the site we don't want crawled. Those are the most common types. It's not all of the types we'd see, but it's a lot of them and the most common ones.

That's pretty much it for this video. Like I said, if you want to dig in, we have our free SEO audit tool, which is under our More Free Stuff menu at wpspeedfix.com. Our site speed test tool is under there too, the free site speed test, and we have our free Core Web Vitals report. If you want help with SEO, we do have a Google Search Console audit service. We also have a full SEO audit service, which you'll see on there. If you're not sure, go to the homepage of our site and just request a free audit. We can have a look at your site and tell you what we can help with, and if we can help you reach your goals. Otherwise, post any questions in the comment section and we'll come back to you. Cheers.