How to fix the Yoast image pages problem causing thin content

The May Google update (around 7th May 2018) hit a bunch of sites. There were threads on many forums, facebook groups and Reddit.

It also hit two of my sites which got me reading around about what might have caused the drops in traffic. After reading this article from Glenn Gabe I started looking for an issue that might have flagged my two sites as low quality.

Naturally I started digging for issues in GWT.

When I checked the index status there seemed something off. Here is how it looked on both sites:

google indexed pages

The issue was caused by the YOAST plugin. In one of the updates, they have included a feature that creates separate pages for all your images. They look like this: yourdomain.com/your-article/image1

This has created a ton of thin pages with only one image indexed by Google.

Here is how the setting looked by default and you should check your site if you have Yoast for your SEO plugin.

yoast bad image setting

And here is how it SHOULD be. This is the correct setting.

Then again this leaves us with all the indexed URLs that would take quite some time to get crawled by Google again. I decided to not wait it out but remove all the URLs from the search manually.

How can you do that?

First, go to the new search consol in GWT.

new search console

Then go to status > index coverage

index coverage

Update to make it more clear:

Click on the Indexed, not submittted in sitemap(this is where those image URLs should be)

indexed not submitted in sitemap

highlighted in yellow

Then click on the button on the right to download it as CSV.

downalod url list

highlighted in yellow

 

Now open that with Excel or Open office and make a list of all the bad URLs you want to remove.

Then head to Notepad++ (Download it here if you don’t already have it) and paste the column with the bad URLs here. Should be one URL per line. Save it.

Google Remove URLs tool only allows you to do it one at a time. And this takes a lot of time.

Here is a way to automate the URL removal from GWT

  1. Download this extension from GitHub.
  2. Unzip it
  3. Go to Chrome and in the browser type chrome://extensions/ and turn on Developer mode(on the right corner).
    chrome extensions
  4. Click on Load unpacked extension . . . and load the extension.
  5. Go to GWT Google index > Remove URLs and when you click temporary hide you should have the option to choose a file.
  6. Select the text file you already have and Google will start adding the URLs. Every 40-50 URLs you will have a notice that your quota is over so you need to refresh the page to continue.

After I made the correct setting in Yoast and removed the URLs from GWT I saw a fast recovery of the lost traffic. Here is a screen from analytics.

Conclusion

In my case, the quality issue was the thin image pages created by Yoast. Check your index and Yoast settings and if you have this issues as well, remove the URLs as soon as possible.

However, if this is not your case, look for overall quality issues with your site.

Check if you have canonicals and they are properly set.

Check you indexed status.

Make sure your site is not hacked and is not serving malware.

Check for duplicate content like author archives/dates/tags and so on.

Gid deeper and fix all your technical issues.

Good luck and wish you all a fast recovery.

52 Responses

  1. After submitting to Google how long does it take to have the pages and posts removed?

    sammy mah 7 months ago Reply
  2. You said – Download this extension from GitHub.

    What extension 😐 ?

    MD 7 months ago Reply
    • Sorry, my bad. The words GitHub are a link but it ain’t visible enough, making it blue now to stand out.

      georgeg 7 months ago Reply
  3. I don’t get a list of the indexed URLs when I export the data from the Index Coverage in Search Console. Am I doing something wrong? I’m in the new SC and click on download CSV. All I get is a few lines with text like “Excluded,Submitted URL not selected as canonical,N/A,0.00” and “Excluded,Crawled – currently not indexed,N/A,2.00”.

    Thanks for an informative post!

    Katrin 7 months ago Reply
    • Updated to make it more clear.

      First click on the Indexed, not submitted in sitemap

      Then download it as CSV and it should have the list with the URLs. Usually here are the indexed image pages. Hope this helps.

      georgeg 7 months ago Reply
      • Ah, that explains it, thank you! Worked perfectly now.

        Katrin 7 months ago Reply
  4. Hi George, great guide- I was totally in the dark about this till I saw a post this morning complaining that Yoast had not alerted users to the problem. Certainly explains why some of my positions dropped last month, I’d been scratching my head as to the possible cause

    Nicky Walker 7 months ago Reply
    • Yep. The amount of thin pages is apparently considered a low-quality domain signal by Google. Good luck with the rankings.

      georgeg 7 months ago Reply
  5. How many days did it take for you to see the traffic picking up again? After submitting those links for removal.

    MD 7 months ago Reply
  6. Massive thanks for putting this up in the Reddit sub! I’d heard a quick snipe by someone but not really understood what the issue was. After checking our client sites it’s happened to a number of them. Hoping we’ll regain a number of those positions now!
    Yoast is great and I love the fact its Free but I think this should have been brought to our attention by them and not rely on others to fix a problem of their making!
    Thanks for doing so!

    Matt Tomkin 7 months ago Reply
    • Hi Matt, glad that you like it. Hope it helps.

      georgeg 7 months ago Reply
  7. Hi George,
    Thank you for this amazing post. I need a clarification.
    Apart from the image urls, I am getting such links in the list:

    https://mywebsite/page/2/
    https://mywebsite/page/3/
    https://mywebsite/page/9/

    Should I also remove them?

    Moses 7 months ago Reply
    • Hi Moses.

      It depends on what is on the URLs. If those are duplicate of other pages or are some thin content that will not help your visitors – by all means, deindex them.

      georgeg 7 months ago Reply
  8. Hi George, thank you so much for this guide. Thanks to this catastrophic bug of Yoast we have 12.000 indexed pages. 11.000 of them are attachment-pages. Yesterday I already started to remove them in the manner you’re recommending, but I can “only” remove 1.000 per day. So it will take some time.

    Are you sure that it’s enough to remove the incorrect indexed URLs and set the button of “Redirect attachment URLs to the attachment itself?” to “YES”? I further disallowed this URLs in the robots.txt, but I’m not sure, if this is a good idea.

    It’s so annoying and absolutely incredible what has happened with this plugin – one of the most famous wordpress plugins with 5 million of active installations. I shudder to think about the total loss!

    So thanks again for this useful article!

    Michael van Bevern 7 months ago Reply
    • Hi Michael. Yea 12k URLs is a LOT of work. Here is a reddit comment that suggests google enginers have caught up with the bug and now have excluded those type of pages from the algo and some people see recovery without doing anything about those image URLs.

      Also, when you set the redirect to images, the next time the google bot crawls this url, it will remove it from the index automatically. This may take some time tho, especially with 12k URLs. And this takes from your site’s crawling budget. So, if you have the resources – remove those URLs.

      georgeg 7 months ago Reply
      • Hi George,

        thanks for your reply. So let’s hope that Google has already come to the attention of this issue. But I’m still not sure if the Google crawls will remove those attachment URLs automatically just by redirecting them to the images. Some people believe that this will not be enough – see the comment from Jack, at the end of this post: https://goo.gl/ijSbfU

        Sounds logical to me. I already asked the Yoast support what to do, but no answer until now. Nobody from Yoast has given a clear explanation what exactly to do in this case. This is the worst part of the whole thing for me. A huge amount of websites is affected due to a big, big error they made. Bugs can happen, but not a word of explanation or even any excuse from them.

        Michael van Bevern 7 months ago Reply
        • Redirecting to image files should work since the page gets 301 to an image and Google will move it to the image search (where it belongs) and remove it from the search index.

          As for Yoast, instead of saying – our bad, very sorry we made you work all those extra hours – all they say is it is the webmaster’s fault. Here is their official Twitter statement https://twitter.com/jdevalk/status/998503743330115585

          georgeg 7 months ago Reply
  9. Hey George,
    Thanks a lot for this extension and explanation. Thanks to Charles Benjamin’s post I figured that one of my sites lost traffic to this problem.. Now I have to clean the index from more then 3k pages. Could you please tell me what shall I do when I exceed the daily limit (Michael van Bevern mentioned this above). Should I leave the page opened and just refresh it after one day?

    Stefan 7 months ago Reply
    • Hi Stefan. Yep, this should work.

      Even if something goes wrong, just upload the same list. Google will notify you that this URL already exists in the Remove URLs list and will not count it.

      georgeg 7 months ago Reply
  10. I just go to every page and deindex it . Can i have to do your way again? how long it take to deindex image page?

    rifat al mahmud 7 months ago Reply
    • No, you don’t have to do it my way if you have already noindexed those pages. However, the way it is described in the article will remove them from the index faster and you’ll see faster results.

      georgeg 7 months ago Reply
  11. Thanks for the great writeup! I’ve done those exact steps about 5 days ago, and I’ve see great recovery so far. Not sure if it has to do with it, but as you said, having those images indexed is bad and should be removed anyways. But here’s the thing, checking my GSC today, I noticed some image links are starting to get reindexed again! Any idea how to fix that?

    Dee 7 months ago Reply
    • Hi Dee. GSC is slow and it would need time to catch up with the URLs. Those could be some of the old ones that it just shows now. (results in GSC for indexing could be a week old)

      The best way to check that is if you go to the URL. If it redirects fine to the image file then it’s all good. To be on the safe side add the URL to the Remove URLs tool.

      If it does not redirect however it means there is a problem somewhere in the settings.

      georgeg 7 months ago Reply
      • Thanks for your reply, George. The indexing date is new, after I’ve already removed all links, and unfortunately, the images are not redirecting. Going through Yoast’s support forum, I see some people have the same issue. Looks like I’ll have to find another solution.

        Dee 7 months ago Reply
        • Dee, in this case, you might try manually redirecting the attachment URLs to the parent post.

          Try creating image.php in your theme folder. If it already exists, edit it an include

          < ?php wp_redirect( get_permalink( $post->post_parent ), 301 ); exit; ?>

          should look like this, without the spaces php code

          at the top of the file. (make sure you edit this with notepad++ and its saved in UTF-8 and not UTF-8 BOM because it might throw a 403 error)

          then go to the attachment URL and see if it redirects to the parent post.

          let me know if this works.

          George G 7 months ago Reply
          • Thanks George. For now, I’ve switched from Yoast to another plugin to see how it goes. If that doesn’t work too, I’ll apply your method and let you know.

            Dee 7 months ago
  12. Dear George, Your information helps me a lot. Before reading this, I was doing all the process manually and it took so many time. Now I am following this method and it saves lots of time. Thanks a lot, Dear 🙂

    Zaheer Rayhan 7 months ago Reply
  13. Hi George,
    Thanks for the enlightening guide. In my Yoast SEO plugin, I don’t have the ‘Search Appearance’ option, so I can’t see the ‘Media’ button.
    How can I go about this?

    Leonard 7 months ago Reply
    • Hi Leonard. Your issue is most likely an older version of WordPress and Yoast. Make sure your WP version matches the current version at https://wordpress.org/news/category/releases/

      Then update Yoast.

      I suggest that you always update your WP to the most current versions since old versions are vulnerable.

      George G 7 months ago Reply
  14. Hello Guys,

    I am in GWT(New Version).
    http://prntscr.com/jnhmcs

    I cant find the indexed, not submitted in sitemap

    Chukwuemeka Madusha 7 months ago Reply
    • Make sure you have URLs indexed by Google. Go to google.com and type site:yourdomain.com and see if it returns any results

      George G 7 months ago Reply
      • I have over 625 pages indexed on google when i rum that command.
        I am just baffled i cant find the indexed, not submitted in sitemap.

        Chukwuemeka Madusha 7 months ago Reply
  15. Hey George, very good post!
    I followed all these steps and removed 25 bad URL’s (image attachment links ) with help of the extension.
    then next day with bad URLs one of my post ranking on 5th position got deindexed,
    do you know what could be the possible reason?

    Irwin Dezi 7 months ago Reply
    • Hi Irwin,
      Make sure you have not included this URL by accident in the Remove URLs tool. This is the only reason I could think of that the URL will be deindexed based on what you did if you have followed the article.

      Also, make sure when you got to this URL on the view source – you don’t have robots=”noindex” in the document itself. Just Ctr + F noindex. Make sure it is not in your robots.txt file either.

      Other than that, it could be many things, but this should not be Yoast related.

      George G 7 months ago Reply
      • Thanks for the reply, George!
        here’s the update
        no, I didn’t include any good URL accidentally and robots file didn’t have any no-index tag either.
        I fetched deindexed URL with GWM yesterday and today it got indexed!
        still, don’t know what went wrong.
        would like to add one more point, my ranked post got deindexed only in few countries.

        Irwin Dezi 7 months ago Reply
  16. Hi George!

    Thanks for the super informative post.

    I’ve definitely seen a nice increase in rankings after implementing your Yoast fix and submitting the image attachment pages to Google to be removed.

    Strange thing is that I still see those pages indexed in Google (attachment pages).

    Do you think this is something I should be concerned about? It’s been more than a week since I implemented your fix.

    Josh 6 months ago Reply
    • Hi Josh. When you go to the attachment URL does it redirect to the image?

      George G 6 months ago Reply
      • Hi George, yes the URL redirects to an image page.

        Josh 6 months ago Reply
  17. Informative article, but that first Yoast image is a bit misleading. It’s okay to have both settings set to No. With the second setting set to No, the attachments won’t be part of the sitemap and won’t be indexed. You’re in trouble if you answer No to the first question and Yes to the second question.

    AJ 6 months ago Reply
  18. Hi George, yeah it’s unfortunate what happened. It could have been prevented, and website owners ought to be monitoring their site for any SEO changes and issues.

    I put my thoughts together on how to prevent this in the future: https://www.contentkingapp.com/blog/yoast-seo-bug/. Curious what you think.

    Best, Steve

    Steven van Vessum 6 months ago Reply
  19. Hi George, is this limited to just Yoast? The only reason I ask is that prefer All-in-one SEO pack and that I have also seen a similar spike In Search Console. The plugin is set to ignore indexing of attachment pages.

    Chris L 6 months ago Reply
    • Hi Chris. It is a WordPress thing, the Yoast bug was that it let them indexed by default. So, yes, it could be an issue with any other seo plugin that does not handle the redirection the right way.

      You can check that by going to the image attachment URL. If it redirects to an image, it is all good. If not, look for a setting in the plugin to turn that redirection on or make those pages noindex.

      George G 6 months ago Reply
  20. Thanks! Worked for me. Sad fact: After disabling the yoast attachement plugin, Yoast turned that media settings back to the wrong “No” again. It´s wise to recheck that, after disabling that plugin. Otherwise the mess starts all over again.

    Sele 6 months ago Reply
  21. Hi George,

    I know I’m bit late here.

    I recently came to know that one of my sites also faced the issue. Didn’t know what was happening and it was that time I saw this post of yours.

    Great post mate. Very informative and easy to understand.

    So, here is my question:
    I did all the steps suggested by you.
    1. Made the “Redirect attachment URLs to the attachment itself?” to “Yes” in Yoast.
    2. Uploaded the URLs (which I don’t want to index) to remove URLs.

    Is that it? Or do I have to do something else?

    If you don’t mind, can you please tell how many days it took for you to get the traffic back?

    Arunima R 3 months ago Reply
    • About a week after deindex in the GWT.

      George G 3 months ago Reply
  22. thank you for the extension. I have about 360 pages to remove… it would take ages without your help.

    Miki 2 months ago Reply
    • glad the article helped you and good luck with the recovery

      George G 2 months ago Reply

Leave a Comment