• 0

robots.txt and Windows Live/Yahoo


Question

I recently (July) released a major update to a site I manage. Part of this update included the use of mod_rewrite to make some nicer URLs. I added a robots.txt file to the root (www.example.com/robots.txt) in early August with some rules to get rid of the old, ugly links that reside in the indexes of the search engines.

I submitted the URL to the robots.txt file to Google and the pages I wanted to remove from the index disappeared within a couple of days. However, live.com/MSN and Yahoo still index them two months on. They seem to be ignoring robots.txt completely!

Anyone have any experience of this? The links I want removing do not return 404s as they need to be supported in the meantime.

Hope that's clear enough. Any suggestions?

Thanks!

Edited by boynamedfoo
Link to comment
Share on other sites

0 answers to this question

Recommended Posts

There have been no answers to this question yet

This topic is now closed to further replies.
  • Recently Browsing   0 members

    • No registered users viewing this page.