• Sign in to Neowin Faster!

    Create an account on Neowin to contribute and support the site.

  • 0
Sign in to follow this  

Robots.txt redirect issue

Question

firey    4,130

So I have a website (http://stattik.net/) that I am trying to get setup with SEO as I really want to start it.  The website uses angular with routing to handle navigation.  I modified the .htaccess so that any images, css, icons, txt, xml, etc will not be routed through the index.html page however for some reason 

http://stattik.net/robots.txt keeps redirecting me back to the home page.  However http://stattik.net/sitemap.xml doesn't redirect nor does http://stattik.net/test.txt

 

Here is my entire .htaccess

 

RewriteEngine On

  Options -MultiViews
    RewriteEngine On

	#robots.txt
    RewriteRule ^robots.txt$ http://stattik.net/robots.txt [NC]
    
	#make www. link to non-www.
    RewriteCond %{HTTP_HOST} ^www\.stattik\.net$
    RewriteRule ^/?$ "http\:\/\/stattik\.net\/" [R=301,L]
    
    #any images, js, css,robots,sitemap,etc don't use the index.html
    RewriteCond %{REQUEST_FILENAME} !-f
    #RewriteCond %{REQUEST_FILENAME} !-d
    RewriteCond %{REQUEST_URI} !.*\.(img|js|css|txt|ico|xml)
    RewriteRule ^ index.html [QSA,L]

    #disable libwww-perl
    RewriteCond %{HTTP_USER_AGENT} libwww-perl.* 
    RewriteRule .* ? [F,L]

    #Cache static content for 6 months
    ExpiresActive On
    ExpiresByType image/png A15552000
    ExpiresByType image/jpeg A15552000
    ExpiresByType image/ico A15552000
    ExpiresByType text/javascript A15552000
    ExpiresByType text/css A15552000

Anyone have any ideas what could be going on?

Share this post


Link to post
Share on other sites

4 answers to this question

Recommended Posts

  • 0
game_over    802

robots.txt is loading fine for me.

 

You have either fixed it or the problem is at your end? maybe caching problem?

Share this post


Link to post
Share on other sites
  • 0
firey    4,130

Yea, it's acting weird.  Seems fine in IE but not chrome.  Not sure what's going on with it.

Share this post


Link to post
Share on other sites
  • 0
Jack W    419

Loads for me in Chrome and Microsoft Edge - out of interest, remove the empty Disallow statement from your robots.txt, then try.

Share this post


Link to post
Share on other sites
  • 0
pqt    275
47 minutes ago, firey said:

Yea, it's acting weird.  Seems fine in IE but not chrome.  Not sure what's going on with it.

It's a caching issue probably, it's automatically completing the redirect on chrome to save time. Clear out the necessary cache and give it another go. Chances are that'll fix the issue since others are able to see it - which tells me the code is just fine.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Recently Browsing   0 members

    No registered users viewing this page.