• 0

Robots.txt redirect issue


Question

So I have a website (http://stattik.net/) that I am trying to get setup with SEO as I really want to start it.  The website uses angular with routing to handle navigation.  I modified the .htaccess so that any images, css, icons, txt, xml, etc will not be routed through the index.html page however for some reason 

http://stattik.net/robots.txt keeps redirecting me back to the home page.  However http://stattik.net/sitemap.xml doesn't redirect nor does http://stattik.net/test.txt

 

Here is my entire .htaccess

 

RewriteEngine On

  Options -MultiViews
    RewriteEngine On

	#robots.txt
    RewriteRule ^robots.txt$ http://stattik.net/robots.txt [NC]
    
	#make www. link to non-www.
    RewriteCond %{HTTP_HOST} ^www\.stattik\.net$
    RewriteRule ^/?$ "http\:\/\/stattik\.net\/" [R=301,L]
    
    #any images, js, css,robots,sitemap,etc don't use the index.html
    RewriteCond %{REQUEST_FILENAME} !-f
    #RewriteCond %{REQUEST_FILENAME} !-d
    RewriteCond %{REQUEST_URI} !.*\.(img|js|css|txt|ico|xml)
    RewriteRule ^ index.html [QSA,L]

    #disable libwww-perl
    RewriteCond %{HTTP_USER_AGENT} libwww-perl.* 
    RewriteRule .* ? [F,L]

    #Cache static content for 6 months
    ExpiresActive On
    ExpiresByType image/png A15552000
    ExpiresByType image/jpeg A15552000
    ExpiresByType image/ico A15552000
    ExpiresByType text/javascript A15552000
    ExpiresByType text/css A15552000

Anyone have any ideas what could be going on?

Link to comment
https://www.neowin.net/forum/topic/1296152-robotstxt-redirect-issue/
Share on other sites

4 answers to this question

Recommended Posts

  • 0
  On 29/04/2016 at 19:31, firey said:

Yea, it's acting weird.  Seems fine in IE but not chrome.  Not sure what's going on with it.

Expand  

It's a caching issue probably, it's automatically completing the redirect on chrome to save time. Clear out the necessary cache and give it another go. Chances are that'll fix the issue since others are able to see it - which tells me the code is just fine.

This topic is now closed to further replies.
  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Posts

    • Oof my bad.. I just realized the WAN port on the X10 only allows 1Gbit input from the modem >.<  arghh that derails everything lol. OK I'll need a new router then. Since I'm the only wired user in the household (everything else is wireless), would you say the RS100 is a good solution? As for a build your own, I'm afraid i'm limited in power here  PC, Games, TVs, Printer, Receiver/Amp+ secondary Amp.. heck even the Mini Split (don't ask how/why they did it this way). One more major device trips the breaker :P Although I just noticed this tidbit on my router's config page. It almost looks as if the modem could somehow input "INTO" the 10Gbit port, rather than using the 10Gbit port as an output? https://imgur.com/a/uiObRNK What would be the purpose of this?  Even if that were possible, I'm still limited to 1Gbit regular ports ToT  I was also looking at the link aggregation on the router, but apparently that doesn't really help with download from a single stream, just balances.. and Win10/11 seems to have broken this functionality due to it never being intended for consumer OSs. I had it running at one point using the Intel proset driver that came with the mobo (has two LANs, just because I could), but that one update killed it. Not sure if that would at least have contributed to something.
    • YESSS!!!!! Now that this comes up, interesting that MS hasn't planned on a 25H2... 🤔
    • Just give a great simple monitor. No AI nonsense. What's next? AI mouse that knows where you want to move?
  • Recent Achievements

    • One Month Later
      jfam earned a badge
      One Month Later
    • First Post
      TheRingmaster earned a badge
      First Post
    • Conversation Starter
      Kavin25 earned a badge
      Conversation Starter
    • One Month Later
      Leonard grant earned a badge
      One Month Later
    • Week One Done
      pcdoctorsnet earned a badge
      Week One Done
  • Popular Contributors

    1. 1
      +primortal
      561
    2. 2
      ATLien_0
      192
    3. 3
      +FloatingFatMan
      177
    4. 4
      Michael Scrip
      146
    5. 5
      Xenon
      115
  • Tell a friend

    Love Neowin? Tell a friend!