• 0

Search engines don't index my dynamic website


Question

What is the best way to get search engines, particularly google, to index dynamic content?

I've made a website which uses MySQL and PHP but google can't see the database content. I've read that google is OK with GET requests which is what my site uses. I've also read that a good way to see if a site is easily accessable to search engine robots is to use the lynx browser which i've tried. I can navigate my site fine with lynx but if i put "penally site:www.westwalesholidaycottages.co.uk", without the quotes, into google it won't find anything. Penally is a word which only appears in the database.

I know about mod_rewrite but unfortunately my host doesn't have it enabled :angry: . Has anyone used this? is there an alternative? I was thinking of moving hosts to one which does have it enabled if it was worth it.

Edit: I had an idea about mod_rewrite. the reason my host doesn't have it enabled is because it takes a lot of processing time. Is it possible to add something to the .htaccess file so that apache only translates the address for googlebot and other search bots?

Edit again: ignore my first edit, i realise that wouldn't work :s

Edited by phoe*nix

12 answers to this question

Recommended Posts

  • 0

Yes, mod_rewrite is the solution to this, and yes, you can add something to your .htaccess. However, I am not aware of a way to only rewrite URLs for Googlebot, but, why would it be a problem to rewrite them for users?

Post what you want the link to look like and where you want it to go to, and I will give you the appropriate .htaccess code. ;)

  • 0

I think you are mixed up about what does the Googlebot and other search engines index of your site.

Penally as you said it's only in the database, so in order for google to see it is to implement it in a webpage for example: www.westwalesholidaycottages.co.uk?p=penally and then make a link in your index, and in that page put the word penally, that way google will find it.

If you wan't to make a search for your site your best way is to program yourself one or make all of the content static and suscribe to google so google can search your site (only if is static).

  • 0

the point of redirecting only search bots is because mod_rewrite takes a lot of processing power because, i'm guessing here, everytime the server receives a request for say www.westwalesholidaycottages.co.uk/browse/b/1/pl/p it has to convert that to www.westwalesholidaycottages.co.uk/browse.php?b=1&pl=p. So my host has it disabled.

i realised redirecting only search bots wouldn't work.

in theory google should find penally because, on the first page there's a link to the browse page and from there, there is a link to searchresult.php?b=1&pl=p, where pl=p is a list of cottages where their names begin with p. penally would appear in that search result page.

the problem is, my host doesn't have mod_rewrite enabled. is there an alternative to using mod_rewrite?

  • 0
  teh h4x0r5 said:
That's the whole point of mod_rewrite; it makes it so that dynamic content can be indexed.

586772182[/snapback]

I think you'll find that this is *not* the whole point of mod_rewrite... Just one of its many uses.

  • 0

I'm a big fan of mod_rewrite but I use it mainly for usability.

Since it's been implemented I've seen a noticeable rise in inbound links, people seem much more happy with linking the tidy URLs:

http://www.andrewkendall.com/pages/photogallery/742/9.php

vs

http://www.andrewkendall.com/pages/photoga...picturenumber=9

That said, I've never found a problem with Google indexing dynamic pages, especially since I started using Google Sitemaps.

i.e. "My Chemical Romance Pictures" shows up fine.

  • 0
  Computer Guru said:
sitemaps are the answer. just write a php script that automatically updates hte xml after every change.

586775905[/snapback]

Can you elaborate?

Can you add dynamic URLs to sitemaps? for example: <loc>http://www.westwalesholidaycottages.co.uk/housedetails.php?propref=llan</loc>

  • 0

I'm in agreement with all the guys saying to *not* use ?foo=foo&foo2=foo2 etc.

Check out this tutorial on how to make clean URIs without mod_rewrite: http://www.evolt.org/article/Making_clean_...d_PHP/18/22880/.

Dougal.

This topic is now closed to further replies.
  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Posts

    • OneDrive search is currently broken for many users by Taras Buria If you are trying to find a certain file in your OneDrive storage, but the service just won't give you the stuff you need, you are likely experiencing an issue that Microsoft has just confirmed. The company updated the official OneDrive documentation with a list of known bugs to clarify that the service currently exhibits problems with the search functionality. According to Microsoft, the bug affects four platforms: Windows, iOS, Android, and Web (macOS users are spared). The company says that affected users see black search results, and the storage does not return files that certainly exist (do not blame Microsoft if OneDrive cannot find the file you accidentally deleted or renamed). Fortunately, files are intact—it is just the search that cannot find them and present them to you. Another good thing is that the issue does not appear to affect 100% of users. Microsoft says only "a subset of users" experience the bug. Neowin checked a bunch of systems with personal and work OneDrive accounts, and all of them found the requested files without any issues. Unfortunately, if you are affected, there is nothing you can do at this point. Microsoft made it clear that there are no workarounds. Therefore, you will have to manually scavenge your OneDrive folders (they can now be colored!) to search for the needed file. Microsoft added that it understands how disruptive this bug could be, and it is "working with urgency" to resolve the bug as quickly as possible. You can check out the list of known OneDrive bugs and their possible workarounds in the official support document. In other news about OneDrive frustration, Microsoft recently locked out a user who was attempting to move a bunch of important data from old hard drives to OneDrive. Eighteen attempts to resolve the issue led to nothing but automated replies from Microsoft. You can read more about that wild story here.
    • If anyone thinks bringing manufacturing back will result in a bazillion jobs should heed this information. They will be plants full of robots with human supervisors. It’s the only way the products have a chance to be affordable.
    • You can’t sell BS unless you have autonomy as a sales gimmick. They don’t have the technology, I’ve been in several Tesla Swasticars, they are not safe and self driving barely works - their implementation is incredibly dangerous and still very error prone. They are selling this as a feature to assure safety, the reality is they know their technology isn’t safe and have humans there when it’s not fully able to do what you want it to. If they hired drivers and didn’t have any self driving they are then basically competing with uber and they don’t have the numbers of cars to compete so they sell fake self driving with humans and hope they fix their problems in the meantime. Felon skum has always been a snake oil salesman and people fall for it.
    • Exactly, this defeats the point entirely.
  • Recent Achievements

    • Week One Done
      BlakeBringer earned a badge
      Week One Done
    • Week One Done
      Helen Shafer earned a badge
      Week One Done
    • First Post
      emptyother earned a badge
      First Post
    • Week One Done
      Crunchy6 earned a badge
      Week One Done
    • One Month Later
      KynanSEIT earned a badge
      One Month Later
  • Popular Contributors

    1. 1
      +primortal
      670
    2. 2
      ATLien_0
      271
    3. 3
      Michael Scrip
      229
    4. 4
      Steven P.
      166
    5. 5
      +FloatingFatMan
      161
  • Tell a friend

    Love Neowin? Tell a friend!