• 0

Search engines don't index my dynamic website


Question

What is the best way to get search engines, particularly google, to index dynamic content?

I've made a website which uses MySQL and PHP but google can't see the database content. I've read that google is OK with GET requests which is what my site uses. I've also read that a good way to see if a site is easily accessable to search engine robots is to use the lynx browser which i've tried. I can navigate my site fine with lynx but if i put "penally site:www.westwalesholidaycottages.co.uk", without the quotes, into google it won't find anything. Penally is a word which only appears in the database.

I know about mod_rewrite but unfortunately my host doesn't have it enabled :angry: . Has anyone used this? is there an alternative? I was thinking of moving hosts to one which does have it enabled if it was worth it.

Edit: I had an idea about mod_rewrite. the reason my host doesn't have it enabled is because it takes a lot of processing time. Is it possible to add something to the .htaccess file so that apache only translates the address for googlebot and other search bots?

Edit again: ignore my first edit, i realise that wouldn't work :s

Edited by phoe*nix

12 answers to this question

Recommended Posts

  • 0

Yes, mod_rewrite is the solution to this, and yes, you can add something to your .htaccess. However, I am not aware of a way to only rewrite URLs for Googlebot, but, why would it be a problem to rewrite them for users?

Post what you want the link to look like and where you want it to go to, and I will give you the appropriate .htaccess code. ;)

  • 0

I think you are mixed up about what does the Googlebot and other search engines index of your site.

Penally as you said it's only in the database, so in order for google to see it is to implement it in a webpage for example: www.westwalesholidaycottages.co.uk?p=penally and then make a link in your index, and in that page put the word penally, that way google will find it.

If you wan't to make a search for your site your best way is to program yourself one or make all of the content static and suscribe to google so google can search your site (only if is static).

  • 0

the point of redirecting only search bots is because mod_rewrite takes a lot of processing power because, i'm guessing here, everytime the server receives a request for say www.westwalesholidaycottages.co.uk/browse/b/1/pl/p it has to convert that to www.westwalesholidaycottages.co.uk/browse.php?b=1&pl=p. So my host has it disabled.

i realised redirecting only search bots wouldn't work.

in theory google should find penally because, on the first page there's a link to the browse page and from there, there is a link to searchresult.php?b=1&pl=p, where pl=p is a list of cottages where their names begin with p. penally would appear in that search result page.

the problem is, my host doesn't have mod_rewrite enabled. is there an alternative to using mod_rewrite?

  • 0
  teh h4x0r5 said:
That's the whole point of mod_rewrite; it makes it so that dynamic content can be indexed.

586772182[/snapback]

I think you'll find that this is *not* the whole point of mod_rewrite... Just one of its many uses.

  • 0

I'm a big fan of mod_rewrite but I use it mainly for usability.

Since it's been implemented I've seen a noticeable rise in inbound links, people seem much more happy with linking the tidy URLs:

http://www.andrewkendall.com/pages/photogallery/742/9.php

vs

http://www.andrewkendall.com/pages/photoga...picturenumber=9

That said, I've never found a problem with Google indexing dynamic pages, especially since I started using Google Sitemaps.

i.e. "My Chemical Romance Pictures" shows up fine.

  • 0
  Computer Guru said:
sitemaps are the answer. just write a php script that automatically updates hte xml after every change.

586775905[/snapback]

Can you elaborate?

Can you add dynamic URLs to sitemaps? for example: <loc>http://www.westwalesholidaycottages.co.uk/housedetails.php?propref=llan</loc>

  • 0

I'm in agreement with all the guys saying to *not* use ?foo=foo&foo2=foo2 etc.

Check out this tutorial on how to make clean URIs without mod_rewrite: http://www.evolt.org/article/Making_clean_...d_PHP/18/22880/.

Dougal.

This topic is now closed to further replies.
  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Posts

    • You will soon have fewer reasons to open Control Panel in Windows 11 by Taras Buria Good news: more Control Panel bits are making their way to the Settings app, giving you fewer reasons to use the legacy UI that Microsoft buried deep inside its operating system. This time, the company is migrating certain clock-related settings, giving them a more modern look in the Settings app. The upgraded Settings app now lets you enable additional clocks in the notification center and adjust AM/PM indicators. These features work the same as they do in the legacy Control Panel: you can enable up to two additional clocks for two extra time zones and name them as you wish. Thanks to the redesign, the UI for setting up additional clocks supports dark and light themes, bigger UI elements, and improved accessibility. The same goes for the AM/PM indicators: you can customize them in Settings > Time & Language > Language & Region > Regional Format. Other Control Panel bits that recently made their way to the Settings app include some keyboard settings. Microsoft has not confirmed any of those changes yet, so stand by for announcements in future preview builds. Speaking of preview builds, this week, Microsoft released a new Canary build with improved context menus, taskbar changes, new accessibility features, and more. Check out the full release notes here.
    • Everything 1.4.1.1028 by Razvan Serea "Everything" is search engine that locates files and folders by filename instantly for Windows. Unlike Windows search "Everything" initially displays every file and folder on your computer (hence the name "Everything"). You type in a search filter to limit what files and folders are displayed. "Everything" only indexes file and folder names and generally takes a few seconds to build its database. A fresh install of Windows 10 (about 120,000 files) will take about 1 second to index. 1,000,000 files will take about 1 minute. "Everything" will run on Windows XP, Vista, Windows 7, Windows 8, Windows 10 and Windows 11. NTFS indexing requires the Everything service or running "Everything" as administrator. What is the Lite version of "Everything"? The Lite version of Everything is the same as the normal multilingual version of Everything, except it has some features removed: Removed ETP/FTP Server. Removed HTTP Server. Removed IPC. The Command line interface / ES will not work with the Lite version. The SDK will not work with the Lite version. Windows accessibility features and screen readers will not work with the Lite version. Everything 1.4.1.1028 changes: fixed a crash when getting help text from a context menu item that throws an exception. updated localization. "Everything" is Freeware. If you use "Everything" in a commercial environment and find it useful a donation would be appreciated. Download: Everything 1.4.1.1028 64-bit | Portable ~1.0 MB (Freeware) Download: Everything 1.4.1.1028 32-bit | Portable Download: Lite 32-bit | Lite 64-bit View: Everything Website | Screenshot Get alerted to all of our Software updates on Twitter at @NeowinSoftware
    • So you just don't like the statement? That's completely different than the statement not being true. And one doesn't have to be "full of hate" to acknowledge a cow is, in fact, a cow.
    • They are trying to pretend there is demand for this crap. That it's for special, wealthy, niche customers only right now. You know, the Apple marketing model.
    • Allow all task bar elements on all monitors, allow us to move around any taskbar element. When in Desktop mode, separate the icons in the system tray area only combine them when in touch mode.
  • Recent Achievements

    • Week One Done
      BlakeBringer earned a badge
      Week One Done
    • Week One Done
      Helen Shafer earned a badge
      Week One Done
    • First Post
      emptyother earned a badge
      First Post
    • Week One Done
      Crunchy6 earned a badge
      Week One Done
    • One Month Later
      KynanSEIT earned a badge
      One Month Later
  • Popular Contributors

    1. 1
      +primortal
      669
    2. 2
      ATLien_0
      273
    3. 3
      Michael Scrip
      232
    4. 4
      Steven P.
      163
    5. 5
      +FloatingFatMan
      160
  • Tell a friend

    Love Neowin? Tell a friend!