• 0

Search engines don't index my dynamic website


Question

What is the best way to get search engines, particularly google, to index dynamic content?

I've made a website which uses MySQL and PHP but google can't see the database content. I've read that google is OK with GET requests which is what my site uses. I've also read that a good way to see if a site is easily accessable to search engine robots is to use the lynx browser which i've tried. I can navigate my site fine with lynx but if i put "penally site:www.westwalesholidaycottages.co.uk", without the quotes, into google it won't find anything. Penally is a word which only appears in the database.

I know about mod_rewrite but unfortunately my host doesn't have it enabled :angry: . Has anyone used this? is there an alternative? I was thinking of moving hosts to one which does have it enabled if it was worth it.

Edit: I had an idea about mod_rewrite. the reason my host doesn't have it enabled is because it takes a lot of processing time. Is it possible to add something to the .htaccess file so that apache only translates the address for googlebot and other search bots?

Edit again: ignore my first edit, i realise that wouldn't work :s

Edited by phoe*nix

12 answers to this question

Recommended Posts

  • 0

Yes, mod_rewrite is the solution to this, and yes, you can add something to your .htaccess. However, I am not aware of a way to only rewrite URLs for Googlebot, but, why would it be a problem to rewrite them for users?

Post what you want the link to look like and where you want it to go to, and I will give you the appropriate .htaccess code. ;)

  • 0

I think you are mixed up about what does the Googlebot and other search engines index of your site.

Penally as you said it's only in the database, so in order for google to see it is to implement it in a webpage for example: www.westwalesholidaycottages.co.uk?p=penally and then make a link in your index, and in that page put the word penally, that way google will find it.

If you wan't to make a search for your site your best way is to program yourself one or make all of the content static and suscribe to google so google can search your site (only if is static).

  • 0

the point of redirecting only search bots is because mod_rewrite takes a lot of processing power because, i'm guessing here, everytime the server receives a request for say www.westwalesholidaycottages.co.uk/browse/b/1/pl/p it has to convert that to www.westwalesholidaycottages.co.uk/browse.php?b=1&pl=p. So my host has it disabled.

i realised redirecting only search bots wouldn't work.

in theory google should find penally because, on the first page there's a link to the browse page and from there, there is a link to searchresult.php?b=1&pl=p, where pl=p is a list of cottages where their names begin with p. penally would appear in that search result page.

the problem is, my host doesn't have mod_rewrite enabled. is there an alternative to using mod_rewrite?

  • 0
  teh h4x0r5 said:
That's the whole point of mod_rewrite; it makes it so that dynamic content can be indexed.

586772182[/snapback]

I think you'll find that this is *not* the whole point of mod_rewrite... Just one of its many uses.

  • 0

I'm a big fan of mod_rewrite but I use it mainly for usability.

Since it's been implemented I've seen a noticeable rise in inbound links, people seem much more happy with linking the tidy URLs:

http://www.andrewkendall.com/pages/photogallery/742/9.php

vs

http://www.andrewkendall.com/pages/photoga...picturenumber=9

That said, I've never found a problem with Google indexing dynamic pages, especially since I started using Google Sitemaps.

i.e. "My Chemical Romance Pictures" shows up fine.

  • 0
  Computer Guru said:
sitemaps are the answer. just write a php script that automatically updates hte xml after every change.

586775905[/snapback]

Can you elaborate?

Can you add dynamic URLs to sitemaps? for example: <loc>http://www.westwalesholidaycottages.co.uk/housedetails.php?propref=llan</loc>

  • 0

I'm in agreement with all the guys saying to *not* use ?foo=foo&foo2=foo2 etc.

Check out this tutorial on how to make clean URIs without mod_rewrite: http://www.evolt.org/article/Making_clean_...d_PHP/18/22880/.

Dougal.

This topic is now closed to further replies.
  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Posts

    • Very cool next level tech (for Witcher 4, CybePunk 2, etc.), but I'd be more impressed with seeing this at 4k than through a too low-bitrate YouTube ~1080p filter.
    • I was thinking, and was going to post, almost exactly what @Brandon Hjust posted!
    • Last chance to download DearMob iPhone Manager for PC/Mac (worth $79.95) for free by Steven Parker Claim your complimentary copy worth $79.95 for free today, before the offer ends today on June.3. Download a free licensed copy of DearMob iPhone Manager for effortless backup, restore, and flexible data transfer. Manage videos, photos, music, apps, contacts, ebooks, and 17+ other file types without data erasing risks. Easily transfer, manage, and back up your iPhone or iPad data--safely and without data loss. Free up space and clean unwanted files in just a few clicks. Get your free licensed copy for PC or Mac, fully compatible with all iOS and iPadOS devices. Highlights: Lossless Photo Transfer: Move or albums with zero quality loss; convert HEIC to JPG for easy sharing. Batch Video Export: Transfer videos to your PC without compression, freeing up valuable iPhone space. Custom Music & Ringtones: Add music, playlists, or create ringtones from your favorite songs. One-Click Backup & Restore: Protect your data with fast, reliable backups and restores, plus password encryption Comprehensive Data Management: Manage photos, music, videos, messages, apps, contacts, ebooks, and more. Offline Management: Take full control of your data without needing an internet connection. This free offer ends on 6/3/2025—don’t miss out! How to get it Please ensure you read the terms and conditions to claim this offer. Complete and verifiable information is required in order to receive this free offer. If you have previously made use of these free offers, you will not need to re-register. While supplies last! Free download: DearMob iPhone Manager for PC/Mac (worth $79.95) Offered by Digiarty WinXDVD, view other free resources The below offers are also available for free in exchange for your (work) email: DearMob iPhone Manager for PC/Mac ($79.95 Value) FREE – Expires today 6/3 The Cybersecurity Control Playbook ($100 Value) FREE – Expires 6/4 The Embedded Linux Security Handbook ($31.99 Value) FREE – Expires 6/4 Teach Yourself VISUALLY Microsoft 365, 2nd Edition ($20 Value) FREE – Expires 6/4 Winxvideo AI V3.0 Lifetime License for PC ($69.95 Valued) FREE – Expires 6/8 Aiarty Image Enhancer for PC/Mac ($85 Value) FREE – Expires 6/8 Solutions Architect's Handbook, Third Edition ($42.99 Value) FREE – Expires 6/10 AI and Innovation ($21 Value) FREE – Expires 6/11 Macxvideo AI ($39.95 Value) Free for a Limited Time – Expires 6/22 The Ultimate Linux Newbie Guide – Featured Free content Python Notes for Professionals – Featured Free content Learn Linux in 5 Days – Featured Free content Quick Reference Guide for Cybersecurity – Featured Free content
  • Recent Achievements

    • Week One Done
      Adam Todd earned a badge
      Week One Done
    • Contributor
      Ed B went up a rank
      Contributor
    • One Month Later
      moporcho earned a badge
      One Month Later
    • One Month Later
      Parotel earned a badge
      One Month Later
    • Reacting Well
      Cryptecks earned a badge
      Reacting Well
  • Popular Contributors

    1. 1
      +primortal
      202
    2. 2
      snowy owl
      146
    3. 3
      ATLien_0
      133
    4. 4
      Xenon
      120
    5. 5
      +FloatingFatMan
      110
  • Tell a friend

    Love Neowin? Tell a friend!