• 0

Search engines don't index my dynamic website


Question

What is the best way to get search engines, particularly google, to index dynamic content?

I've made a website which uses MySQL and PHP but google can't see the database content. I've read that google is OK with GET requests which is what my site uses. I've also read that a good way to see if a site is easily accessable to search engine robots is to use the lynx browser which i've tried. I can navigate my site fine with lynx but if i put "penally site:www.westwalesholidaycottages.co.uk", without the quotes, into google it won't find anything. Penally is a word which only appears in the database.

I know about mod_rewrite but unfortunately my host doesn't have it enabled :angry: . Has anyone used this? is there an alternative? I was thinking of moving hosts to one which does have it enabled if it was worth it.

Edit: I had an idea about mod_rewrite. the reason my host doesn't have it enabled is because it takes a lot of processing time. Is it possible to add something to the .htaccess file so that apache only translates the address for googlebot and other search bots?

Edit again: ignore my first edit, i realise that wouldn't work :s

Edited by phoe*nix

12 answers to this question

Recommended Posts

  • 0

Yes, mod_rewrite is the solution to this, and yes, you can add something to your .htaccess. However, I am not aware of a way to only rewrite URLs for Googlebot, but, why would it be a problem to rewrite them for users?

Post what you want the link to look like and where you want it to go to, and I will give you the appropriate .htaccess code. ;)

  • 0

I think you are mixed up about what does the Googlebot and other search engines index of your site.

Penally as you said it's only in the database, so in order for google to see it is to implement it in a webpage for example: www.westwalesholidaycottages.co.uk?p=penally and then make a link in your index, and in that page put the word penally, that way google will find it.

If you wan't to make a search for your site your best way is to program yourself one or make all of the content static and suscribe to google so google can search your site (only if is static).

  • 0

the point of redirecting only search bots is because mod_rewrite takes a lot of processing power because, i'm guessing here, everytime the server receives a request for say www.westwalesholidaycottages.co.uk/browse/b/1/pl/p it has to convert that to www.westwalesholidaycottages.co.uk/browse.php?b=1&pl=p. So my host has it disabled.

i realised redirecting only search bots wouldn't work.

in theory google should find penally because, on the first page there's a link to the browse page and from there, there is a link to searchresult.php?b=1&pl=p, where pl=p is a list of cottages where their names begin with p. penally would appear in that search result page.

the problem is, my host doesn't have mod_rewrite enabled. is there an alternative to using mod_rewrite?

  • 0
  teh h4x0r5 said:
That's the whole point of mod_rewrite; it makes it so that dynamic content can be indexed.

586772182[/snapback]

I think you'll find that this is *not* the whole point of mod_rewrite... Just one of its many uses.

  • 0

I'm a big fan of mod_rewrite but I use it mainly for usability.

Since it's been implemented I've seen a noticeable rise in inbound links, people seem much more happy with linking the tidy URLs:

http://www.andrewkendall.com/pages/photogallery/742/9.php

vs

http://www.andrewkendall.com/pages/photoga...picturenumber=9

That said, I've never found a problem with Google indexing dynamic pages, especially since I started using Google Sitemaps.

i.e. "My Chemical Romance Pictures" shows up fine.

  • 0
  Computer Guru said:
sitemaps are the answer. just write a php script that automatically updates hte xml after every change.

586775905[/snapback]

Can you elaborate?

Can you add dynamic URLs to sitemaps? for example: <loc>http://www.westwalesholidaycottages.co.uk/housedetails.php?propref=llan</loc>

  • 0

I'm in agreement with all the guys saying to *not* use ?foo=foo&foo2=foo2 etc.

Check out this tutorial on how to make clean URIs without mod_rewrite: http://www.evolt.org/article/Making_clean_...d_PHP/18/22880/.

Dougal.

This topic is now closed to further replies.
  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Posts

    • XnView 2.52.2 by Razvan Serea XnView is freeware software that allows you to view and convert graphic files. It's very fast and easy to use. But more importantly it supports more than 500 graphics formats! Import about 500 graphic file formats Export about 50 graphic file formats Multipage TIFF, Animated GIF, Animated ICO support Image IPTC, EXIF metadata support EXIF auto rotation support IPTC editing Resize, rotate, crop support Lossless rotate & crop (jpeg) support Adjust brightness, contrast... Auto levels, contrast Modify number of colors Apply filters (blur, average, emboss, ...) Apply effects (lens, wave, ...) Fullscreen mode Slide show with effects Batch convert, batch rename Create WEB page easily Screen capture Create contact Sheet Create or edit Multi-page file (TIFF, DCX, LDF) TWAIN & WIA support (Windows only) Print support (Windows only) Drag & Drop support (Windows only) Compare image side by side Filmstrip layout 44 languages support (Windows only) XnView 2.52.2 changelog: NConvert: -xyb for JPEGli XnView vs XnView MP: XnView is the classic, lightweight version with basic features and Windows-only support. XnView MP is the modern, cross-platform version with an improved interface, better performance, and extended format support. Download: XnView 2.52.2 Installer | Standalone | ~24.0 MB (Non-commercial freeware) Download: XnView MP 1.9.2 64-bit | Standalone | ~100.0 MB Download: XnView MP 1.9.2 32-bit | Standalone | ~60.0 MB View: XnView Website | Screenshot Get alerted to all of our Software updates on Twitter at @NeowinSoftware
    • Correct. You don’t buy anything that’s free (if it were).
    • Vivaldi 7.4.3684.52 by Razvan Serea Vivaldi is a cross-platform web browser built for – and with – the web. A browser based on the Blink engine (same in Chrome and Chromium) that is fast, but also a browser that is rich in functionality, highly flexible and puts the user first. A browser that is made for you. Vivaldi is produced with love by a founding team of browser pioneers, including former CEO Jon Stephenson von Tetzchner, who co-founded and led Opera Software. Vivaldi’s interface is very customizable. Vivaldi combines simplicity and fashion to create a basic, highly customizable interface that provides everything a internet user could need. The browser allows users to customize the appearance of UI elements such as background color, overall theme, address bar and tab positioning, and start pages. Vivaldi features the ability to "stack" and "tile" tabs, annotate web pages, add notes to bookmarks and much more. Vivaldi 7.4.3684.52 changelog: [Address bar] URL is searched instead of opening (VB-117198) [Chromium] Update to 136.0.7103.179 [Settings][Workspaces] Add Workspace Rule button hidden by 4th rule (VB-117181) Download: Vivaldi 64-bit | 125.0 MB (Freeware) Download: Vivaldi 32-bit | ARM64 View: Vivaldi Home Page | Screenshot Get alerted to all of our Software updates on Twitter at @NeowinSoftware
    • NTLite 2025.06.10482 by Razvan Serea NTLite is a Windows configuration tool that allows you to modify your existing Windows install or an image yet to be deployed, remove Windows components, configure and integrate, speed up the Windows deployment process. Reduce Windows footprint on your RAM and storage drive memory. Remove components of your choice, guarded by compatibility safety mechanisms, which speed up finding that sweet spot. Windows Unattended feature support, providing many commonly used options on a single page for easy setup. Easily integrate a single or multiple drivers, update or language packages. Package integration features smart sorting, enabling you to seamlessly add packages for integration and the tool will apply them in the appropriate order, keeping hotfix compatibility in check. One of the important new features of NTLite (compared to its predecessors) is the ability to modify an already installed the operating system, by removing unnecessary components. Supports Windows 11, 10, 8.1 and 7, x86 and x64, live and image. Server editions of the same versions, excluding support for component removals and feature configuration. ARM64 image support in the alpha stage. Does not support Checked/Debug, Embedded, IoT editions, nor Vista or XP. NTLite 2025.06.10482 changelog: Fix Image: Image optimization would be skipped if both boot.wim and install.wim would keep only one edition Image: Crash after exporting unloaded image in certain situations Upgrade UI-Translation: Thanks for Turkish (Salih DURDU), Georgian (Anonymous) Download: NTLite 64-bit | 21.5 MB (Free, paid upgrade available) Download: NTLite 32-bit | 19.3 MB Link: NTLite Home Page | NTLite Features | Screenshot Get alerted to all of our Software updates on Twitter at @NeowinSoftware
  • Recent Achievements

    • Week One Done
      Wayne Robinson earned a badge
      Week One Done
    • One Month Later
      Karan Khanna earned a badge
      One Month Later
    • Week One Done
      Karan Khanna earned a badge
      Week One Done
    • First Post
      MikeK13 earned a badge
      First Post
    • Week One Done
      OHI Accounting earned a badge
      Week One Done
  • Popular Contributors

    1. 1
      +primortal
      681
    2. 2
      ATLien_0
      265
    3. 3
      Michael Scrip
      202
    4. 4
      +FloatingFatMan
      166
    5. 5
      Steven P.
      141
  • Tell a friend

    Love Neowin? Tell a friend!