• 0

Search engines don't index my dynamic website


Question

What is the best way to get search engines, particularly google, to index dynamic content?

I've made a website which uses MySQL and PHP but google can't see the database content. I've read that google is OK with GET requests which is what my site uses. I've also read that a good way to see if a site is easily accessable to search engine robots is to use the lynx browser which i've tried. I can navigate my site fine with lynx but if i put "penally site:www.westwalesholidaycottages.co.uk", without the quotes, into google it won't find anything. Penally is a word which only appears in the database.

I know about mod_rewrite but unfortunately my host doesn't have it enabled :angry: . Has anyone used this? is there an alternative? I was thinking of moving hosts to one which does have it enabled if it was worth it.

Edit: I had an idea about mod_rewrite. the reason my host doesn't have it enabled is because it takes a lot of processing time. Is it possible to add something to the .htaccess file so that apache only translates the address for googlebot and other search bots?

Edit again: ignore my first edit, i realise that wouldn't work :s

Edited by phoe*nix

12 answers to this question

Recommended Posts

  • 0

Yes, mod_rewrite is the solution to this, and yes, you can add something to your .htaccess. However, I am not aware of a way to only rewrite URLs for Googlebot, but, why would it be a problem to rewrite them for users?

Post what you want the link to look like and where you want it to go to, and I will give you the appropriate .htaccess code. ;)

  • 0

I think you are mixed up about what does the Googlebot and other search engines index of your site.

Penally as you said it's only in the database, so in order for google to see it is to implement it in a webpage for example: www.westwalesholidaycottages.co.uk?p=penally and then make a link in your index, and in that page put the word penally, that way google will find it.

If you wan't to make a search for your site your best way is to program yourself one or make all of the content static and suscribe to google so google can search your site (only if is static).

  • 0

the point of redirecting only search bots is because mod_rewrite takes a lot of processing power because, i'm guessing here, everytime the server receives a request for say www.westwalesholidaycottages.co.uk/browse/b/1/pl/p it has to convert that to www.westwalesholidaycottages.co.uk/browse.php?b=1&pl=p. So my host has it disabled.

i realised redirecting only search bots wouldn't work.

in theory google should find penally because, on the first page there's a link to the browse page and from there, there is a link to searchresult.php?b=1&pl=p, where pl=p is a list of cottages where their names begin with p. penally would appear in that search result page.

the problem is, my host doesn't have mod_rewrite enabled. is there an alternative to using mod_rewrite?

  • 0
  teh h4x0r5 said:
That's the whole point of mod_rewrite; it makes it so that dynamic content can be indexed.

586772182[/snapback]

I think you'll find that this is *not* the whole point of mod_rewrite... Just one of its many uses.

  • 0

I'm a big fan of mod_rewrite but I use it mainly for usability.

Since it's been implemented I've seen a noticeable rise in inbound links, people seem much more happy with linking the tidy URLs:

http://www.andrewkendall.com/pages/photogallery/742/9.php

vs

http://www.andrewkendall.com/pages/photoga...picturenumber=9

That said, I've never found a problem with Google indexing dynamic pages, especially since I started using Google Sitemaps.

i.e. "My Chemical Romance Pictures" shows up fine.

  • 0
  Computer Guru said:
sitemaps are the answer. just write a php script that automatically updates hte xml after every change.

586775905[/snapback]

Can you elaborate?

Can you add dynamic URLs to sitemaps? for example: <loc>http://www.westwalesholidaycottages.co.uk/housedetails.php?propref=llan</loc>

  • 0

I'm in agreement with all the guys saying to *not* use ?foo=foo&foo2=foo2 etc.

Check out this tutorial on how to make clean URIs without mod_rewrite: http://www.evolt.org/article/Making_clean_...d_PHP/18/22880/.

Dougal.

This topic is now closed to further replies.
  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Posts

    • I got too many apps I use actively. Theres been too often I completely blank on the name of the application and had to manually look through the list for it. Now I'm using Start11 and got my apps sorted in the Start menu by categories so that if I'm looking for Krita i can find it under "Art editors". If MS are going to change it, they should consider making automated categories. Seen some Linux distros do that. But if they do, theres probably gonna be a lot of wrongly categorized apps though, unfortunately.
    • LAV Filters 0.80.0 by Razvan Serea LAVFSplitter is a multi-format media splitter that uses libavformat (the demuxing library from ffmpeg) to demux all sorts of media files. LAV Splitter is a Souce Filter/Splitter required to demux the files into their separate elementary streams. LAV Audio and Video Decoder are powerful decoders with a focus on quality and performance, without any compromises. Supported Formats: MKV/WebM, AVI, MP4/MOV, MPEG-TS/PS (including basic EVO support), FLV, OGG, and many more that are supported by ffmpeg! LAV Filters are based on ffmpeg and libbluray and is aimed to offer a all-around solution to perfect playback of file-based Media as well as Blu-rays. LAV Filters 0.80.0 changelog: LAV Splitter NEW: Introduced the IURLSourceFilterLAV interface to allow opening URLs with custom user agent and referrer NEW: Added support for WebP images Changed: Increased the length of the advanced subtitle selection field, so its no longer cut off after 255 characters Changed: Improved buffering behavior on badly interleaved video files Fixed: Audio streams with an unknown/unsupported codec are no longer selected for playback, as long as others are present Fixed: Improved accuracy of reported FPS from AviSynth scripts LAV Video NEW: D3D11 support for HEVC 4:2:2 and 4:4:4 hardware decoding NEW: Dolby Vision extension metadata is exported for renderers to use Changed: Added additional media types to support more video streams Changed: Updated dav1d for significant AV1 decoding improvements Fixed: Improved handling of H.264 4:4:4 files encoded by certain versions of x264 Fixed: VP9 DXVA2/D3D11 decoding could result in artifacts on some clips Fixed: Decoding ProRes reports more accurate color details LAV Audio Changed: Added support for additional ADPCM audio codecs Download: LAV Filters 0.80.0 | 15.5 MB (Open Source) View: LAV Filters Website | Screenshot Get alerted to all of our Software updates on Twitter at @NeowinSoftware
    • Haven't really used it much for years, not specific to Windows 11.. it's there but I mostly ignore it. I do install StartAllBack though bust mostly for the other bits that it brings. I prefer the "alt-space" type launchers (KRunner in Plasma, Flow in Windows or the like), a lot less clutter and more smarts like bookmark/history searches and other useful plugins.
    • Calibre 8.5 by Razvan Serea  Calibre is an open source e-book library management application that enables you to manage your e-book collection, convert e-books between different formats, synchronize with popular e-book reader devices, and read your e-books with the included viewer. It acts as an e-library and also allows for format conversion, news feeds to e-book conversion, as well as e-book reader sync features and an integrated e-book viewer. Calibre's features include: library management; format conversion (all major ebook formats); syncing to e-book reader devices; fetching news from the Web and converting it into ebook form; viewing many different e-book formats, giving you access to your book collection over the internet using just a browser. Calibre 8.5 changelog: New features The scrollbars used in calibre in light mode are now the same style as the ones in dark mode, this improves the contrast making the scrollbar more accessible Kobo driver: add an option to change the how the Kobo displays series numbers using a template. Manage data files dialog: Add a button to cancel remaining books when managing multiple books Kobo driver: add support for new Tolino firmware Bug fixes Prevent Windows 11 from starting a conhost.exe process for every calibre worker process E-book viewer: Improve highlight grouping with recurring chapter names When sending emails to amazon and pocketbook use random English text instead of UUIDs for subject/body. Improved news sources NYTimes WSJ Financial Times Eenadu Fokus.se Business standard Go comics NZ Herald TLS Magazine Download: Calibre 8.5 | Portable | ~200.0 MB (Open Source) Download: Calibre for MacOS | 316.0 MB Download: Calibre for Linux View: Calibre Home Page | Calibre Screenshot Get alerted to all of our Software updates on Twitter at @NeowinSoftware
  • Recent Achievements

    • First Post
      emptyother earned a badge
      First Post
    • Week One Done
      Crunchy6 earned a badge
      Week One Done
    • One Month Later
      KynanSEIT earned a badge
      One Month Later
    • One Month Later
      gowtham07 earned a badge
      One Month Later
    • Collaborator
      lethalman went up a rank
      Collaborator
  • Popular Contributors

    1. 1
      +primortal
      674
    2. 2
      ATLien_0
      277
    3. 3
      Michael Scrip
      220
    4. 4
      +FloatingFatMan
      168
    5. 5
      Steven P.
      160
  • Tell a friend

    Love Neowin? Tell a friend!