• 0

Search engines don't index my dynamic website


Question

What is the best way to get search engines, particularly google, to index dynamic content?

I've made a website which uses MySQL and PHP but google can't see the database content. I've read that google is OK with GET requests which is what my site uses. I've also read that a good way to see if a site is easily accessable to search engine robots is to use the lynx browser which i've tried. I can navigate my site fine with lynx but if i put "penally site:www.westwalesholidaycottages.co.uk", without the quotes, into google it won't find anything. Penally is a word which only appears in the database.

I know about mod_rewrite but unfortunately my host doesn't have it enabled :angry: . Has anyone used this? is there an alternative? I was thinking of moving hosts to one which does have it enabled if it was worth it.

Edit: I had an idea about mod_rewrite. the reason my host doesn't have it enabled is because it takes a lot of processing time. Is it possible to add something to the .htaccess file so that apache only translates the address for googlebot and other search bots?

Edit again: ignore my first edit, i realise that wouldn't work :s

Edited by phoe*nix

12 answers to this question

Recommended Posts

  • 0

Yes, mod_rewrite is the solution to this, and yes, you can add something to your .htaccess. However, I am not aware of a way to only rewrite URLs for Googlebot, but, why would it be a problem to rewrite them for users?

Post what you want the link to look like and where you want it to go to, and I will give you the appropriate .htaccess code. ;)

  • 0

I think you are mixed up about what does the Googlebot and other search engines index of your site.

Penally as you said it's only in the database, so in order for google to see it is to implement it in a webpage for example: www.westwalesholidaycottages.co.uk?p=penally and then make a link in your index, and in that page put the word penally, that way google will find it.

If you wan't to make a search for your site your best way is to program yourself one or make all of the content static and suscribe to google so google can search your site (only if is static).

  • 0

the point of redirecting only search bots is because mod_rewrite takes a lot of processing power because, i'm guessing here, everytime the server receives a request for say www.westwalesholidaycottages.co.uk/browse/b/1/pl/p it has to convert that to www.westwalesholidaycottages.co.uk/browse.php?b=1&pl=p. So my host has it disabled.

i realised redirecting only search bots wouldn't work.

in theory google should find penally because, on the first page there's a link to the browse page and from there, there is a link to searchresult.php?b=1&pl=p, where pl=p is a list of cottages where their names begin with p. penally would appear in that search result page.

the problem is, my host doesn't have mod_rewrite enabled. is there an alternative to using mod_rewrite?

  • 0
  teh h4x0r5 said:
That's the whole point of mod_rewrite; it makes it so that dynamic content can be indexed.

586772182[/snapback]

I think you'll find that this is *not* the whole point of mod_rewrite... Just one of its many uses.

  • 0

I'm a big fan of mod_rewrite but I use it mainly for usability.

Since it's been implemented I've seen a noticeable rise in inbound links, people seem much more happy with linking the tidy URLs:

http://www.andrewkendall.com/pages/photogallery/742/9.php

vs

http://www.andrewkendall.com/pages/photoga...picturenumber=9

That said, I've never found a problem with Google indexing dynamic pages, especially since I started using Google Sitemaps.

i.e. "My Chemical Romance Pictures" shows up fine.

  • 0
  Computer Guru said:
sitemaps are the answer. just write a php script that automatically updates hte xml after every change.

586775905[/snapback]

Can you elaborate?

Can you add dynamic URLs to sitemaps? for example: <loc>http://www.westwalesholidaycottages.co.uk/housedetails.php?propref=llan</loc>

  • 0

I'm in agreement with all the guys saying to *not* use ?foo=foo&foo2=foo2 etc.

Check out this tutorial on how to make clean URIs without mod_rewrite: http://www.evolt.org/article/Making_clean_...d_PHP/18/22880/.

Dougal.

This topic is now closed to further replies.
  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Posts

    • PicView 3.1.4 by Razvan Serea PicView is a fast, free and fully customizable image viewer for Windows 10 and 11. It supports a vast range of image file types, including WEBP, GIF, SVG, PNG, JXL, HEIC, PSD and many others. Additional features includes viewing EXIF metadata, image compression, batch resizing, viewing images within archives and comic books, image effects, image galleries, and more. Available in portable and installable versions. PicView 3.1.4 changelog: What's new Mouse Side Buttons Customization: You can now change how the mouse side buttons work. Choose to navigate file history or switch between directories. Find this setting under the Mouse tab in the settings window (#199). Improvements Directory Navigation: Improved navigation between directories. If Search subdirectories is enabled, PicView moves to the next directory in the list; if off (or if there are no directories in the list), it navigates as before. PicView now remembers your startup directory (including subdirectories) and restores it on the next launch. Performance: Preloader has been fine-tuned for better performance and lower memory usage. Bug Fixes Fixed incorrect saved settings path when saving in portable mode (#213). Corrected cases where keybindings and file history were not saved when there was no write permission. Restoring the window from a maximized state now correctly keeps its position when auto-fit is enabled. Fixed start-up menu text alignment and missing text issues. Translations Hebrew translation by @Y-PLONI (#212) Hungarian translation by @JohnFowler58 (#209) Japanese update by @coolvitto (#214) Miscellaneous Updated to Avalonia 11.3.1. Now using ZLinq and ZLinq.FileSystem to reduce memory allocation and possibly improve performance. Future versions managed via Scoop will now preserve configuration files (#15555). Experimental settings (not yet available in the UI—edit UserSettings.json directly): Disable the file watcher (not recommended, untested). Choose preload amount and direction (forward/backward). Download: PicView 3.1.4 | Portable ~50.0 MB (Open Source) Download: PicView ARM64 | Portable ARM64 Links: PicView Home Page | Github Project Page | Screenshot Get alerted to all of our Software updates on Twitter at @NeowinSoftware
    • The design looks good on their phones, but I can't say that I like the look of these.
    • AMD preps Ryzen 9600X3D for those who want a taste of the fastest gaming CPU for cheap by Sayan Sen AMD desktop processors are available across two platforms: socket AM4 and socket AM5. Both of these sockets are still going pretty strong, and the company is offering deals and discounts for both as it continues to release new SKUs for its older platform. For example, the company launched an affordable X3D processor for AM4 this month with the new 5500X3D. These gaming CPUs from AMD have been a massive success, as both AM4 and AM5 X3D chips are quite popular among gamers, and for good reason, too. They are fast, very fast. The massive chunk of vertical 3D stacked last-level cache (LLC) Level 3 cache helps these processors a ton in scenarios such as gaming. In our Ryzen 9 9950X3D review, we saw that 300+ frames per second are possible depending on the game title. While we did not review the 8-core 9800X3D, it has been found to be even slightly faster than the 9950X3D and is currently the most powerful gaming CPU on the planet. The Ryzen 9 9800X3D is selling these days for around $440-$500, and that may still be out of budget for many users. As such, similar to the 5600X3D, which is a 6-core 5000X3D processor cut down from the 8-core 5800X3D, AMD is planning to launch a hexacore Ryzen 5 9600X3D that will have two cores disabled compared to the 9800X3D. The information has been uncovered from the recent system integrator (SI) graphics driver for the recently launched Radeon AI PRO R9700 discrete GPU. The driver (Windows SI Driver for Radeon AI PRO R9700, Version 25.10.13.01) confirms the existence of 9600X3D as well as a 9600 non-X SKU and several other Ryzen PRO 9000 CPUs. The full list of upcoming CPUs is given below: Ryzen 5 9600X3D Ryzen 5 9600 AMD Ryzen 9 PRO 9945 Ryzen 7 PRO 9745 Ryzen 5 PRO 9645 Ryzen 5 PRO 9400 In terms of specs, the core configuration of the 9600X3D will be identical to the Ryzen 9600X and 9600, barring things like clock speeds. However, the six-core X3D will have a lot more L3 cache at 96 MB vs 32 MB on the 9600X and 9600. There is no information on pricing at the moment. Source: AMD (spotted by Mellodic Warrior on X)
    • With the dwindling number of TV license payers the BBC has to look for alternative avenues for income.
  • Recent Achievements

    • Week One Done
      Crunchy6 earned a badge
      Week One Done
    • One Month Later
      KynanSEIT earned a badge
      One Month Later
    • One Month Later
      gowtham07 earned a badge
      One Month Later
    • Collaborator
      lethalman went up a rank
      Collaborator
    • Week One Done
      Wayne Robinson earned a badge
      Week One Done
  • Popular Contributors

    1. 1
      +primortal
      681
    2. 2
      ATLien_0
      275
    3. 3
      Michael Scrip
      221
    4. 4
      +FloatingFatMan
      170
    5. 5
      Steven P.
      164
  • Tell a friend

    Love Neowin? Tell a friend!