• 0

Search engines don't index my dynamic website


Question

What is the best way to get search engines, particularly google, to index dynamic content?

I've made a website which uses MySQL and PHP but google can't see the database content. I've read that google is OK with GET requests which is what my site uses. I've also read that a good way to see if a site is easily accessable to search engine robots is to use the lynx browser which i've tried. I can navigate my site fine with lynx but if i put "penally site:www.westwalesholidaycottages.co.uk", without the quotes, into google it won't find anything. Penally is a word which only appears in the database.

I know about mod_rewrite but unfortunately my host doesn't have it enabled :angry: . Has anyone used this? is there an alternative? I was thinking of moving hosts to one which does have it enabled if it was worth it.

Edit: I had an idea about mod_rewrite. the reason my host doesn't have it enabled is because it takes a lot of processing time. Is it possible to add something to the .htaccess file so that apache only translates the address for googlebot and other search bots?

Edit again: ignore my first edit, i realise that wouldn't work :s

Edited by phoe*nix

12 answers to this question

Recommended Posts

  • 0

Yes, mod_rewrite is the solution to this, and yes, you can add something to your .htaccess. However, I am not aware of a way to only rewrite URLs for Googlebot, but, why would it be a problem to rewrite them for users?

Post what you want the link to look like and where you want it to go to, and I will give you the appropriate .htaccess code. ;)

  • 0

I think you are mixed up about what does the Googlebot and other search engines index of your site.

Penally as you said it's only in the database, so in order for google to see it is to implement it in a webpage for example: www.westwalesholidaycottages.co.uk?p=penally and then make a link in your index, and in that page put the word penally, that way google will find it.

If you wan't to make a search for your site your best way is to program yourself one or make all of the content static and suscribe to google so google can search your site (only if is static).

  • 0

the point of redirecting only search bots is because mod_rewrite takes a lot of processing power because, i'm guessing here, everytime the server receives a request for say www.westwalesholidaycottages.co.uk/browse/b/1/pl/p it has to convert that to www.westwalesholidaycottages.co.uk/browse.php?b=1&pl=p. So my host has it disabled.

i realised redirecting only search bots wouldn't work.

in theory google should find penally because, on the first page there's a link to the browse page and from there, there is a link to searchresult.php?b=1&pl=p, where pl=p is a list of cottages where their names begin with p. penally would appear in that search result page.

the problem is, my host doesn't have mod_rewrite enabled. is there an alternative to using mod_rewrite?

  • 0
  teh h4x0r5 said:
That's the whole point of mod_rewrite; it makes it so that dynamic content can be indexed.

586772182[/snapback]

I think you'll find that this is *not* the whole point of mod_rewrite... Just one of its many uses.

  • 0

I'm a big fan of mod_rewrite but I use it mainly for usability.

Since it's been implemented I've seen a noticeable rise in inbound links, people seem much more happy with linking the tidy URLs:

http://www.andrewkendall.com/pages/photogallery/742/9.php

vs

http://www.andrewkendall.com/pages/photoga...picturenumber=9

That said, I've never found a problem with Google indexing dynamic pages, especially since I started using Google Sitemaps.

i.e. "My Chemical Romance Pictures" shows up fine.

  • 0
  Computer Guru said:
sitemaps are the answer. just write a php script that automatically updates hte xml after every change.

586775905[/snapback]

Can you elaborate?

Can you add dynamic URLs to sitemaps? for example: <loc>http://www.westwalesholidaycottages.co.uk/housedetails.php?propref=llan</loc>

  • 0

I'm in agreement with all the guys saying to *not* use ?foo=foo&foo2=foo2 etc.

Check out this tutorial on how to make clean URIs without mod_rewrite: http://www.evolt.org/article/Making_clean_...d_PHP/18/22880/.

Dougal.

This topic is now closed to further replies.
  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Posts

    • Microsoft, Indian police bust AI-powered tech support scam ring targeting elderly in Japan by Paul Hill Pop-up scams pretending to be Microsoft Working with India’s Central Bureau of Investigation (CBI), Microsoft recently assisted in busting a scam network that was targeting the elderly in Japan. The CBI raided 19 locations on May 28, leading to the arrest of six key operatives and the taking down of two call centers. The scammers were impersonating Microsoft specifically and using tech support scams against Japanese seniors. The raid led to the seizure of both digital and physical infrastructure, including computers, storage devices, and phones. The scammers were targeting older adults, who are more vulnerable to fraud. To put this activity to an end, Microsoft’s Digital Crimes Unit (DCU), the Japan Cybercrime Control Center (JC3), Japan’s National Police Agency (NPA), and India’s CBI conducted significant cross-border collaboration to trace the criminals. Thanks to the internet, cross-border crimes like these have been around for a while and multinational tech firms like Microsoft are making significant efforts to help law enforcement agencies crack down on cybercrime. Artificial intelligence is also starting to be used to make more sophisticated scams. The evolving threat This case reveals an evolution in how Microsoft’s DCU addresses cybercrime involving tech support fraud. Thanks to AI, scammers have been able to scale their operations. In response, Microsoft has moved away from focusing on individual call centers to target the heads of criminal operations and disrupting their technical infrastructure. Notably, Microsoft’s collaboration with JC3 is the first time the DCU has partnered with a Japan-based organization to assist victims. Microsoft is continually getting tips from JC3 about malicious pop-ups urging recipients to call fake technical support lines that claim to be Microsoft. This data has allowed Microsoft to shut down 66,000 malicious domains and URLs globally since May 2024. Microsoft noted that artificial intelligence is now being used by criminals to scale their operations. Some ways in which these entities leverage AI are for victim identification, writing convincing scam emails and building fake web pages, as well as for convincing translations. Anyone can use AI for malicious purposes so it could increase the number of people or groups carrying out attacks. It also makes attacks much more sophisticated and harder to detect and necessitates better consumer protections and more sophisticated security tools such as passkeys to reduce hacks. Protecting vulnerable populations and what readers can do Tech support fraud attacks have been found by the FBI to disproportionately affect older people, resulting in $590 million in losses in 2023 for just older Americans alone. In this operation that targeted Japanese victims, around 90% of the 200 affected people were over 50. If you’ve ever received suspicious communications from a party claiming to be Microsoft, you should know that Microsoft never sends unsolicited emails or makes phone calls requesting personal or financial information, and it doesn’t offer unsolicited tech support. If you do get any suspicious communications, then you should report it to Microsoft so that it can take action.
    • Firefox is irrelevant in today's internet. Most websites don't work as smooth as any Chromium browsers. Web developers are monopolizing and responsible for this situation.
  • Recent Achievements

    • Week One Done
      luxoxfurniture earned a badge
      Week One Done
    • First Post
      Uranus_enjoyer earned a badge
      First Post
    • Week One Done
      Uranus_enjoyer earned a badge
      Week One Done
    • Week One Done
      jfam earned a badge
      Week One Done
    • First Post
      survivor303 earned a badge
      First Post
  • Popular Contributors

    1. 1
      +primortal
      433
    2. 2
      +FloatingFatMan
      239
    3. 3
      snowy owl
      213
    4. 4
      ATLien_0
      211
    5. 5
      Xenon
      157
  • Tell a friend

    Love Neowin? Tell a friend!