• 0

Search engines don't index my dynamic website


Question

What is the best way to get search engines, particularly google, to index dynamic content?

I've made a website which uses MySQL and PHP but google can't see the database content. I've read that google is OK with GET requests which is what my site uses. I've also read that a good way to see if a site is easily accessable to search engine robots is to use the lynx browser which i've tried. I can navigate my site fine with lynx but if i put "penally site:www.westwalesholidaycottages.co.uk", without the quotes, into google it won't find anything. Penally is a word which only appears in the database.

I know about mod_rewrite but unfortunately my host doesn't have it enabled :angry: . Has anyone used this? is there an alternative? I was thinking of moving hosts to one which does have it enabled if it was worth it.

Edit: I had an idea about mod_rewrite. the reason my host doesn't have it enabled is because it takes a lot of processing time. Is it possible to add something to the .htaccess file so that apache only translates the address for googlebot and other search bots?

Edit again: ignore my first edit, i realise that wouldn't work :s

Edited by phoe*nix

12 answers to this question

Recommended Posts

  • 0

Yes, mod_rewrite is the solution to this, and yes, you can add something to your .htaccess. However, I am not aware of a way to only rewrite URLs for Googlebot, but, why would it be a problem to rewrite them for users?

Post what you want the link to look like and where you want it to go to, and I will give you the appropriate .htaccess code. ;)

  • 0

I think you are mixed up about what does the Googlebot and other search engines index of your site.

Penally as you said it's only in the database, so in order for google to see it is to implement it in a webpage for example: www.westwalesholidaycottages.co.uk?p=penally and then make a link in your index, and in that page put the word penally, that way google will find it.

If you wan't to make a search for your site your best way is to program yourself one or make all of the content static and suscribe to google so google can search your site (only if is static).

  • 0

the point of redirecting only search bots is because mod_rewrite takes a lot of processing power because, i'm guessing here, everytime the server receives a request for say www.westwalesholidaycottages.co.uk/browse/b/1/pl/p it has to convert that to www.westwalesholidaycottages.co.uk/browse.php?b=1&pl=p. So my host has it disabled.

i realised redirecting only search bots wouldn't work.

in theory google should find penally because, on the first page there's a link to the browse page and from there, there is a link to searchresult.php?b=1&pl=p, where pl=p is a list of cottages where their names begin with p. penally would appear in that search result page.

the problem is, my host doesn't have mod_rewrite enabled. is there an alternative to using mod_rewrite?

  • 0
  teh h4x0r5 said:
That's the whole point of mod_rewrite; it makes it so that dynamic content can be indexed.

586772182[/snapback]

I think you'll find that this is *not* the whole point of mod_rewrite... Just one of its many uses.

  • 0

I'm a big fan of mod_rewrite but I use it mainly for usability.

Since it's been implemented I've seen a noticeable rise in inbound links, people seem much more happy with linking the tidy URLs:

http://www.andrewkendall.com/pages/photogallery/742/9.php

vs

http://www.andrewkendall.com/pages/photoga...picturenumber=9

That said, I've never found a problem with Google indexing dynamic pages, especially since I started using Google Sitemaps.

i.e. "My Chemical Romance Pictures" shows up fine.

  • 0
  Computer Guru said:
sitemaps are the answer. just write a php script that automatically updates hte xml after every change.

586775905[/snapback]

Can you elaborate?

Can you add dynamic URLs to sitemaps? for example: <loc>http://www.westwalesholidaycottages.co.uk/housedetails.php?propref=llan</loc>

  • 0

I'm in agreement with all the guys saying to *not* use ?foo=foo&foo2=foo2 etc.

Check out this tutorial on how to make clean URIs without mod_rewrite: http://www.evolt.org/article/Making_clean_...d_PHP/18/22880/.

Dougal.

This topic is now closed to further replies.
  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Posts

    • Lol, just a month after unveiling and we're already seeing these discounts? Must be getting loads of tacoed units returned or not selling in the first place...
    • Voice of reason. Added bonus, not only we do not get what we want / need - idiotic company management makes idiotic decisions like these, then sales don't happen, then its random Joes that are let go because shareholders and profit. And this being an M$ focused site, you don't even have to go to Samsung to get the idea. I'm looking at you, Windows 10 obsoleted, proper start menu discontinued, telemetrics and AI and bloat shoved down our throats.
    • "This brings the price down from $1219.99 to just $969.99" JUST $969.99?! Trying to make it sound cheaper, does not really make it cheaper, IMO.
    • Windows 11 is getting unexpected new customization feature by Taras Buria Windows' volume and brightness indicators have been a running joke among PC users for many years. Microsoft introduced touch-friendly sliders with the release of Windows 8 in 2012, but they remained unchanged for nearly a decade. Even the initial Windows 11 release in 2021 shipped with those unchanged sliders, sparking discontent among users. However, later, Microsoft finally redesigned the volume and brightness indicator, aligning it with the rest of Windows 11's design. At long last, Windows 11 could breathe a sigh of relief. Now, it looks like Microsoft is not done with this area. Users discovered that Windows 11 preview builds have a hidden personalization feature that lets you change where those flyouts appear on the screen. @phantomofearth shared the discovery on X, showing how Windows 11 will soon let you select where on-screen indicators appear (volume, brightness, virtual desktops, camera, and more). You will be able to choose between the bottom center (current version), the top left, or the top center. For reference, Windows 10 and 8/8.1 have their own on-screen indicators in the top-left corner of the screen. If you miss that behavior, this new feature will soon let you make things a bit old-school. The ability to change the position of on-screen indicators will be available in Settings > System > Notifications > Position of on-screen indicators. Microsoft has not announced this feature yet, so it is possible that more changes will show up in the official announcement. What Microsoft confirmed, though, is a new adaptive power saver mode for tablets and laptops. It is coming to Windows Insiders soon. What do you think about this customization feature? Do you think Windows 11 needs this option? Share your thoughts in the comments.
    • Download The Inclusion Equation: Leveraging Data & AI (worth $21) for free by Steven Parker Claim your complimentary eBook worth $21 for free, before the offer ends on June 24. The Inclusion Equation is a comprehensive, one-of-a-kind guide to merging DEI and employee wellbeing concepts with data analytics and AI. In this book, renowned thought leader and professional keynote speaker Dr. Serena Huang explains exactly how to quantify the effectiveness of new talent strategies by connecting them to a firm ROI estimate, enabling readers to approach and win the favor of higher-ups in any organization with the same effectiveness that marketing and financial departments do. This book is written in a style that is appealing and accessible to all readers regardless of technical background, but with enough depth to provide real insight and strategies. Dr. Serena H. Huang distills her 10 years of Fortune 500 people analytics leadership experience into tools and framework you can leverage to measure and improve DEI and wellbeing in your workplace. Some of the topics explored in this book include: Attract and retain top talent, including Gen Z and Millennials, with tailored DEI and wellbeing strategies Quantifying not only a talent strategy's perceived initial effect on an organization, but also its improvement and expansion over time Turning DEI and wellbeing from illusive corporate concepts to quantifiable metrics Harness the power of AI to create synchronized DEI and wellbeing strategies that maximize ROI Getting serious attention from your CEO and CFO by quantifying HR initiatives Using data storytelling to demonstrate the business impact of DEI and wellbeing Preparing for the future by understanding the role of AI in creating an inclusive and healthy workplace The Inclusion Equation is a complete guide for DEI and wellbeing, covering getting started in measurement to using storytelling to influence leadership. This is the contemporary playbook for any organization intending to substantially improve their diversity, equity, inclusion, and employee wellbeing by leveraging data & AI. This book is also perfect for any data analytics professionals who want to understand how to apply analytics to issues that keep their CEOs up at night. Whether you are a data expert or data novice, as long as you are serious about improving DEI and wellbeing, this book is for you. This free to download offer expires June 24. How to get it Please ensure you read the terms and conditions to claim this offer. Complete and verifiable information is required in order to receive this free offer. If you have previously made use of these free offers, you will not need to re-register. While supplies last! Download The Inclusion Equation: Leveraging Data & AI (worth $21) for free Offered by Wiley, view other free resources The below offers are also available for free in exchange for your (work) email: AI and Innovation ($21 Value) FREE – Expires 6/11 Unruly: Fighting Back when Politics, AI, and Law Upend [...] ($18 Value) FREE - Expires 6/17 SQL Essentials For Dummies ($10 Value) FREE – Expires 6/17 Continuous Testing, Quality, Security, and Feedback ($27.99 Value) FREE – Expires 6/18 VideoProc Converter AI v7.5 for FREE (worth $78.90) – Expires 6/18 Macxvideo AI ($39.95 Value) Free for a Limited Time – Expires 6/22 Excel Quick and Easy ($12 Value) FREE – Expires 6/24 The Inclusion Equation: Leveraging Data & AI ($21 Value) FREE – Expires 6/24 Microsoft 365 Copilot At Work ($60 Value) FREE – Expires 6/25 Natural Language Processing with Python ($39.99 Value) FREE – Expires 6/25 How to Engage Buyers and Drive Growth in the Age of AI ($22.95 Value) FREE – Expires 7/1 Using Artificial Intelligence to Save the World ($30.00 Value) FREE – Expires 7/1 Essential: How Distributed Teams, Generative AI, [...] ($18.00 Value) FREE – Expires 7/2 The Chief AI Officer's Handbook: Master AI leadership with strategies to innovate, overcome challenges, and drive business growth ($9.99 Value) FREE for a Limited Time – Expires 7/2 The Ultimate Linux Newbie Guide – Featured Free content Python Notes for Professionals – Featured Free content Learn Linux in 5 Days – Featured Free content Quick Reference Guide for Cybersecurity – Featured Free content We post these because we earn commission on each lead so as not to rely solely on advertising, which many of our readers block. It all helps toward paying staff reporters, servers and hosting costs. Other ways to support Neowin The above deal not doing it for you, but still want to help? Check out the links below. Check out our partner software in the Neowin Store Buy a T-shirt at Neowin's Threadsquad Subscribe to Neowin - for $14 a year, or $28 a year for an ad-free experience Disclosure: An account at Neowin Deals is required to participate in any deals powered by our affiliate, StackCommerce. For a full description of StackCommerce's privacy guidelines, go here. Neowin benefits from shared revenue of each sale made through the branded deals site.
  • Recent Achievements

    • First Post
      emptyother earned a badge
      First Post
    • Week One Done
      Crunchy6 earned a badge
      Week One Done
    • One Month Later
      KynanSEIT earned a badge
      One Month Later
    • One Month Later
      gowtham07 earned a badge
      One Month Later
    • Collaborator
      lethalman went up a rank
      Collaborator
  • Popular Contributors

    1. 1
      +primortal
      664
    2. 2
      ATLien_0
      270
    3. 3
      Michael Scrip
      218
    4. 4
      Steven P.
      161
    5. 5
      +FloatingFatMan
      157
  • Tell a friend

    Love Neowin? Tell a friend!