• 0

What's the best way to..


Question

What's the best way to hide a web page from the general public but still allow it to be indexed by search engines?

I may even want to hide the page from the search results of search engines.

Is this possible? I couldn't find anything of the Google :no:

Link to comment
Share on other sites

19 answers to this question

Recommended Posts

  • 0

You can't hide a page but allow search engines--essentially when bots crawl your site, it's because it's 'public' (or the robots.txt allows it.) You can hide a page and by default it just won't be able to crawl because it just doesn't know it exists.

If you want to allow humans but disallow search indexing then you can setup a robots.txt file.

Link to comment
Share on other sites

  • 0

Why though? I can't figure out why you would want to do that...

If a search engine is able to find it, the general pubic would be able to see it...

Only thing I can think of is to use a server side script, and if the useragent is a bot, show the content, otherwise, show nothing..

Link to comment
Share on other sites

  • 0

I suppose you could use that but if you hide it from humans but allow it for search engines, that just kind of seems like you're trying to manipulate your SEO.

What are you trying to do anyway? :)

Link to comment
Share on other sites

  • 0

I know that Google has checks for this and your pagerank will be hurt heavily by doing this. I'm sure other search engines do similar checks.

Link to comment
Share on other sites

  • 0

Seems like you're trying to manipulate your SEO.

:shiftyninja:

Basically I have a network of websites that I manage, which are growing in numbers everyday.

Individually they produce some good seo figures. What I want to do, is tie them all together but not visually, so they rank higher as a group which will significantly benefit any new sites that are created and tied into the network of websites.

Link to comment
Share on other sites

  • 0

I know that Google has checks for this and your pagerank will be hurt heavily by doing this. I'm sure other search engines do similar checks.

That's what I heard too! But I haven't been able to find a source. Do you have one?

I know they look for things like hidden text etc etc - but I haven't seen anything around hiding a full page from users but not from search engines.

I'd advice you to not then. If you plan on leaving the page completely unavailable to render in the browser (ie: using display: none;) but still allow it to be indexed by SE.. you're only hurting yourself.

http://en.wikipedia.org/wiki/Search_engine_optimization#White_hat_versus_black_hat

Cupckaes thats too long to read! :p I tried but I'm really tired and my eyes are going blurry from looking at code all day :(

Does what i'm trying to do have a name as such that I could do some research on?

Link to comment
Share on other sites

  • 0

It's not that that long. I gave you the anchor link to the section you should be looking at.

It's called Blackhat SEO and honestly, if this is what you're trying to do, don't expect any help from me, that's for sure.

Link to comment
Share on other sites

  • 0

It's not that that long. I gave you the anchor link to the section you should be looking at.

It's called Blackhat SEO and honestly, if this is what you're trying to do, don't expect any help from me, that's for sure.

Yeh I know all about Black Hat techniques.. but I'm trying to find something specific on exactly what I was going to do.

if this is what you're trying to do, don't expect any help from me, that's for sure.

Woah! Relax.. it's purely fact finding mission.. of course im not out to cheat my way to the top - im all for karma and wouldn't want someone to steal my hits illegitimately either.. hence why im posting up first to see if there is a legitimate way or not.

Sorry for the confusion k? :) I'm definately the last person to start cheating at things.

I just want to find a way to tie all my sites together (which is ok in todays SEO rules) but not allow it to be seen by my visitors.

Link to comment
Share on other sites

  • 0

You can't hide it from the public but allow it to be indexable. In order for a bot to index a site, it has to be accessible by the public.

That's basically what it comes down to. When you say a way to "tie" your sites together, what is it that you're doing exactly? Just curious.

Link to comment
Share on other sites

  • 0

Couldn't you just check if it's a bot server side and if it is add a link to page that connects all your pages?

Sorry im not following? I can add a bit of code that tells the crawler that there are other links connected to this page?

What about creating a page and only make accessible through login? There's got to be a legitimate way to do this. Or am I seriously chasing my tail?

Link to comment
Share on other sites

  • 0

I just want a clean way of linking all my sites up to create my own network. Which is what sites like smh.com.au and news.com.au do.

But I want it kinda hidden. I can't put links all of their sites. You know what I mean?

They basically have their own network so when they release a new site, that new site is picked up and indexed quicker by search engines.

Just like the envato network :(

My main clientele are small business owners who can't afford the monthly costs of maintaining there seo. I want help them rank better in search engines by creating my own network which they can feed off :(

Trying to help the little guys so they don't have to invest so much money at the start. Something ridiculous like 50 - 80% of businesses fail within the first year of opening.

And it sucks cos my clients are legit (if I know or have a feeling they are dodgy I refuse to take them on). And so many times I hear how the dodgy ones kill it on search engines cos they have the money to play with seo

I just want a clean way of linking all my sites up to create my own network. Which is what sites like smh.com.au and news.com.au do.

They basically have their own network so when they release a new site, that new site is picked up and indexed quicker by search engines.

Just like the envato network :(

My main clientele are small business owners who can't afford the monthly costs of maintaining there seo. I want help them rank better in search engines by creating my own network which they can feed off :(

Trying to help the little guys so they don't have to invest so much money at the start. Something ridiculous like 50 - 80% of businesses fail within the first year of opening.

And it sucks cos my clients are legit (if I know or have a feeling they are dodgy I refuse to take them on). And so many times I hear how the dodgy ones kill it on search engines cos they have the money to play with seo cos they rip people off.. I see it almost every week. It's stupid.

Link to comment
Share on other sites

  • 0

What you are trying to do is generally referred to as "web ring" or "link farm" and Google hates them. It will be a drain on your PR. Unless you are ok with risking what you have gained so far I would not recommend that.

That being said a link from a popular website will bring you traffic but not necessarily PR. If that was the case PR would be irrelevant simply because of the amount of abuse it will suffer because of linking.

Link to comment
Share on other sites

  • 0

What you are trying to do is generally referred to as "web ring" or "link farm" and Google hates them. It will be a drain on your PR. Unless you are ok with risking what you have gained so far I would not recommend that.

That being said a link from a popular website will bring you traffic but not necessarily PR. If that was the case PR would be irrelevant simply because of the amount of abuse it will suffer because of linking.

Hey sweetsam - thanks for getting me the terminology. Just read up on it and understand what you were all saying now. That sucks though.. cos I know big businesses do it everyday. I think I'm going to have to create a new site like a directory and just link them all to that and have that as my second portoflio or something.

Will need to put more thought into it I think. Thanks for all your help guys!

Link to comment
Share on other sites

This topic is now closed to further replies.
  • Recently Browsing   0 members

    • No registered users viewing this page.