How does Youtube manage to store enormous data.

[Del-Funky Smelly Be.]

Now everybody knows there are really millions of videos on Youtube. Videos in HD require , i think processing power because if someone scale down HD to 360p then maybe it works by recoverting video or just there are already samples of that HD video ( which I think is incredibily large storage it has) How can you justify ?

[Stetson]

It's owned by Google, and Google has over a million dedicated servers, which is around 2% of the entire world's servers. (source)

They generate a lot of revenue from advertisements, and they also have Google's resources to lean on.

[Del-Funky Smelly Be.]

Does server use traditional hard disks like we do?

[Southern Patriot]

Yes, basically like the hard drive in your computer, but usually many of them in a RAID array. And if you think Youtube has it rough, you should read up on the estimated daily traffic on Usenet. Large Usenet servers like Giganews likely dwarf Youtube in capacity.

[Stetson]

Does server use traditional hard disks like we do?

Yep, and as probably the world's largest owner of consumer-grade hard drives they collect a lot of interesting data on hard disk lifetimes, reliability, etc: Google Paper on Hard Disk Failures

[smachat]

Most of the data are in some kind of SANs... However how many times are they replicated on what number of physical places would be probably part of google's know how :)

But to answer your questions: the harddrives used in the SANs are quite similar (but not completely same) as those used in desktop computers/laptops. They are mostly SCSI or SAS (faster and more reliable brother of SATA), 7,2 or 10 or 15krpm (the faster are used mostly for databases or mailservers) 3.5' or 2.5's...

[Del-Funky Smelly Be.]

Wikipedia's article on Usenet says that its traffic is 8TB which I think is only fraction of Youtube. This article (2006) claims 200 tb per day.

[+John Teacake MVC]

It doesnt! its all a myth! hehe

[Del-Funky Smelly Be.]

One more msyterious question is that if there is a video that is stored on hard disk , then how does youtube manage to cop when 100 users are viewing that same video. It would slow down harddisk/server speed immensly. :whistle:

[+John Teacake MVC]

You really do have a lot to understand how the things like this work on such an immense scale I haven't even scratched the surface when I say the words like "SAN Replication" and "Load Balancing" and "Cache". Do you think there is One Server with One Copy of Each file. Its replicated round the world. I cant even begin to explain how nieve you are about how the whole thing fits together. Theres sooooooooo much to think about. The most viewed videos for example might be cached in Memory . . . Have you thought about that?

[CPressland]

One more msyterious question is that if there is a video that is stored on hard disk , then how does youtube manage to cop when 100 users are viewing that same video. It would slow down harddisk/server speed immensly. :whistle:

Cache servers with large amounts of memory and rapid storage to deploy the most popular videos? Just an idea. :p

[sbauer]

Here's some info. http://highscalability.com/youtube-architecture

[Southern Patriot]

Wikipedia's article on Usenet says that its traffic is 8TB which I think is only fraction of Youtube. This article (2006) claims 200 tb per day.

Either way, that usenet traffic number is DAILY uploads, and some premium usenet servers have over 2 years of retention. Youtube's number is simply the amount of traffic they serve to users (which is basically no relation to what they actually store). You can't directly measure the downloads from usenet since it is served by so many different servers around the world, but you can be sure that the daily downloads from usenet would be many hundreds or thousands of times larger than the daily uploads.