Sun confirms OS X Leopard will use ZFS

Sun Microsystems has confirmed that Apple will use Sun's Zettabyte File System (ZFS) for the forthcoming OS X 10.5 Leopard due in October. Sun chief executive Jonathan Schwartz said at a company event in Washington DC on Wednesday that Apple will officially reveal the technology at its World Wide Developer Conference scheduled for San Francisco next week. ZFS is the world's first 128-bit file system, supporting 18 billion times the storage capacity of current-generation 64-bit systems. A zettabyte is equal to 1,024 exabytes or 1,000,000,000,000,000,000,000 bytes.

Sun developed the technology and has released it under an open source licence. The file system is currently deployed in Sun's Solaris operating system. ZFS promises improved data integrity compared with Apple's current Journaled HFS+ file system.

Link: The full story
News source: Vnunet

Report a problem with article
Previous Story

Symantec Releases Beta of Norton AntiBot

Next Story

Analyst: Vista's UAC Will Reform Developers

53 Comments

Commenting is disabled on this article.

I don't think Apple will be marketing ZFS itself, but the tangible real-world things you can do with it, eg. Time Machine. Sure, Apple doesn't invent everything, it just makes it easy for "the rest of us" to use.


Addressing a block of this bytes can be obtained by creating address with 16bits, 32bits or more. The more bytes addressing the more space are allowed (at least in the same partition).

Why don't use a 1024bits or more?. Because every bytes used come for a price, for example a address of 128 bits(16bytes) and a block of bytes :

For every 1024 bytes, 16 bytes are lossed in the address.
For every block readed/written, the system must read the 16 bytes adress, usually they are readed directly from the cache but a high address means more cache used.

So, in theory 128 bits vs 32 bits:

Space used for addressing :128 bits :1.53% and 32bits :0.39%. For example a 100gb disk, with 128bits 1.5gb are used for address vs 0.4gb used by address in 32bits.

Bytes read x block (block of 1KB) : 128 bits :1040 bytes and 32bits :1028 bytes , so 32bits must be 1.15% more fast.

And adding more features (such crc) can sum used bytes x block.

So, at least that you will really need more space, 128bits is a disadvantage :slow and less usable space.

Because you'll be forced to pay the "premium".

Even if they were selling water, they'd sell it as if the water you drink from Apple not only quenches your thirst, but gives you muscles, makes you handsome, lose weight, and become immune to the aging of time.

Got to admit it though, Jobs is the master at finding people stupid enough to buy into the crap that comes out his mouth.

Are most of people stupid that they don't see anything else expect "zettabyte"?
No one even bother to read how the hole system works.

Read

What is ZFS?

ZFS is a new kind of filesystem that provides simple administration, transactional semantics, end-to-end data integrity, and immense scalability. ZFS is not an incremental improvement to existing technology; it is a fundamentally new approach to data management. We've blown away 20 years of obsolete assumptions, eliminated complexity at the source, and created a storage system that's actually a pleasure to use.

ZFS presents a pooled storage model that completely eliminates the concept of volumes and the associated problems of partitions, provisioning, wasted bandwidth and stranded storage. Thousands of filesystems can draw from a common storage pool, each one consuming only as much space as it actually needs. The combined I/O bandwidth of all devices in the pool is available to all filesystems at all times.

All operations are copy-on-write transactions, so the on-disk state is always valid. There is no need to fsck(1M) a ZFS filesystem, ever. Every block is checksummed to prevent silent data corruption, and the data is self-healing in replicated (mirrored or RAID) configurations. If one copy is damaged, ZFS will detect it and use another copy to repair it.

ZFS introduces a new data replication model called RAID-Z. It is similar to RAID-5 but uses variable stripe width to eliminate the RAID-5 write hole (stripe corruption due to loss of power between data and parity updates). All RAID-Z writes are full-stripe writes. There's no read-modify-write tax, no write hole, and — the best part — no need for NVRAM in hardware. ZFS loves cheap disks.

But cheap disks can fail, so ZFS provides disk scrubbing. Like ECC memory scrubbing, the idea is to read all data to detect latent errors while they're still correctable. A scrub traverses the entire storage pool to read every copy of every block, validate it against its 256-bit checksum, and repair it if necessary. All this happens while the storage pool is live and in use.

ZFS has a pipelined I/O engine, similar in concept to CPU pipelines. The pipeline operates on I/O dependency graphs and provides scoreboarding, priority, deadline scheduling, out-of-order issue and I/O aggregation. I/O loads that bring other filesystems to their knees are handled with ease by the ZFS I/O pipeline.

ZFS provides unlimited constant-time snapshots and clones. A snapshot is a read-only point-in-time copy of a filesystem, while a clone is a writable copy of a snapshot. Clones provide an extremely space-efficient way to store many copies of mostly-shared data such as workspaces, software installations, and diskless clients.

ZFS backup and restore are powered by snapshots. Any snapshot can generate a full backup, and any pair of snapshots can generate an incremental backup. Incremental backups are so efficient that they can be used for remote replication — e.g. to transmit an incremental update every 10 seconds.

There are no arbitrary limits in ZFS. You can have as many files as you want; full 64-bit file offsets; unlimited links, directory entries, snapshots, and so on.

ZFS provides built-in compression. In addition to reducing space usage by 2-3x, compression also reduces the amount of I/O by 2-3x. For this reason, enabling compression actually makes some workloads go faster.

In addition to filesystems, ZFS storage pools can provide volumes for applications that need raw-device semantics. ZFS volumes can be used as swap devices, for example. And if you enable compression on a swap volume, you now have compressed virtual memory.

ZFS administration is both simple and powerful. Please see the zpool(1M) and zfs(1M) man pages for more information — and be sure to check out the Getting Started section for a whirlwind tour.

ZFS is already quite snappy on most workloads — and we're just getting started.

P.S.
Just watch some Demos, it amazing how easy to work with this file system, manage everything.
http://opensolaris.org/os/community/zfs/demos/

Nobody is stupid here..the specs look nice for some things indeed. But trust me, 2 things stand out for Apple here, zetabytes and constant-time snapshots for their new Time Machine feature in Leopard.

The problem we all have is that we know what's going to happen in a week at Apple Developer Conference.

They will spin it and spin it, how they invented this, how they improved this, how they can store zetabytes (even though most of the stuff doesn't mean squat to a normal user). They just bought the damn thing and merged it in their product. I don't think this says that Apple is SUPER innovative company. They've always been and will always be a marketeers and designers of nice packaging and UI for someone's elses products they either rip or buy.

Now, I'm not saying they are not good at it, and that I don't want to buy some of their devices (hell I own a couple), but let's not make Apple something they are not. Inventors. They are simply guys that apply other people's innovation to their products and have Steve Jobs that is a MASTER in selling stuff and telling you how you need something that you most likely don't need. He sure knows how to make a presentation slide with numbers that look incredible on paper, but not so when consider numerous facts and overall numbers. This is how they managed to survive, Jobs came back. If he hadn't done that, Apple wouldn't be here today.

So the problem most people here have is eventual crap we'll hear from Jobs' mouth, when we know that even though specs look great on paper, they don't give us anything concrete for at least next few years, when the next revisions of filesystems and OSs will come out.

So basically they say "let's add a useless storage technology in to our OS instead of worthwhile technology". Unless of course this will speed up access, which I doubt it.

Good move Apple. Now if only you could do something to compete with the industry besides iPods.

They're not only competing, they're the aknowledged leaders in innovation and the example the industry tries to follow, my friend. Insofar as a Porsche can compete with a Festiva.

And, not surprisingly, shares of Apple Inc. [AAPL] continue to their upward climb, today gaining $0.421, or 0.34%, to set a new all-time closing high of $124.49 per share on volume of 44,281,961.

Apple's previous 52-Week and All-Time High closing high was $124.069 set yesterday. AAPL's all-time intraday high was set yesterday at $127.61.

Apple's 52 Week Low stands at $50.16, set on July 14, 2006.

Apple's market value currently stands at $107,677,376,520.

This is from NASDAQ.

Laura Goldman, investment advisor, LSG Capital, on May 21, 2007 remarked: "I am putting a sell on Apple, the company that created the iPhone,". AAPL closed at $111.98 that day.

This is dawn of the era of Apple + Google. Better get used to it.

LTD said,
They're not only competing, they're the aknowledged leaders in innovation and the example the industry tries to follow, my friend. Insofar as a Porsche can compete with a Festiva.

And, not surprisingly, shares of Apple Inc. [AAPL] continue to their upward climb, today gaining $0.421, or 0.34%, to set a new all-time closing high of $124.49 per share on volume of 44,281,961.

Apple's previous 52-Week and All-Time High closing high was $124.069 set yesterday. AAPL's all-time intraday high was set yesterday at $127.61.

Apple's 52 Week Low stands at $50.16, set on July 14, 2006.

Apple's market value currently stands at $107,677,376,520.

This is from NASDAQ.

Laura Goldman, investment advisor, LSG Capital, on May 21, 2007 remarked: "I am putting a sell on Apple, the company that created the iPhone,". AAPL closed at $111.98 that day.

This is dawn of the era of Apple + Google. Better get used to it.

THAT's like SOOOO FUNNY! People said that in the 80's....90's... and OMFG!!!! It's gonna happen NOW? WOW! I'm sooo like... super excited!

I like OS X; hate Apple's attitude. Their "fans" are comparable to fans of "The Radier Nation"

So basically they say "let's add a useless storage technology in to our OS instead of worthwhile technology"

For data security and seamless storage, I don't think this is useless at all...

I can see a lot more "useless" features in the latest release of a competing OS in that case. :p

It was strange how the Zettabyte thing took that place in the article summary however, because THAT is among the minor things. Modern file systems other than this one don't exactly have small limits either. Doesn't NTFS support 16 Eb volumes? Now, the doing away of partitions, fast snapshots, the RAID mode, and better checksummed data, that's where it's at to me, as for ZFS.

uh even the largest server clusters in the world have a few hundred terabytes tops. NO ONE has even reached exabytes yet nevermind zetabytes. hell its hard enough for the average user to fill up a 500 GB hard drive, its good that Apples thinking ahead and using these technologies but its just overkill right now.

black_death said,
uh even the largest server clusters in the world have a few hundred terabytes tops. NO ONE has even reached exabytes yet nevermind zetabytes. hell its hard enough for the average user to fill up a 500 GB hard drive, its good that Apples thinking ahead and using these technologies but its just overkill right now.

sun admits themselfes that is physically impossible to fill a zetabyte harddrive. We never would have enough energy to do it

ZFS is interesting, but I don't think the average end-user will see many benefits or advantages, despite the fact they may be there.

Apple is playing an intriguing game of chess the strategy of which will end in Checkmate. MicroHoff whould watch out. Seriously.

alsheron said,
Apple is playing an intriguing game of chess the strategy of which will end in Checkmate. MicroHoff whould watch out. Seriously.

a game of chess that ends with checkmate would certainly be intriguing but you lose points for using the word MicroHoff.

ah, enter quotes that sound meaningful and intelligent but mean absolutley nothing at all. all Apple did was a implement a new (beyond useless) technology into their new OS and make a press release about it so their stock can go up, explain to me how this is "an intriguing game of chess the strategy of which will end in Checkmate"? if anyone in this "intriguing game of chess" has gotten a checkmate youd think its the side with 45X as many pieces on the board

black_death said,
ah, enter quotes that sound meaningful and intelligent but mean absolutley nothing at all. all Apple did was a implement a new (beyond useless) technology into their new OS and make a press release about it so their stock can go up, explain to me how this is "an intriguing game of chess the strategy of which will end in Checkmate"? if anyone in this "intriguing game of chess" has gotten a checkmate youd think its the side with 45X as many pieces on the board :rolleyes:

First, I suggest you actually read about the ZFS instead of, you'd see it is far from 'useless'. Secondly, Apple has said nothing about ZFS. If you had actually read the article, you'd know it was Sun that was quoted in the article.

Lasker said,
so true! Microsoft is still using with VISTA the 14 years old NTFS file system

tell me, what's wrong with ntfs, i'm curious? why do you think microsoft should change, what are the disadvantages of NTFS?

No, NTFS is more advanced then anything anyone else can through at it!
Plus, NTFS had had loads of updates over the years, we're on NTFS 5.5 or something now

I am not an expert in that area but why Microsoft does not do something new and innovative in the file system of windows? is always Apple or other company who develop to their system a new and better software technology

^^For example, Have you seen the new transactional capabilities of Vista NTFS? ZFS only supports transactions for individual writes but it lacks a module to clump multiple writes into a transaction.

Lasker said,
I am not an expert in that area but why Microsoft does not do something new and innovative in the file system of windows? is always Apple or other company who develop to their system a new and better software technology

Well Apple are buying it in, which is somewhat different from developing it themselves ZFS is a really nice file system and i can see big pluses over the Mac's (awful) journaled system but would it really make any difference in the Windows world? NTFS is pretty good all in all - it's not like I wake up every day and think 'i wish my FS wasn't such a POS' - cos it ain't NTFS hasn't remained static either - it's been progressively updated over the years to add new features.

Lasker said,
I am not an expert in that area but why Microsoft does not do something new and innovative in the file system of windows? is always Apple or other company who develop to their system a new and better software technology

Well, you're right about not being an expert, lol. Apple isn't making this. And as much as I support new technology, I don't see this as a "Well my OS has ZFS!" as a way of being "better". At least, not for the moment as we don't have files that big to test it. Maybe in the future, yes, but we we're barely entering the terabyte era. Considering we've been using GB as a measure for HDDs for around a decade if not longer, how can you expect to see zettabytes anytime soon? Unless there's a huge breakthrough in technology, I just don't see it happening.

Besides, other developers have yet to show what there plans are. And NTFS is running dandy as well.

For now, not so big news. I'd really like to look back into it when we can hold files of that size. And if we can, they better be able to hold our entire life memories and junk. I mean, jeez, that's a lot of data. :P

Even with RAID and extreme rapid development of hard-drive technology it doesn't seem plausible that we will be anywhere close to using zettabytes within the timeframe of Leopard (which, based on Apple's track record, is around 18 months after release). It sounds good on paper but it serves no practical use.

Isn't it better to add support for products so that they're already able to support new technologies then to hack them in later? Especially when the new features do not effect the performance of the machines not using it?

Perhaps you should read up on the features of ZFS before you start saying that it has no practical use. Here's a hint: the file system capacity is only one feature of ZFS, there are many others.

Yeah, just read the Wikipedia page for a start - you'll see there's a lot more to it than just supporting larger files. You have to remember to that the film industry (that Apple plays a part in) produces huge files - gigabytes for a few seconds of video for example.

RAID 0 said,
Very impressive. RAID Z? OH yes! I'd buy that for a dollar!
You might consider a name change away from "RAID 0" then. :P

eAi said,
Yeah, just read the Wikipedia page for a start - you'll see there's a lot more to it than just supporting larger files. You have to remember to that the film industry (that Apple plays a part in) produces huge files - gigabytes for a few seconds of video for example.

Ok.

They may produce files that are extremely large by normal standards, but nothing anywhere close to a zetabyte. The film industry has gotten along fine so far. I don't think they've been sitting around going, "Man, I wish we could finish this film, but we just don't have the storage capacity! When will someone invent a file system that supports huge ass files that we haven't come close necessity yet?"

The storage feature has no practical use yet. We have no hardware that can take advantage of it.

And don't tell me, "look at the other features!" Because I'm not talking about the other features; I'm talking about the storage capacity limit.

roadwarrior said,
Perhaps you should read up on the features of ZFS before you start saying that it has no practical use.

I didn't say that ZFS had no practical use (far from it)... I said that support for zettabytes has no practical use. However, as long as there is no performance hit then there is no disadvantage to supporting it. It just seems strange to support something now that won't be around for nearly a decade, particularly when Apple release a new OS every 18-months.

theyarecomingforyou said,

I didn't say that ZFS had no practical use (far from it)... I said that support for zettabytes has no practical use. However, as long as there is no performance hit then there is no disadvantage to supporting it. It just seems strange to support something now that won't be around for nearly a decade, particularly when Apple release a new OS every 18-months.


...still don't get it. You just don't understand.

theyarecomingforyou said,

Don't understand what?

The fact that it's not strange, but rather a very good thing to have support for new technology that would supposedly last a long time in the future?