Jump to content



Photo

ZFS?


  • Please log in to reply
13 replies to this topic

#1 Mindovermaster

Mindovermaster

    Neowinian Senior

  • Tech Issues Solved: 10
  • Joined: 25-January 07
  • Location: /USA/Wisconsin/
  • OS: Mint Debian LMDE
  • Phone: HTC ONE V

Posted 15 March 2014 - 03:47

I remember Orange talking to me about this, but I never fully understood it. What is ZFS? I remember him talking about different pools you can make, and whatnot. It just went through my eyes and left my head. I only remembered it because I saw it while using NAS4Free. (didn't really care for FreeNAS)

 

The "server" is just an old i3 2100 on 1156 socket (H67 iirc) system of mine. I have 2 x 640GB HDDs and 2 x 320GB HDDs.

 

I plan to RAID 1 each set of those. BUT, is ZFS a RAID on its own, or will it enhance that RAID?

 

Please explain it to me in simple english words, please. :laugh:




#2 +macoman

macoman

    Neowinian

  • Tech Issues Solved: 3
  • Joined: 25-December 13

Posted 15 March 2014 - 03:56

ZFS as far as I am concern is a file system for Linux. Kind like NTFS is in Windows with the option for a bootable MBR. Learn more about it: http://en.m.wikipedia.org/wiki/ZFS

#3 Sikh

Sikh

    Neowin Addict!

  • Tech Issues Solved: 2
  • Joined: 11-March 07
  • Location: localhost
  • OS: Windows 7 / 10.8 / Ubuntu Server
  • Phone: Nexus 5 PA 4.4.2 / iPhone 5

Posted 15 March 2014 - 04:03

ZFS is a file system. The raid it  uses is called "RAIDZ". Its a software raid. Zpools are pools(really..) of hard drives. The data is managed obviously managed via the software raid algorithm / level of raid you choose. Its built to handle data corruption, deduplication, copy on right, NFS ACLs (v4), etc. Its a nice file system made by SUN.

One thing about ZFS is you need a LOT of ram. The general rule of thumb is 1 for every T and then enough ram for your actually "server" functionality.

I am building a 8bay RAID Enclosure with a built in 1U server that will be a 16T FreeNAS solution with ZFS being the file system. Im putting a minimum of 32g ECC ram in there. 16 for the ZPool and 16 for the server to use for streaming, file transfers, etc.

Hope this helps.



#4 +macoman

macoman

    Neowinian

  • Tech Issues Solved: 3
  • Joined: 25-December 13

Posted 15 March 2014 - 04:22

ZFS is a file system. The raid it uses is called "RAIDZ". Its a software raid. Zpools are pools(really..) of hard drives. The data is managed obviously managed via the software raid algorithm / level of raid you choose. Its built to handle data corruption, deduplication, copy on right, NFS ACLs (v4), etc. Its a nice file system made by SUN.

One thing about ZFS is you need a LOT of ram. The general rule of thumb is 1 for every T and then enough ram for your actually "server" functionality.

I am building a 8bay RAID Enclosure with a built in 1U server that will be a 16T FreeNAS solution with ZFS being the file system. Im putting a minimum of 32g ECC ram in there. 16 for the ZPool and 16 for the server to use for streaming, file transfers, etc.

Hope this helps.

Can you able to use it as a Linux desktop with 512mb of ram?

#5 Sikh

Sikh

    Neowin Addict!

  • Tech Issues Solved: 2
  • Joined: 11-March 07
  • Location: localhost
  • OS: Windows 7 / 10.8 / Ubuntu Server
  • Phone: Nexus 5 PA 4.4.2 / iPhone 5

Posted 15 March 2014 - 04:50

Can you able to use it as a Linux desktop with 512mb of ram?

 

I would say No. Could you, possibly.  But thats not the idea behind ZFS. 

 

This from the FreeNAS wiki explains why "FreeNAS® with ZFS typically requires a minimum of 8 GB of RAM in order to provide good performance and stability. The more RAM, the better the performance, and the FreeNAS® Forums provide anecdotal evidence from users on how much performance is gained by adding more RAM. For systems with large disk capacity (greater than 8 TB), a general rule of thumb is 1 GB of RAM for every 1 TB of storage. This post describes how RAM is used by ZFS."



#6 +macoman

macoman

    Neowinian

  • Tech Issues Solved: 3
  • Joined: 25-December 13

Posted 15 March 2014 - 06:14

Damn it, I only have in that ancient laptop 40GB of storage and 512MB of ram

#7 Xahid

Xahid

    Anokha Neowinian

  • Tech Issues Solved: 1
  • Joined: 04-November 01
  • OS: Windows 7 Ultimate

Posted 15 March 2014 - 06:30

ZFS is software Raid, & it don't like hardware raid, ZFS like to control all the HDD's by its own.



#8 +snaphat (Myles Landwehr)

snaphat (Myles Landwehr)

    Electrical & Computer Engineer

  • Tech Issues Solved: 29
  • Joined: 23-August 05
  • OS: Win/Lin/Bsd/Osx
  • Phone: dumb phone

Posted 15 March 2014 - 06:49

Personally, I would take a look at the Btrfs article (http://en.wikipedia.org/wiki/Btrfs) as it does a better job summing up the purpose of a copy-on-write filesystem a bit better than the ZFS article.



#9 PGHammer

PGHammer

    Neowinian Senior

  • Tech Issues Solved: 1
  • Joined: 31-August 03
  • Location: Accokeek, MD
  • OS: Windows 8 Pro with Media Center x64

Posted 15 March 2014 - 07:11

ZFS as far as I am concern is a file system for Linux. Kind like NTFS is in Windows with the option for a bootable MBR. Learn more about it: http://en.m.wikipedia.org/wiki/ZFS

Actually, ZFS started as a UNIX file system - and specifically a SOLARIS file system for servers and workstations running the OS.  (It's still the default filesystem in Solaris today.)

 

ZFS is old - in fact, it's almost as old as NTFS, and older than ext3, let alone ext4.

 

Unlike most other filesystems, however, ZFS was designed with both networking AND disk arrays in mind from the beginning (at Sun Microsystems).

It also has extensive audit-trail capabilities - that are designed in.  (It's why Solaris has remained popular in financial services and any area where cradle-to-grave audits are a given.)

 

Btrfs takes a lot from ZFS - however, unlike ZFS, it's not patent-encumbered (remember, ZFS is owned by Oracle).



#10 +macoman

macoman

    Neowinian

  • Tech Issues Solved: 3
  • Joined: 25-December 13

Posted 15 March 2014 - 12:33

Personally, I would take a look at the Btrfs article (http://en.wikipedia.org/wiki/Btrfs) as it does a better job summing up the purpose of a copy-on-write filesystem a bit better than the ZFS article.

i will check it out ;)

Actually, ZFS started as a UNIX file system - and specifically a SOLARIS file system for servers and workstations running the OS. (It's still the default filesystem in Solaris today.)

ZFS is old - in fact, it's almost as old as NTFS, and older than ext3, let alone ext4.

Unlike most other filesystems, however, ZFS was designed with both networking AND disk arrays in mind from the beginning (at Sun Microsystems).
It also has extensive audit-trail capabilities - that are designed in. (It's why Solaris has remained popular in financial services and any area where cradle-to-grave audits are a given.)

Btrfs takes a lot from ZFS - however, unlike ZFS, it's not patent-encumbered (remember, ZFS is owned by Oracle).

interesting :)

#11 OP Mindovermaster

Mindovermaster

    Neowinian Senior

  • Tech Issues Solved: 10
  • Joined: 25-January 07
  • Location: /USA/Wisconsin/
  • OS: Mint Debian LMDE
  • Phone: HTC ONE V

Posted 15 March 2014 - 17:17

I found the message from ZFS, with Orange.

 

 

FreeNAS, however, uses a more intelligent type of software RAID. Its RAID capabilities are tied directly to the filesystem: ZFS. There are many advantages to modern filesystems like ZFS and BTRFS which meld the domains of RAID software and the filesystem. Because these filesystems know about the files stored on disk - not just the raw blocks that hardware RAID and traditional software RAID sees - they can make more intelligent decisions about file storage. They are also orders of magnitude more fault-tolerant in most scenarios because losing a single disk in the RAID does not necessarily mean the whole array has to be stopped to prevent damage to the filesystem. Since the filesystem is aware of the damage - not just the RAID solution - it can work in a degraded state until the failed hardware can be replaced. This new generation of filesystem also allows you to alter the RAID online (including switching between RAID types), pair disks of different sizes (with a performance penalty, of course), and many other things that are not possible with a separated RAID solution and filesystem.

 
...but the filesystem has been stable for quite some time. Sun Microsystems started supporting it in Solaris 10 Update 2 back in 2006, after it had been tested in OpenSolaris since early 2005. FreeBSD 7 gained support for ZFS (imported from OpenSolaris) in early 2007. Admittedly, FreeBSD 7 kind of a dog - it was still recovering from the incredibly disruptive changes the FreeBSD Project made when they implemented AMD64 support in FreeBSD 5 - but that had no bearing on its newfound ZFS support. Nearly 2 years later, in late 2008, FreeBSD 8 moved ZFS from experimental to the base system, and started offering enterprise-grade support for it, much like Solaris had been doing for several years. Since Oracle bought Sun Microsystems and made Solaris closed-source, the OpenZFS Project was started to carry on ZFS development. Today ZFS is supported on most major operating systems - including Solaris, Illumos, FreeBSD, NetBSD, and Linux (through the out-of-tree kernel module ZFS-on-Linux) - and has seen a number of large-scale deployments (by companies such as Netfilx and Yahoo). I recommend it without reservation.
 
BTRFS has similar goals to ZFS, but it is Linux-only and still undergoing heavy development. The stability and reliability of BTRFS has been questioned many times, and up until a year ago the undisputed answer was that it was not ready for wide-scale deployment. That recommendation has changed in the last year as the filesystem has matured. Novell believes BTRFS is stable enough that they started offering support for it in SUSE Linux Enterprise Server and SUSE Linux Enterprise Desktop through their support contracts late last year. Following suite, Red Hat will be offering official support for BTRFS in Red Hat Enterprise Linux 7, which is due to be released early next year. Arch Linux, Fedora, and OpenSUSE have offered it as an option since it was introduced in mainline (Linux 2.6.29 - 22 March 2009), and OpenSUSE 13.1 will begin offering it as the root filesystem when it is released later this year. It has been available in Debian since Squeeze (2011), and had its scary "experimental" warning removed in Wheezy (2013). Since BTRFS is still under heavy development, has not reached feature parity with ZFS, and only recently achieved stability, there are no major deployments of it of which I am aware. However, I currently use BTRFS on several systems because it is much faster than ZFS, supports transparent compression (which actually makes it perform better when coupled with a reasonably strong CPU), and has much better (simpler) command line utilities than ZFS.
 
Since you need your disks to be formatted before you create their entries in fstab, I'm going to answer the second question first.
 
Installing ZFS: To use ZFS in Debian, you either need to be running the kfreebsd port (either kfreebsd-i386 or kfreebsd-amd64) or install the out-of-tree kernel module ZFS on Linux (amd64 only). Due to licensing issues (Sun Microsystems licensed ZFS under the CDDL, which is not compatible with the GPL), ZFS on Linux is permanently maintained by Lawrence Livermore National Laboratory as an out-of-tree kernel module. It will be included in the main archive in Debian Jessie, but for now you can install it on Debian Wheezy using the Debian repository provided by the ZFS on Linux project.
 
Using ZFS: Regardless of whether you chose the Linux or kFreeBSD route for getting ZFS in Debian, its usage is the same. The filesystem is configured using two primary commands. The zfs command is used to configure the filesystem at a high level. Use it to format, deallocate, and set advanced parameters on any physical disk. The zpool command is used to configure your storage pools. Use it to add or remove disks from a RAID, change the type or configuration of a zpool, etc. I recommend that you read the man pages for each command first-and-foremost. They are very well documented, and contain many examples of common usage. Speaking of common usage, section 20.2.2.2 of the FreeBSD Handbook walks you through an example of setting up a RAID-Z. Just in case that isn't enough documentation, Aaron Toponce has extensive ZFS on Linux documention (focusing on Debian Wheezy specifically) on his website.
 


#12 +snaphat (Myles Landwehr)

snaphat (Myles Landwehr)

    Electrical & Computer Engineer

  • Tech Issues Solved: 29
  • Joined: 23-August 05
  • OS: Win/Lin/Bsd/Osx
  • Phone: dumb phone

Posted 15 March 2014 - 19:49

^ Unless things have changed recently, orange is right, zfs is more stable than btrfs. Also, wow, that is detailed  :laugh:



#13 OP Mindovermaster

Mindovermaster

    Neowinian Senior

  • Tech Issues Solved: 10
  • Joined: 25-January 07
  • Location: /USA/Wisconsin/
  • OS: Mint Debian LMDE
  • Phone: HTC ONE V

Posted 15 March 2014 - 20:10

Off-Topic: Anyone know where Orange (Karl) went? Hasn't been on since Nov.



#14 +snaphat (Myles Landwehr)

snaphat (Myles Landwehr)

    Electrical & Computer Engineer

  • Tech Issues Solved: 29
  • Joined: 23-August 05
  • OS: Win/Lin/Bsd/Osx
  • Phone: dumb phone

Posted 15 March 2014 - 21:27

^ Nope, he disappeared a bit after I started actually posting.