What file system for RAID?

[thebrad]

Hi all,

I'm building a home file server soonish. It will have an atom motherboard with 4gb ram, 4x 1tb hdds and I'm going to run Arch Linux on it. I'm going to put the four 1tb hdds into a software raid 5 (no hardware support) just wondering what file system I should on the raid array. I'll be storing everything from movies to documents. What do you recommend?

Also since the 4 hdds will be in a RAID and this will use all of the sata ports. Where do you recommend I should put the OS, on the RAID or on an IDE drive?

Thanks,

Brad.

[markwolfe Veteran]

Since I don't think you can boot from a software RAID (and I may be wrong) at least at the GRUB level, I might just throw in an IDE drive for the core OS, then that leaves the 1TB SATA drives as fully RAID drives.

As for the filesystem, I don't see any reason to use something odd. Just a standard ext4 should do fine.

[Elv13]

I did the same thing (4x 2TB) and I took BTRFS. It is not perfect yet, but your data should be safe. You don't have to RAID your whole disk. BTRFS replace LVM and RAID, so it is more flexible, as it is data aware. You can select folder or subvolume to raid and leave unessential data with only one copy. On top of that, you have copy on write, so you can snapshot folders and file (this will get better over time). Full checksumming, block level compression (compress your "backup" folder as you don't need it to be fast, live pool resize and some more feature. Well, OpenSolaris+ZFS and Linux+BTRFS are the two best choices, but btrfs will get better, it is not complete yet, but it is forward compatible, so you wont lose your drive if you update Linux.

[00000]

You can make a small (~32MB) boot partition on the first hard drive that has the kernel on it and use raid 5 for the root partition. On the other drives you can either duplicate the boot partition to have a backup to boot from if the primary drive dies, or use it for swap.

[Vlad]

This is easier than everyone seems to think (although I would recommend using a flash disk for your /boot and / and mounting leaving your RAID array just for data). Using BTRFS is a terrible idea. Avoid the risk and don't do that.

Create two partitions on each hard drive. The first should be 256MB or so, and the rest whatever is left.

Both partition types should be linux software raid (type 0xfd I believe).

Create a software RAID1 array consisting of /dev/sd{a,b,c,d}1.

Format /dev/md0 something not journaled, such as ext2. This is /boot, you don't need journaling. Label it or use it's UUID for mounting it.

Mount /dev/md0 to /boot. Install your kernel.

Install grub to EACH hard drive, ie, /dev/sda, /dev/sdb, /dev/sdc, /dev/sdd. This will allow you to boot any disk if something fails.

Create a software RAID5 array consisting of /dev/sd{a,b,c,d}2. Label it or use it's UUID for mounting it.

Format /dev/md1 something that isn't experimental (wtf, BTRFS? Last I heard they were very explicit that the disk format was NOT finalized, as in, NOT forward compatible. Maybe if this was a throw away VM, not my file server). I recommend xfs or ext4, in that order. Are you trying to mess around or put your data somewhere where it will be in 6, 12, or 48 months?

I really recommend using flash instead though, it's good policy to keep your data separate from your system on your fileservers.

[Elv13]

The format should be finalised during this developemnt cycle, and the format is forward compatible since it entered the linux kernel in 2.6.32, the page is just not up to date. Btrfs is about to leave experimental stage, devs say that all data loss bugs have been ironed out and that it is now safe, even if it may not be totally final.

MeeGo use it as the default format, Fedora will probably switch to it for F15 and Ubuntu may even use it as default for 10.10 if everything go well and Chris Mason is cool with that, but it will probably be 11.04.

[Vlad]

The format should be finalised during this developemnt cycle, and the format is forward compatible since it entered the linux kernel in 2.6.32, the page is just not up to date. Btrfs is about to leave experimental stage, devs say that all data loss bugs have been ironed out and that it is now safe, even if it may not be totally final.

MeeGo use it as the default format, Fedora will probably switch to it for F15 and Ubuntu may even use it as default for 10.10 if everything go well and Chris Mason is cool with that, but it will probably be 11.04.

I would be extremely surprised if anything switched to a filesystem as relatively untested as BTRFS for their defaults this soon. I'm not saying it's not a fine filesystem with lots of nifty features, but I'd remove my UPS power supplies before I trusted BTRFS on production systems.

[ToneKnee]

As we're talking about RAID setups, I recommend RAID-5.

[Elv13]

I would be extremely surprised if anything switched to a filesystem as relatively untested as BTRFS for their defaults this soon. I'm not saying it's not a fine filesystem with lots of nifty features, but I'd remove my UPS power supplies before I trusted BTRFS on production systems.

In this case, your statement make no sense, BTRFS is a CoW file system (copy on write). Standard file system (btree) have to rely on a journal to prevent too bad dataloss on power outage. CoW file system never overwrite data, it is stored alongside. If a power outage occur, then the CoW tree will be reversed until the safety checksum is matched. By design, even the most buggy file system would tolerate power failure better than the best btree only file system. The ability to go back in time until you are stable is the greatest safety level for a file system. Better than trying to repair the last writing error by having a copy of it in the journal. If the journal is the one corrupted, it will fail, causing data loss.

[REM2000]

i agree with mark i would go with a bootable IDE and then use EXT4 on the RAID, if your storing that much data then i wouldn't use BTRFS until it's fully tested and released.

ZFS is a great file system however there is a bit of an overhead with it, FreeBSD is also a good alternative if you want to go the ZFS route.

[Elv13]

I tested the 3 and benched them before picking BRTFS, FreeBSD (as of 8.0) had performances issue with ZFS and used a really old version of it, not the latest and greatest with deduplications support. OpenSolaris 2009.06 is still the best OS to run ZFS,

[Vlad]

In this case, your statement make no sense, BTRFS is a CoW file system (copy on write). Standard file system (btree) have to rely on a journal to prevent too bad dataloss on power outage. CoW file system never overwrite data, it is stored alongside. If a power outage occur, then the CoW tree will be reversed until the safety checksum is matched. By design, even the most buggy file system would tolerate power failure better than the best btree only file system. The ability to go back in time until you are stable is the greatest safety level for a file system. Better than trying to repair the last writing error by having a copy of it in the journal. If the journal is the one corrupted, it will fail, causing data loss.

Wooooosshh.