Jump to content



Photo

So I just got four WD Red 4TB drives...


  • Please log in to reply
77 replies to this topic

#46 +BudMan

BudMan

    Neowinian Senior

  • Tech Issues Solved: 106
  • Joined: 04-July 02
  • Location: Schaumburg, IL
  • OS: Win7, Vista, 2k3, 2k8, XP, Linux, FreeBSD, OSX, etc. etc.

Posted 08 May 2014 - 15:31

@Draconian - not sure I would consider it a thread hijack, since we are discussing lots of things related to storage.. There didn't seem to be a actual topic when the thread started other than hey I got some new toys ;) So to me the thread is free to go in any direction the discussion takes it ;)

There are advantages in drivepool has over storage spaces imho. One being ability to balance storage across disks in the pool. Another being ability to duplicate folders or files across disks in the pool. Done at file level not block level like storage spaces slabs that stripe data across the disks in the pool

Another being folder/file placement, where you can be specific on what disk a folder/file is placed in the pool. That with integration of of their scanner software, where all files can be moved off a disk that shows possible failure (if space in the pool permits of course)

While spaces does have the ability for redundancy, my understating if you want to change the mode you have to redo the whole pool. With stablebit software I can say hey this folder or file is copied to X number of disks in the pool. If I at latter time want to change that, bing bang zoom its done.

Big draw back for me with spaces vs the alternatives is I can not just take a drive out of the pool and connect it to a pc and get my files. Sure you can move the whole pool to another pc. But with stablebit for example.. I can just pop a drive out of pool and connect it to any other OS that reads ntfs and see my files in their folders with their exact name, etc. The pooling is is handled outside of the normal file structure.

With spaces all data on the disk is gone if you place it in a pool. With drivepool I can take any ntfs disk with files on it and add it to the pool while still having access to the original files, and then just move them over into the pool, etc. This makes it much simpler to implement over spaces if you ask me.

I am also not 100% sure but I believe with spaces and the use of slabs and the striping it does, etc. I don't think you can fully utilize the full space of the disks if they are different sizes.. With drivepool, I could have a 500, a 1TB or 2TB and 3TB disk all connected and get all 6.5 as space.

For the $20 cost of the software it to me just blows away what you can do with spaces from lots of different aspects. I would bundle it with their scanner software to be sure, $35 for both is killer price point. You can also just move your license over to another machine, etc.

"Redundancy does come at a price, and it's not really that high "

If your talking 1, or 10 or 0+1 sure you have a redundancy -- if your talking 5 or 6, etc. than not so much you have parity not redundant copies. You can hope that your data can be restored from the parity. Depending on the size of your array the odds get worse and worse that it will happen.

While to a company that needs its data online, I agree raid is small price to pay. When your talking non critical files in a home, that don't even really justify backup I just do see it being the best bang for the buck. But its your data, your money, etc. If you want to run raid - have fun, its interesting and sure provides some mitigation of loss of data on disk failure. I just don't believe it makes sense cost wise in the home.

The best parts about these threads that head in the direction the discussion takes them is the exchange of ideas - you have yours, I have mine lets discuss and let the readers get exposed to both points of view. To me the typical raid 5 for sure is on its last legs - as disks become bigger and bigger and cheaper and cheaper it really just is not cost effective means of mitigation of failure..

Lets make sure even if we debate over the actual definition of redundancy -- I think we can all agree it is not a "backup" which if your files are critical they should have.. A simple virus/malware infection could wipe out your whole array in a heartbeat.. Delete, corrupt, encrypt all your data -- now what does your redundancy get you? This goes for 1 or 10 not just 5.. Same goes for my "pool" - run the wrong file, get the wrong driveby and WTF my data is not there any more -- you want how much money to decrypt them ;) etc. etc. Critical files require "backup" Your copy of sound of music -- not so much ;)

Raid has a specific use, the mitigation of lack immediate access to your files on failure. For a home I don't see the need of this principle.. There is normally a very small subset of data that is "critical" and none of it really requires access 24/7/365.. If your media library is offline because of crash, what does it really matter. It's up to the user to determine if keeping their data for easy access justifies the cost of running raid. To me, it just doesn't - not in the home setup.

If I have a loss of disk in my pool, all I have to do is restore the files that were on that specific disk. Now hopefully with watching the disks for signs of failure, just change out because of more space required so I take out the old smaller and replace with faster bigger I never actually have a loss. But if I do, the files are not "critical" anyway - so I take the opportunity to put in a bigger faster drive and either restore the files I lost at some point or not. They are not critical - so maybe I don't put back that copy of ST-TOS.. Or if I do, so what if it takes 6 months?


#47 Auditor

Auditor

    Neowinian

  • Joined: 16-May 10
  • Location: Alberta, Canada

Posted 08 May 2014 - 15:32

I got two of these, one 4TB and another 2TB. I wanted both to be 4 TB but these were expensive. I paid almost $200 CAD for 4 TB and $140 CAD for 2 TB. I use these in my Qnap NAS and load tons of music and movies and it works flawlessly. My next plan is to get another 4 TB and replace current 2 TB.



#48 Klownicle

Klownicle

    I get a custom title? Sweett.

  • Joined: 22-June 02

Posted 08 May 2014 - 15:56

Loosing 4tb of personal data when a drive dies.  Priceless.

 

I hope your running a raid 10 or raid 5 with that.  Raid 10 would likely be more recommended as I hear bad things with Raid 5 Rebuilding and larger drivers.  I run 8x 2tb wd red drives in a Raid 50 for 12tb. 



#49 +BudMan

BudMan

    Neowinian Senior

  • Tech Issues Solved: 106
  • Joined: 04-July 02
  • Location: Schaumburg, IL
  • OS: Win7, Vista, 2k3, 2k8, XP, Linux, FreeBSD, OSX, etc. etc.

Posted 08 May 2014 - 16:05

Actually your 2TB was more expensive than your 4..

200/4TB vs 160/2TB, so 5 cents a GB for the 4 vs 0.08 a GB for your 2..

If you really need 8, prob would of been cheaper to just wait until you could of gotten another 4 ;)

"Loosing 4tb of personal data when a drive dies"

I really doubt he has 4TB of "personal" data - more like 4TB of movies/music.. Now if they are HOME Movies, then raid is not a backup.. Better to put the money into BACKUP of that space vs spending money on some array with parity..

Raid 50, if you loose 2 disk in the same 5, you just lost the whole 12TB.. Ouch.. 8 disks is lot of chance of failure ;) Did you buy them all at the same time, are they all from the same batch? That ups the likely hood of very close failures if you ask me. What is cost of expansion of that system to say 14TB of storage vs your 12?

#50 +LogicalApex

LogicalApex

    Software Engineer

  • Tech Issues Solved: 8
  • Joined: 14-August 02
  • Location: Philadelphia, PA
  • OS: Windows 7 Ultimate x64
  • Phone: Nexus 5

Posted 08 May 2014 - 16:17

Raid has a specific use, the mitigation of lack immediate access to your files on failure. For a home I don't see the need of this principle.. There is normally a very small subset of data that is "critical" and none of it really requires access 24/7/365.. If your media library is offline because of crash, what does it really matter. It's up to the user to determine if keeping their data for easy access justifies the cost of running raid. To me, it just doesn't - not in the home setup.

 

Of course, the structure of a home setup will vary wildly depending on the user and their specific situation. Especially the user base here on Neowin...

 

To me, ensuring my critical systems and data can survive a drive failure is important. At the very least, it helps to keep my wife happy. When you have everything in the house running via the server and network you need to ensure its continued availability even when you are away at work... Missed emails, inaccessible live tv, lost phone service, and others can be problematic...

 

I think, like everything else in tech, the user should make the choice that best fits their needs...



#51 TPreston

TPreston

    Neowinian Senior

  • Tech Issues Solved: 2
  • Joined: 18-July 12
  • Location: Ireland
  • OS: Windows 8.1 Enterprise & Server 2012R2/08R2 Datacenter
  • Phone: Nokia Lumia 1520

Posted 08 May 2014 - 16:18

I really feel sorry for the people who pay close to a thousand$ for a pathetic 4 bay qnap nas.

Forget about adding a 15$  card off ebay that can run rings around a 1000$ 4 bay pos with 8 ports,RAID6,Battery Backed Cache,... You know you could have got a 12 bay SAS expander off ebay for like 80$ hell add another 30$ and you could get a sodding dual domain module for MPIO  :rofl:

 

EOL Enterprise gear > any consumer storage on the market at a fraction of the price. These boxes are one of the biggest rip-off's around

 

When you arnt restricted by the insane prices of synoligy and qnap you wont need to buy expensive drives in the hope that your 4x4 RAID 5 wont die during rebuild because you can just add an extra drive and get double parity  (Y)

 

You can say that there is the increased complexity of the OS but that comes with the advantage of having something more powerful than PHP for software



#52 Klownicle

Klownicle

    I get a custom title? Sweett.

  • Joined: 22-June 02

Posted 08 May 2014 - 16:28

Actually your 2TB was more expensive than your 4..

200/4TB vs 160/2TB, so 5 cents a GB for the 4 vs 0.08 a GB for your 2..

If you really need 8, prob would of been cheaper to just wait until you could of gotten another 4 ;)

"Loosing 4tb of personal data when a drive dies"

I really doubt he has 4TB of "personal" data - more like 4TB of movies/music.. Now if they are HOME Movies, then raid is not a backup.. Better to put the money into BACKUP of that space vs spending money on some array with parity..

Raid 50, if you loose 2 disk in the same 5, you just lost the whole 12TB.. Ouch.. 8 disks is lot of chance of failure ;) Did you buy them all at the same time, are they all from the same batch? That ups the likely hood of very close failures if you ask me. What is cost of expansion of that system to say 14TB of storage vs your 12?

 

You take a gamble with any raid 5 loosing 2 discs.  To date, I have had one drive go bad and rebuild without issue.  At least with the raid 50, you can loose "2" and still have a working array as long as they aren't in the same set.  And as far cost in upgrading to larger drives I bought the 2tb drives when the red were just coming out, at the time no 4tb existed, maybe the 3.  I can't recall for sure. 

 

We all take risks and have our comments on raid setups.  I'm sure the OP is willing to take his/her risks as well.  Before I went to raid 50, I just had a jbod setup.  To each their own.



#53 +BudMan

BudMan

    Neowinian Senior

  • Tech Issues Solved: 106
  • Joined: 04-July 02
  • Location: Schaumburg, IL
  • OS: Win7, Vista, 2k3, 2k8, XP, Linux, FreeBSD, OSX, etc. etc.

Posted 08 May 2014 - 16:42

"Missed emails, inaccessible live tv, lost phone service, and others can be problematic..."

That is not the same as "storage" So you run your OS on a raid 1, 2 SSDs for example this is low space requirements if you need a PC online in case of failure. Or just backup the OS and restore from image to new disk you either have on the shelf or stop by the store on the way home and buy. This seems more like internet access in your examples to me.. Nothing to do with your PCs on your network from those examples... More your modem or router need a spare, etc ;)

My plan if my server died that has my router on it in VM, is to just change over one of my AP to router mode again. If the modem died, have to stop by the computer store or order 1 with over night delivery, etc. ;) She could always access email via hotspot fire up off phone service on her laptop or ipad. If either of those die there is other equipment she can use. If the server did die, the only loss of access would be to media library - if she really wants to listen to a song or what movie that was in the library - she can pull out the optical media copy of it, etc.

"To each their own."

Exactly - run raid if you want, its your money, its your data.. The fun is in the discussion of different opinions and options.

#54 +LogicalApex

LogicalApex

    Software Engineer

  • Tech Issues Solved: 8
  • Joined: 14-August 02
  • Location: Philadelphia, PA
  • OS: Windows 7 Ultimate x64
  • Phone: Nexus 5

Posted 08 May 2014 - 18:11

"Missed emails, inaccessible live tv, lost phone service, and others can be problematic..."

That is not the same as "storage" So you run your OS on a raid 1, 2 SSDs for example this is low space requirements if you need a PC online in case of failure. Or just backup the OS and restore from image to new disk you either have on the shelf or stop by the store on the way home and buy. This seems more like internet access in your examples to me.. Nothing to do with your PCs on your network from those examples... More your modem or router need a spare, etc ;)

My plan if my server died that has my router on it in VM, is to just change over one of my AP to router mode again. If the modem died, have to stop by the computer store or order 1 with over night delivery, etc. ;) She could always access email via hotspot fire up off phone service on her laptop or ipad. If either of those die there is other equipment she can use. If the server did die, the only loss of access would be to media library - if she really wants to listen to a song or what movie that was in the library - she can pull out the optical media copy of it, etc.

"To each their own."

Exactly - run raid if you want, its your money, its your data.. The fun is in the discussion of different opinions and options.

 

As usual, all things are situational. For my setup, all of my servers are VMs running on a RAID5 datastore. I'm willing to take the IOPS hit for the RAID5 as I'm using SAS drives entirely (I got them for ridiculously cheap on Amazon via Warehouse Deals... I have never seen them this low again so I'm sure it was a pricing error on them.)...

 

There is no router to put in the middle as I only use pfSense on my network running in a VM. There is no phone service to redirect as that is running on a VM using 3CX. There is no email to access because that too in running on a VM for Exchange 2013... Our Comcast service is also pushed out via Ethernet to HTPCs...

 

It also wouldn't be appreciated if such a drive failure were to happen while I was working from home... Not to easy to explain to my manager that I had no phone or access to the company VPN due to having to restore from backup...

 

For me, VMs going down due to a failed drive wiping out the datastore is too big an issue so I mitigate against it. Of course, there are other points of failure along the chain such as a Comcast outage wiping out my phone and email servers... But that is less likely than a HDD failing.

 

I likewise love the discussion. I just think your view against redundancy at all in the home environment is a bit too far off mark. I would agree that it is likely not needed in most home environments, but there are always edge cases. Ideally, we're smart enough to identify whether or not we're an edge case.



#55 xendrome

xendrome

    In God We Trust; All Others We Monitor

  • Tech Issues Solved: 14
  • Joined: 05-December 01
  • OS: Windows 8.1 Pro x64

Posted 08 May 2014 - 18:35

EOL Enterprise gear > any consumer storage on the market at a fraction of the price. 

 

 

I would never use any EOL enterprise gear for something in production that I needed to keep running. Sure there are used replacement parts out there or complete systems, but when you need it what are you going to do, go search EBay or Craigslist to till you find what you want, then wait for it to ship, while you are down?

 

Not good business sense IMHO.



#56 +BudMan

BudMan

    Neowinian Senior

  • Tech Issues Solved: 106
  • Joined: 04-July 02
  • Location: Schaumburg, IL
  • OS: Win7, Vista, 2k3, 2k8, XP, Linux, FreeBSD, OSX, etc. etc.

Posted 08 May 2014 - 18:46

"but there are always edge cases. Ideally, we're smart enough to identify whether or not we're an edge case."

Very true -- That is great you have mitigated your loss of your datastore with a raid setup, since your Vms being up is critical to your network. So clearly your way beyond a typical home setup.. I also run my router in VM, but other stuff on my esxi is play, or media storage - nothing critical to me working.

We could debate if 5 was the right choice for such a setup.. I would be more prone to 1 in your case. Doesn't sound like you need large amount of space so 1 gives you better protection for what your looking for from my take on it.

I would be concerned with loss of the powersupply or server hardware in your case. Do you have a secondary vm host you can migrate your VMs to in case of this sort of failure? If not doesn't seem like your mitigating the many other points of failure that could cause you grief. And just causing performance hit and more complexity in the setup. To be honest, to mitigate loss of your datastore wouldn't a just a backup work? Vs running online parity? You take a backup of your VMs and if you have drive failure you use a different disk for datastore, mount something over the network and restore your backups and your back up and running.

This is lower cost way to mitigate your setup imho. To me your more critical point of failure is the ability to run the VMs, no so much the data on the datastore on loss of hdd failure. While loss of hdd is a point of failure to be sure, loss of one if correct backups are done would mean min down time in your case. What could cause a much larger down time is loss of the host, or parts in the host. Money on to provide online parity might have been better spent on redundant or alternative host, etc.

#57 TPreston

TPreston

    Neowinian Senior

  • Tech Issues Solved: 2
  • Joined: 18-July 12
  • Location: Ireland
  • OS: Windows 8.1 Enterprise & Server 2012R2/08R2 Datacenter
  • Phone: Nokia Lumia 1520

Posted 08 May 2014 - 18:48

I would never use any EOL enterprise gear for something in production that I needed to keep running. Sure there are used replacement parts out there or complete systems, but when you need it what are you going to do, go search EBay or Craigslist to till you find what you want, then wait for it to ship, while you are down?
 
Not good business sense IMHO.

Perhaps you misunderstood me ? Im not suggesting this be used in enterprise but for home use instead of the pathetic overpriced offerings of synology and qnap

#58 +patseguin

patseguin

    Neowin Addict

  • Tech Issues Solved: 1
  • Joined: 21-May 02
  • Location: Buffalo, NY
  • OS: Windows 10
  • Phone: iPhone 6

Posted 08 May 2014 - 19:18

The Red drives are designed for NAS use. Shouldn't you be looking at the black version maybe?



#59 Tony.

Tony.

    Neowinian Senior

  • Joined: 10-February 05
  • Location: Liverpool, UK
  • OS: Windows 7

Posted 08 May 2014 - 19:32

I wouldn't go with RAID-5 or 6 simply for the write hole issue, unless you have serious RAID hardware to with it.  ZFS is a good choice for storage/NAS usage. :)



#60 +LogicalApex

LogicalApex

    Software Engineer

  • Tech Issues Solved: 8
  • Joined: 14-August 02
  • Location: Philadelphia, PA
  • OS: Windows 7 Ultimate x64
  • Phone: Nexus 5

Posted 08 May 2014 - 19:36

"but there are always edge cases. Ideally, we're smart enough to identify whether or not we're an edge case."

Very true -- That is great you have mitigated your loss of your datastore with a raid setup, since your Vms being up is critical to your network. So clearly your way beyond a typical home setup.. I also run my router in VM, but other stuff on my esxi is play, or media storage - nothing critical to me working.

We could debate if 5 was the right choice for such a setup.. I would be more prone to 1 in your case. Doesn't sound like you need large amount of space so 1 gives you better protection for what your looking for from my take on it.

I would be concerned with loss of the powersupply or server hardware in your case. Do you have a secondary vm host you can migrate your VMs to in case of this sort of failure? If not doesn't seem like your mitigating the many other points of failure that could cause you grief. And just causing performance hit and more complexity in the setup. To be honest, to mitigate loss of your datastore wouldn't a just a backup work? Vs running online parity? You take a backup of your VMs and if you have drive failure you use a different disk for datastore, mount something over the network and restore your backups and your back up and running.

This is lower cost way to mitigate your setup imho. To me your more critical point of failure is the ability to run the VMs, no so much the data on the datastore on loss of hdd failure. While loss of hdd is a point of failure to be sure, loss of one if correct backups are done would mean min down time in your case. What could cause a much larger down time is loss of the host, or parts in the host. Money on to provide online parity might have been better spent on redundant or alternative host, etc.

 

Yeah there are many ways to improve upon my current setup. I do have a backup host that can run the VMs in the case of a hardware failure in the host. It would be minimal downtime to do so (simply putting the disks in the other host will allow me to boot the VMs faster than I can restore from backup)... In my experience, I have HDD failures more than other hardware failures in my hosts so I mitigated the most likely scenario, but I have to trade off on where the cost outweigh the benefits.

 

I do nightly backups of all of my VMs using Veeam locally and another set monthly that goes off site. Restoring from a backup will lose at most a days worth of emails and phone information, but can be less of an issue depending on the day.


I wouldn't go with RAID-5 or 6 simply for the write hole issue, unless you have serious RAID hardware to with it.  ZFS is a good choice for storage/NAS usage. :)

I have a BBU on my RAID controllers to help mitigate against this...