Jump to content



Photo

Case for 15 Drive RAID Setup

case raid hdd

  • Please log in to reply
28 replies to this topic

#16 Vice

Vice

    Bye!

  • Joined: 03-September 04

Posted 06 February 2012 - 11:34

Now this is a genuinely helpful post, do you have any photos of the internals after wiring? A list of used PCI/PCI-E SATA Cards?

Thanks!!


Sure here is a shot with the 15x Sata cables connected. The yellow cable visible is the boot disk. http://i.imgur.com/3G6vm.jpg
And here is the system all wired up and powered on. http://i.imgur.com/RuD7M.jpg

The only reason I'm using a 1000 Watt PSU in the system is that I had it laying around. I would not use something that beefy for this application any other time (The server barely pulls 150 Watts according to my UPS).

RAID card wise in those pictures I am using two Highpoint 2320's they are identical cards but one is a later revision which is why the heatsink is different. They worked fantastic for me for a long time but I've switched out to using an LSI 9620-8i and a HP SAS Expander which provides me with a dedicated processor for parity calculations on the RAID card (which the Highpoints lack) and it gives me 36 ports to play with (8 on the controller + Expander, 4 are used for connectivity to the expander leaving 36 usable for disk drives).


#17 DDStriker

DDStriker

    Neowinian Senior

  • Joined: 01-August 05

Posted 06 February 2012 - 12:01

Disregard this..for now

#18 Vice

Vice

    Bye!

  • Joined: 03-September 04

Posted 06 February 2012 - 12:08

I thought you didn't like the x58 gigabyte boards :whistle:

Nice case...*nvm the rest here*


I don't. If you remember in that original thread I had to switch to another X58 board because the Gigabyte was a piece of **** for a gaming setup. I stuck an Asus P6T6 in my desktop and had the Gigabyte left over gathering dust. So eventually I stuck the UD5 in the server. As there were no graphics cards in the server blocking the power buttons to turn the system on it wasn't a problem.

Although Gigabytes layouts on all their early X58 boards were really poor the UD5 has been rock solid stable for me in the server. At one point recently it crossed 100 days uptime and I've not had any crashes on the system that weren't due to hardware failure of other components.

#19 OP CPressland

CPressland

    www.absolute.com

  • Tech Issues Solved: 3
  • Joined: 16-September 06
  • Location: England
  • OS: OS X Mavericks

Posted 06 February 2012 - 12:10

Well, I've tried setting up Unraid in VMWare Fusion 4.1 but unfortunately it cannot find Networking drivers and thus only has the loopback 127.0.0.1 IP. Will figure that out later.

Can you explain to me how that HP SAS Expander works? I've got no experience with SAS beyond the connector built into my current HP MicroServer. Can I connect multiple SATA into a SAS Channel or similar? *n00b storage question*.

#20 Vice

Vice

    Bye!

  • Joined: 03-September 04

Posted 06 February 2012 - 12:20

Well, I've tried setting up Unraid in VMWare Fusion 4.1 but unfortunately it cannot find Networking drivers and thus only has the loopback 127.0.0.1 IP. Will figure that out later.

Can you explain to me how that HP SAS Expander works? I've got no experience with SAS beyond the connector built into my current HP MicroServer. Can I connect multiple SATA into a SAS Channel or similar? *n00b storage question*.


Sure. Basically the HP SAS Expander features 8x8087 SAS and 1x8088 SAS ports. Each port is basically four singular SAS connectors bonded together.

You can use an 8087 to SATA cable to take one of the 8087 ports and 'fan it out' to connect four hard disks. So with that one expander you can get a potential of 32 Disks connected by using 8x8087 to SATA cables. These cables each cost about £8-£10 retail and you can fit four SATA or SAS hard disks per cable.

Now the 8088 SAS port on the card is just the same as an 8087 but its the external version. So the connector is a bit more beefy and it's on the back of the cards bracket which sticks outside the case. This is the connector I've chosen to use with my RAID card so I can use all 8x8087 ports for 32 Disks. On my RAID card it has 2x8087 connectors so I'm using a 8087 to 8088 cable to take one of those connectors on my RAID card outside of the case through a water cooling grommet in my case and in to the back of the expander.

The HP SAS Expander supports 6Gb/ps when used with SAS disks but it only supports 3Gb/ps when used with SATA disks. This isn't a hardware limitation just something HP has done with the firmware to make the card more attractive to enterprise customers. It still works fine with 6Gb/ps SATA disks but it negotiates them at the slower 3Gb/ps speed.

Most SAS RAID cards which support SAS2008 spec (you'll find this listed on their websites or manual usually) should work with the HP SAS Expander but to be honest it's not always that cut and dry. The best RAID cards compatibility wise feature an Intel processor. Cards such as the Highpoint 4330 4320 4321, LSI 9620 series (4i, 8i, 16i) all work fine with the Expander.

It's cost effective but to a certain point. The HP SAS Expander retails for around £330 in the UK and then you still need to shell out around £100 on cables and about £200 minimum on a compatible RAID card. The main benefit is a 32 port RAID card with RAID6 support is around £1200 so it does cost less than one of those and you always have the option of using the expander in a future system. In my case as I'm using an LSI 9620-8i which features two 8087 connectors I could buy another expander and double my storage potential to 64 disks.

Something else to note is that the expander is not seen by the operating system. Although it connects through PCIe it only does so to receive power and doesn't communicate its presence to the computer at all. Thus it is possible to power it separately from the motherboard you're using it in and it still works fine. It also means it "just works" no configuration needed beyond plugging in the cables and powering the computer on.

#21 OP CPressland

CPressland

    www.absolute.com

  • Tech Issues Solved: 3
  • Joined: 16-September 06
  • Location: England
  • OS: OS X Mavericks

Posted 06 February 2012 - 12:38

Ahh now I understand, so all disks connect to the SAS Ports on the HP SAS Expander Card via 8087 to SATA Cables then the SAS Expander Card connects to a SAS RAID Controller via a 8088 to 8087 Controller. The OS Sees the SAS RAID Controller, not the HP Expander Card but the RAID Controller manages all disks.

Will JBOD work with this setup?

#22 Vice

Vice

    Bye!

  • Joined: 03-September 04

Posted 06 February 2012 - 12:42

Ahh now I understand, so all disks connect to the SAS Ports on the HP SAS Expander Card via 8087 to SATA Cables then the SAS Expander Card connects to a SAS RAID Controller via a 8088 to 8087 Controller. The OS Sees the SAS RAID Controller, not the HP Expander Card but the RAID Controller manages all disks.

Will JBOD work with this setup?


That is exactly how it works yes. And yes JBOD will work. You can use a HBA controller with it such as an LSI 9211-4i or an LSI 3081E-R.

Also in my example I used the 8088 to 8087 but it is also possible to use an 8087 to 8087 cable when connecting your RAID or HBA Controller to the Expander. But obviously doing it that way would lose you an internal 8087 connector on the expander that you could use for disks.

This all means of course you can use the expander with ZFS or UnRAID etc

#23 OP CPressland

CPressland

    www.absolute.com

  • Tech Issues Solved: 3
  • Joined: 16-September 06
  • Location: England
  • OS: OS X Mavericks

Posted 06 February 2012 - 12:49

Indeed, that sounds like the much more promising a future proofed method of doing this! I shall begin building in about 2 - 3 months. Will post up details as I go.

Thanks for all your help Vice.

#24 Vice

Vice

    Bye!

  • Joined: 03-September 04

Posted 06 February 2012 - 12:50

You're welcome, hope it goes well for you :)

#25 Crisp

Crisp

    To infinity and beyond

  • Tech Issues Solved: 2
  • Joined: 06-May 10

Posted 06 February 2012 - 17:58

You're welcome, hope it goes well for you :)


Please get some pictures up of your newer set-up, with the Lian Li PC343B. I love the look of that cube case!

#26 SirEvan

SirEvan

    Neowinian Senior

  • Joined: 17-April 03
  • Location: Santa Clara, CA
  • OS: Windows 8
  • Phone: HTC One (AT&T)

Posted 06 February 2012 - 18:03

look at a NORCO rackmount case. 300$ or so for a 24-48 drive 4U case

#27 lalalawawawa

lalalawawawa

    Neowinian Senior

  • Joined: 20-June 08

Posted 06 February 2012 - 18:09

..


I thought it would look like a mess, but that's actually pretty cool. :D Nicely done.

#28 SirEvan

SirEvan

    Neowinian Senior

  • Joined: 17-April 03
  • Location: Santa Clara, CA
  • OS: Windows 8
  • Phone: HTC One (AT&T)

Posted 06 February 2012 - 19:19

Posted Image

#29 Vice

Vice

    Bye!

  • Joined: 03-September 04

Posted 07 February 2012 - 05:05

I thought it would look like a mess, but that's actually pretty cool. :D Nicely done.


It looked much better after I moved to the Lian Li PC-343B as that had tons more room. Also I used three cable ties on each bundle of SATA cables (3 bundles, each with 5 cables) http://i.imgur.com/MEvhE.jpg

That is before the 24GB RAM upgrade (Picture After) but after I put in the Gigabyte UD5 and changed cases to the Lian-Li PC-343B. I don't have any pictures currently of the newest setup where I changed RAID cards and fitted the expander but I'm intending to change some of the backplanes I'm using in the summer when I can get my hands on the 4TB Hitachi's and I'll be fully documenting it at that time with pictures. I'll probably post a build log on here like I did for my water cooling build.



Click here to login or here to register to remove this ad, it's free!