Using the PCI-E lanes on a X58 system for maximum I/O performance of HDDs/SSDs

[Regert]

Hi Neowinians!

 

I'd like to ask for some help finding the best solution or at least the best PCI-E SATA III expansion card and also the most reasonable USB 3.1 expansion card and enclosure for external storage.

 

The tech background:

I have currently a AMD X6 1100T system with 2 x 1 TB Samsung 950 SSDs and a bunch of HDDs.

I have been give by a friend (he switched to a iMac) a by todays standards perhaps somewhat old X58 system that I also helped to build a prototype for.

It consists:

Intel X970 CPU: http://ark.intel.com/products/47933/Intel-Core-i7-970-Processor-12M-Cache-3_20-GHz-4_80-GTs-Intel-QPI

 

Gigabyte X58a-ud3r: http://www.gigabyte.se/products/product-page.aspx?pid=3449#ov

This motherboard might have it's Intel ICH10R chip ports burnt or the Marvell SATA3 ports burned.

As I understand the motherboard has 2 x 16x PCI-E 2.0 ports; 2 x 8x ports and 2 x 1x ports.

12 GB ram (plan to upgrade to 16 + 4 or perhaps 24 GB later on depending on compatility)

 

So therefore since I use two Samsung SSDs (850 + 840) and also want to up the I/O from HDDs on the system as a whole, I'd like to relocate as much as possible of the SSD/HDD "pipeline" chain to the PCI-E links using a PCI-E SATA 3 expansion card or a good controller card, and also installing a USB 3.1 (only need 2 ports I think) card to later use HDDs in a external enclosure.

What do I need all that speed and I/O for?

Two reasons:

I have ALOT of photos I need to reorganize, analyse and so on (moving from a old Picasa + no order database to probably Lightroom or other program) and for all the scanning, thumbnail/preview rendering etc want to make sure the programs don't have to wait for bottlenecks. Today the hardware is usually adequate, but software still can't utilize all the raw HW power and cores. But since I have 6 cores, why not use it.

Why not let the i7 stretch it's legs fully. Since it also has 32 PCI-E lanes it should be fully able to use a couple for those for a SDD/HDD subsystem, even with say 1-2 GPUs. I usually use 1 GPU anyway (currently have a GTX 650 but might upgrade later).

Besides that most of the HDDs are between 2 and 5 TB, so if I use a card for those it will need to support such big drives.

 

Which cards would you suggest buying/using?

 

Spoiler

 

RAID-controller cards probably won't work since the need empty HDDs/SSDs for initialization and I need to use already populated HDDs.

 

My own googling as yielded that I need to use a card that is compatible with the BIOS, and one that depending on number of SSDs/HDDs attached needs between 1x-4x PCI-E 2.0 to keep som overhead and not saturate the link between the storage and the Northbridge. However the chip in the controller/expansion card compatibility is also essential.

 

 

So far I have been looking at using:

At least a expansion SATA III PCI-E 2.0 card with 2 ports or the SSDs.

If the ICH10 ports work then perhaps place the HDDs there, otherwise a expansion SATA III PCIE- 4-8 port card for that?

A USB 3.1 Gen 2 or Type-C card with 2 ports. The idea is to attach it to a external 3.1 enclosure with 2-4 x HDDs (backup). It would have been really nice to switch to Thunderbolt 2 or even 3, but it's far to expensive.

 

 

---------

 

A bit of a separate question: If anyone of you is using a Linux solution that uses more then just 2x-4x core in software for Photography including: development, storage, facial recognition and all that for several 10 k+ or 100k+ photos own work, I'd be very happy to talk to you.

[fusi0n]

I've had a glasses of whiskey and didn't read everything. It seems that you are worried about bandwidth using a x58 board. On my old x58 setup, I had three GTX 680s in SLI that ran amazing on an Intel i7 950. I eventually went to a Xeon. If you are looking to OC like crazy or just have some cheap fun, you can pick up a Xeon X5660 and overlock that thing to ######. I was going to say death, but I had that thing at 5.2Ghz using a Corsair H100i for cooling.. The x58 is still strong.. The only reason I upgraded is because I have a baby girl on the way and wanted to spend as much money on a new PC as I could because I know that I wouldn't be able too in the next couple of years.. Kids bro.. kids.. I'm happy overall, just have to be smart when buying PC part.. But you have an Intel i7 970X.. Check out one of the top end Xeons like the X5690 and overclock it's balls off.. They run super cool and it's fun.. Don't worry about breaking it.. Your motherboard will protect it.. and if you do break it.. You might be out $100, you have a good CPU to put back in, AND you had a good time.

 

-Fus10n 

[eddman]

The main thing to note with X58 is that those PCI-E x1 slots are version 1.1, which means that their max theoretical speed is 250 MB/s. A modern storage expansion card, mainly USB 3.1 and/or USB type-C will most probably get bottlenecked by those slots.

 

PCI-E slots should be universally compatible, so what I'd suggest is to put your cards (whatever you end up buying) in the PCI-E x16 slots, which are version 2.0. That should give you a 500 MB/s max speed.

 

Still, there are some reports that x16 slots on certain mainboards only accept graphics cards. I don't know how reliable those claims are. An other member here might clarify it.

Edited July 23, 2016 by eddman

[Regert]

10 hours ago, fusi0n said:

I've had a glasses of whiskey and didn't read everything. It seems that you are worried about bandwidth using a x58 board. On my old x58 setup, I had three GTX 680s in SLI that ran amazing on an Intel i7 950. I eventually went to a Xeon. If you are looking to OC like crazy or just have some cheap fun, you can pick up a Xeon X5660 and overlock that thing to ######. I was going to say death, but I had that thing at 5.2Ghz using a Corsair H100i for cooling.. The x58 is still strong.. The only reason I upgraded is because I have a baby girl on the way and wanted to spend as much money on a new PC as I could because I know that I wouldn't be able too in the next couple of years.. Kids bro.. kids.. I'm happy overall, just have to be smart when buying PC part.. But you have an Intel i7 970X.. Check out one of the top end Xeons like the X5690 and overclock it's balls off.. They run super cool and it's fun.. Don't worry about breaking it.. Your motherboard will protect it.. and if you do break it.. You might be out $100, you have a good CPU to put back in, AND you had a good time.

 

-Fus10n 

Thanks I'll remember that.

 

10 hours ago, eddman said:

The main thing to note with X58 is that those PCI-E x1 slots are version 1.1, which means that their max theoretical speed is 250 MB/s. A modern storage expansion card, mainly USB 3.1 and/or USB type-C will most probably get bottlenecked by those slots.

 

PCI-E slots should be universally compatible, so what I'd suggest is to put your cards (whatever you end up buying) in the PCI-E x16 slots, which are version 2.0. That should give you a 500 MB/s max speed.

 

Still, there are some reports that x16 slots on certain mainboards only accept graphics cards. I don't know how reliable those claims are. An other member here might clarify it.

Are you sure?  From Gigabytes spec: "2 x PCI Express x16 slot, running at x8 (PCIEX8_1/PCIEX8_2) (Note 3) (The PCIEX16_1, PCIEX16_2, PCIEX8_1 and PCIEX8_2 slots conform to PCI Express 2.0 standard.)"

 

[eddman]

7 hours ago, Regert said:

Thanks I'll remember that.

 

Are you sure?  From Gigabytes spec: "2 x PCI Express x16 slot, running at x8 (PCIEX8_1/PCIEX8_2) (Note 3) (The PCIEX16_1, PCIEX16_2, PCIEX8_1 and PCIEX8_2 slots conform to PCI Express 2.0 standard.)"

 

Do you see any mention of PCIEX1? It just mentions PCIEX8 and PCIEX16.

 

http://www.anandtech.com/show/2973/6gbps-sata-performance-amd-890gx-vs-intel-x58-p55/2

 

"Bandwidth in a single direction for a single PCIe 1.0 lane (x1) is 250MB/s"

"Intel’s X58 chipset for example has 36 PCIe 2.0 lanes off of the X58 IOH, plus an additional 6 PCIe 1.0 lanes off the ICH." Those PCI-E x1 slots are connected to the ICH.

 

http://www.evga.com/support/manuals/files/132-BL-E758.pdf

 

I know this an evga manual but it applies to all X58 boards when it comes to slot versions. Go to page 32.

 

"PCI Express x1 Slots

 

There is one PCI Express x1 slot that is designed to accommodate less

bandwidth-intensive cards, such as a modem or LAN card. The x1 slots provide

250 MB/sec bandwidth."

Edited July 24, 2016 by eddman

[Regert]

2 hours ago, eddman said:

Do you see any mention of PCIEX1? It just mentions PCIEX8 and PCIEX16.

 

http://www.anandtech.com/show/2973/6gbps-sata-performance-amd-890gx-vs-intel-x58-p55/2

 

"Bandwidth in a single direction for a single PCIe 1.0 lane (x1) is 250MB/s"

"Intel’s X58 chipset for example has 36 PCIe 2.0 lanes off of the X58 IOH, plus an additional 6 PCIe 1.0 lanes off the ICH." Those PCI-E x1 slots are connected to the ICH.

 

http://www.evga.com/support/manuals/files/132-BL-E758.pdf

 

I know this an evga manual but it applies to all X58 boards when it comes to slot versions. Go to page 32.

 

"PCI Express x1 Slots

 

There is one PCI Express x1 slot that is designed to accommodate less

bandwidth-intensive cards, such as a modem or LAN card. The x1 slots provide

250 MB/sec bandwidth."

I see. Well then you are correct.

But since I am planning on using 1 x GPU, and maximum 2 expansion cards the PCIe 2.0 slots should be sufficient, right?

I can put my soundcard in the PCIe 1x slot.

[Regert]

After some googling, I guess theese are my options for a SATA III card:

in no particular order:

1. highpoint rocketraid 640

2. MegaRAID SAS 9240-8i

3. Delock PCI Express Card 89372 http://www.delock.de/produkte/F_418_SATA-6-Gb-s_89372/merkmale.html

4. Delock PCI Express Card Hybrid 4x 89395 http://www.delock.de/produkte/F_418_SATA-6-Gb-s_89395/merkmale.html
5. StarTech 4 Port PCI Express 2.0 SATA III 6Gbps RAID Controller Card  https://www.startech.com/ca/Cards-Adapters/HDD-Controllers/SATA-Cards/4-Port-PCI-Express-SATA-6Gbps-RAID-Controller-Card~PEXSAT34RH

6. Perhaps a some Syba card with a ASMedia chip.

 

As for USB 3.1

Either a Gigabyte USB 3.1 /Type C  card http://www.gigabyte.se/products/product-page.aspx?pid=5430#ov

or a ASUS USB 3.1  card http://www.asus.com/Motherboard-Accessories/USB_31_TYPEA_CARD/

Edited July 24, 2016 by Regert