Storage recommendations.

[StrikedOut]

Hi All.

 

I am putting a proposal together to consolidate all of our project storage into a single solution to retire our aging Dell servers. As we are 3 companies that have merged, we have 3 locations, each with its own file server. With 90% of the people who actively work on the projects based in one location, I want the device to be based in there (bandwidth is sufficient for the remaining people).

 

Currently the site has an HP Proliant DL385 Gen8 server that has Hyper-V installed. AD is in a VM and I will be building a new VM for the file store (Server 2016). Unfortunately, I haven't done a project like this (integrating NAS/SAN) but am looking forward to getting started, having a backup HP DL385 helps for testing too!

 

So the following is what I am looking for, suggestions are appreciated.

• Must be Active Directory Integrated.
• Support for RAID 6 although suggestions on alternative is welcomed.
• Storage for minimum of 24GB
• Rack mounted
• Budget is around £5k.

[StrikedOut]

This is looking to be the type of device I am looking for and could make things easier by having its own OS installed (Server 2012 R2 storage server) although I will probably format and install 2016.

 

https://www.hpe.com/uk/en/product-catalog/storage/file-storage/pip.hpe-storeeasy-1650-32tb-sas-storage.7660573.html

 

It is over my original budget but if it is worth the extra then I will push for this instead.

[Mindovermaster Moderator]

@BudMan to the rescue!

[StrikedOut]

2 minutes ago, Mindovermaster said:

@BudMan to the rescue!

Ha! I did think about tagging Budman and sc302 in but I thought that was a little pretentious! - Thanks though.

[Draconian Guppy]

9 minutes ago, StrikedOut said:

Ha! I did think about tagging Budman and sc302 in but I thought that was a little pretentious! - Thanks though.

How is tagging the guys who know pretentious

[StrikedOut]

9 minutes ago, Draconian Guppy said:

How is tagging the guys who know pretentious

Ever have a word jump to mind only to realise after re-reading the sentence, you used to wrong?!.... Assuming would have been a better choice. Feels like free consulting/support. They both frequent this sub forum so no doubt they would see it. 

[+BudMan MVC]

Bit confused... So you want to run your file storage as a VM.. But then your looking at a HP nas?  That to be honest is way over kill most likely.

 

How much storage do you need, how fast do you feel that will grow?

 

There are many budget friendly nas boxes that will integrate into AD.. How many users?  What sort of network at these locations.. What sort of bandwidth and connectivity between the locations.  What is your backup solution.

[StrikedOut]

1 hour ago, BudMan said:

Bit confused... So you want to run your file storage as a VM.. But then your looking at a HP nas?  That to be honest is way over kill most likely.

 

How much storage do you need, how fast do you feel that will grow?

 

There are many budget friendly nas boxes that will integrate into AD.. How many users?  What sort of network at these locations.. What sort of bandwidth and connectivity between the locations.  What is your backup solution.

My initial plan was to connect a storage device to our network at our busiest site., create a new VM on my existing Hyper-V server and connect a vhdx that would be on the new storage device for the data. I am open to options to get the best out of this project though.

 

The project is part of me pushing for my company to qualify for the Cyber Essentials Plus certification - https://www.cyberessentials.ncsc.gov.uk/. With various changes in data protection laws (GDPR in particular), many of our clients and prospective new clients are requesting this type of certification. So this is to answer a few issues issues I have raised before that will help us working towards certification.

 

Regarding the storage, current data is a little under 3TB in current data with almost the same in archived data. Due to some of our contracts, we need to keep records for anywhere up to and beyond 25 years so little chance for deleting archived data. I am looking to change the way we manage project data, I intend to get very strict on the new server as currently it is awful. I also intend to utilise shared mailboxes for the projects which will mean completed projects will have a much larger data footprint as it will now include all the email relative to that project. With that I expect the storage to increase by up to 400GB a year for project data alone but in the transition from old to new servers, I also expect to be able to delete a significant amount of data too.

 

Regarding the network, all sites are on an MPLS network with the hosting site currently on a 20Mb leased line with the option to upgrade to 100Mb if required. However, there are not many not based at that site that will need access 6 in one site and 4 at the other so I believe the existing network will suffice. Our backup solution is an encrypted off site solution to our ISP who in turn mirror it to their second data centre.

 

I am looking for an enterprise solution because our company is steadily growing (160 employees currently) plus the owners have a habit of buying a new business for me to integrate. I also want a significant amount of available spare storage as i want to move other services into this device.

[DevTech]

On 5/30/2018 at 1:46 PM, StrikedOut said:

My initial plan was to connect a storage device to our network at our busiest site., create a new VM on my existing Hyper-V server and connect a vhdx that would be on the new storage device for the data. I am open to options to get the best out of this project though.

 

The project is part of me pushing for my company to qualify for the Cyber Essentials Plus certification - https://www.cyberessentials.ncsc.gov.uk/. With various changes in data protection laws (GDPR in particular), many of our clients and prospective new clients are requesting this type of certification. So this is to answer a few issues issues I have raised before that will help us working towards certification.

 

Regarding the storage, current data is a little under 3TB in current data with almost the same in archived data. Due to some of our contracts, we need to keep records for anywhere up to and beyond 25 years so little chance for deleting archived data. I am looking to change the way we manage project data, I intend to get very strict on the new server as currently it is awful. I also intend to utilise shared mailboxes for the projects which will mean completed projects will have a much larger data footprint as it will now include all the email relative to that project. With that I expect the storage to increase by up to 400GB a year for project data alone but in the transition from old to new servers, I also expect to be able to delete a significant amount of data too.

 

Regarding the network, all sites are on an MPLS network with the hosting site currently on a 20Mb leased line with the option to upgrade to 100Mb if required. However, there are not many not based at that site that will need access 6 in one site and 4 at the other so I believe the existing network will suffice. Our backup solution is an encrypted off site solution to our ISP who in turn mirror it to their second data centre.

 

I am looking for an enterprise solution because our company is steadily growing (160 employees currently) plus the owners have a habit of buying a new business for me to integrate. I also want a significant amount of available spare storage as i want to move other services into this device.

You are providing a lot of description that does not really translate well to hardware specs due to all sorts of missing detail.

 

So far:

 

1. 100% non-Cloud On-Premise solution using Windows Server 2016 (have you explored Azure?)

2. Primary usage is File Server (what types of files?)

3. Current tiny storage of 3TB to grow to 400 gb times 25 years = rough total of 20 TB which is almost nothing now and certainly nothing then.

4. You want extra storage space for 160 people to do other non-specified stuff

5. "i want to move other services into this device" - which would be? What is the list of current "non-dumb-file-server" uses? What apps will you run in the future? Are there any CPU based compute requirements across 160 users?

6. "I am looking for an enterprise solution" means you should provide an enterprise quality requirements list. I can't imagine anyone building a metal picture of what you need with this list so far. A Synology NAS with about 10 of some NAS 6 TB drives should cover your specs so far.

7. Since you mention "Enterprise" I will add that any modern server architecture in 2018 should be starting with a ground-up Kubernetes orchestration of  Docker Containers which provides self-healing continuous operation at a software level based on Containers being immutable which leads to a different storage persistence architecture all of which leads to a different hardware provisioning perspective.

 

[StrikedOut]

14 hours ago, DevTech said:

**snip**

Valid points, let me see if I can clear things up a little.

1. Correct. The storage device will be based at our main office. Azure is something we may consider but truthfully, I have yet to investigate it thoroughly for our needs.
2. Nothing unusual. Office docs, CAD, small media files of videos of completed builds but no more than a couple minutes, it will also have the archived data so will include PST files. There are no databases or similar to be used on this system.
3. Agreed. Although this was for project data only, the sum of other data would not impact this much to be concerned about.
4. Examples would include investigation in progress folders where data would be duplicated away from existing project data, plasma cutting templates, neither of these or similar will be long term or sizable so little impact. I may also retire the current archive server and incorporate it into this device plus I would also like to get the images of our PC/laptops (~6 different types) uploaded to this device. The last 2 would be noticeable but not game changing with regards to storage size.
5. At this time, I cant think of any service that would require anything other than storage. As mentioned above, images to deploy is high on my list, potentially WSUS but I may use the retiring servers for this service, onsite backup for our core VMs (not to replace the offsite backup).
6. A better term should have been SME Solution that is scalable. I have been looking at the Synology systems in their xs range and they do look more than capable for my needs, the RS3617xs+ in particular. It's also around £1,600 less than the HP I linked above.
7. Until your post, I had not seen this but does look interesting so will take a look into this. The RS3617xs+ is Btrfs capable which on initial impressions does share some features so may in itself be sufficient.

[StrikedOut]

So done a very quick look into Kubernetes and docker, this system will only be a file server for now but should we start to utilise applications in house, then this will certainly be investigated further, Kubernetes implementation looks extremely robust so for high availability, its does look to be a must have.

 

On top of the resilience, the independence docker provides for the application looks to make support and cleanup much simpler too.

[sc302 Veteran]

Something like this may be interesting for you to look at.  25 2.5" bays with 4 slots for 4 individual compute hosts.  

 

http://www.dell.com/en-us/work/shop/povw/poweredge-vrtx

 

If that doesn't do anything for you, there is a bevy of supermicro vendors that would be happy to sell a cheap/fairly cheap solution that is quite capable of a single os and 100's of TB of space.  Digiliant has servers capapble of 972TB raw storage.  81 12 TB drives in an enclosure/server (granted it is 55-60k USD for that) and a bit out of your price range I am sure, but 972TB is probably out of your need as well.

 

FWIW, a 20TB san is probably a 20k USD project.

 

 

[DevTech]

6 hours ago, StrikedOut said:

So done a very quick look into Kubernetes and docker, this system will only be a file server for now but should we start to utilise applications in house, then this will certainly be investigated further, Kubernetes implementation looks extremely robust so for high availability, its does look to be a must have.

 

On top of the resilience, the independence docker provides for the application looks to make support and cleanup much simpler too.

Once everything you do is architect-ed into containers, then deployment to local on-premise or in the cloud is identical and flexible.

 

The VM then becomes just a host for Containers and you need in theory only one or zero VMs or maybe 10 depending on you you might emulate a cluster locally.

 

Every major industry player has finally signed on to this system under the umbrella of the CNCF. The only confusing part is the name, in that "Cloud Native" just means Containers orchestrated by Kubernetes and is perfectly happy to be on-premise with no cloud but also adapts without change to any cloud spin-up you want to do.

 

The easiest way to play with it is download the latest "Docker for Windows" (get the Edge Version which also contains Kubernetes). You need a Hyper-V enabled Windows 10. https://store.docker.com/editions/community/docker-ce-desktop-windows

 

https://www.cncf.io

 

https://landscape.cncf.io

 

 

[DevTech]

It might be helpful for you to review the variety of machine types on Azure as an example of various "suitability to purpose" configs:

 

https://azure.microsoft.com/en-us/pricing/details/virtual-machines/windows/

 

Depending on the longevity you are planning for, there is a growing trend for larger RAM, more cores instead of faster cores, and some sort of SSD cache. Multiple GPU cores are increasingly popular for analytics on large data sets which probably does not apply for you.