esxi ESXi - HP MicroServer N40L Performance

[CPressland]

Hey Guys,

I just finished converting one of my HP MicroServers to run ESXi on a 8GB Kingston SSD with 3 7200RPM Hard Drives.

Unfortunately, regardless of which hard drive I place a VM on, the performance is terrible. It took 4+ Hours to install Windows XP and 2 Server 2003 R2 VMs simultaneously. I've used Thick Provisioning - Lazy Zeros.

Any ideas? Ideally from other people running on a MicroServer.

Thanks

[remixedcat]

8GB??? that's wayyy small.

[CPressland]

8GB is more than enough for a Boot Drive. It's only a few hundred meg of Files.

All the VMs are sitting on the 7200RPM Drives.

[Depicus]

8GB is more than enough for a Boot Drive. It's only a few hundred meg of Files.

All the VMs are sitting on the 7200RPM Drives.

How much memory ?

I have 7 servers running ESXi and some are ?250 1U's which run a few linux boxes but the rest run Windows 2003/2008 just fine, even on low memory.

[CPressland]

8GB DDR3.

Like I said, this looks like an IO issue. The VMs actually run nice and fast, but when installing anything it just seams to take forever compared to a VMWare Fusion VM sitting on my iMac.

[remixedcat]

hardware or software RAID?? what are the brand and model of the drives??

[yeoo_andy_ni]

Depends on the RAM and disk config of the server.

As you're installing the VM on a disk that's not the primary/OS disk, I can't see it being the issue.

RAID array for the 3 disks? Hardware RAID if so?

Any other VM's running at the same time? Possible IO issue?

What about trying one hard disk in the system (with the 8GB ESXi OS drive) and see can you create a VM successfully then. Add an additional disk if it works and check performance. Do the same for the 3rd disk.

Check for physical hardware issues. Try different SATA cables and/or, if possible, different hard disks.

Can't think of anything else at the moment. I take it the ESXi interface performs fine and is as snappy as it needs to be?

[+BudMan MVC]

So you were doing 3 installs at the same time? Reading the media iso's from the same datastore (I assume) creating the disks for these installs on what datastore?

And you what don't see how there might be a bit of I/O problem with that?

edit: I have got 6 VMs on my little N40L currently -- no installs are not as zippy as you might hope for. But after the install the VMs run great! I keep meaning to get ftp working because uploading the iso's to the datastore is like watching paint dry as well ;)

In my setup I have 1 datastore on the 250GB disk. So the Iso for the install is on this disk, and then the disk I am writing too for the OS is on that same disk, etc. So this is not going to be optimal performance! And as already mentioned what else is running off that datastore drive at the same time your trying to do the installs, etc. etc.. No its not going to be a rocketship ;)

Who told you to try 3 installs at the same time?? If you going to do more than 1 of a specific OS, do it once and create a template to install your other copies from.

[ahodgey]

You tried to install the 3 VMs simultaneously? Would imagine you are maxing out the processor. Remember you only have a 1.5 GHz dual core processor in an N40L. I am doing something similar at the moment on my N36L can install a single VM in around 20 minutes. Would try doing one at a time

[CPressland]

You tried to install the 3 VMs simultaneously? Would imagine you are maxing out the processor. Remember you only have a 1.5 GHz dual core processor in an N40L. I am doing something similar at the moment on my N36L can install a single VM in around 20 minutes. Would try doing one at a time

CPU Use shows being around 40%.

So you were doing 3 installs at the same time? Reading the media iso's from the same datastore (I assume) creating the disks for these installs on what datastore?

And you what don't see how there might be a bit of I/O problem with that?

edit: I have got 6 VMs on my little N40L currently -- no installs are not as zippy as you might hope for. But after the install the VMs run great! I keep meaning to get ftp working because uploading the iso's to the datastore is like watching paint dry as well ;)

In my setup I have 1 datastore on the 250GB disk. So the Iso for the install is on this disk, and then the disk I am writing too for the OS is on that same disk, etc. So this is not going to be optimal performance! And as already mentioned what else is running off that datastore drive at the same time your trying to do the installs, etc. etc.. No its not going to be a rocketship ;)

Who told you to try 3 installs at the same time?? If you going to do more than 1 of a specific OS, do it once and create a template to install your other copies from.

ISO(s) was on 8GB SSD

Server 2003 R2 1 (160GB 7200 Maxtor)

Server 2003 R2 2 (250GB 7200 Seagate)

Windows XP SP3 (500GB 7200 Samsung F1)

The SSD should have been able to keep up with installing on three separate drives.

That said, pulling a file over Samba from a completely independent local server only yielded speeds of 9.7MB/s (100Mb) yet should give me the full 1Gbit available to the Server.

Depends on the RAM and disk config of the server.

As you're installing the VM on a disk that's not the primary/OS disk, I can't see it being the issue.

RAID array for the 3 disks? Hardware RAID if so?

Any other VM's running at the same time? Possible IO issue?

What about trying one hard disk in the system (with the 8GB ESXi OS drive) and see can you create a VM successfully then. Add an additional disk if it works and check performance. Do the same for the 3rd disk.

Check for physical hardware issues. Try different SATA cables and/or, if possible, different hard disks.

Can't think of anything else at the moment. I take it the ESXi interface performs fine and is as snappy as it needs to be?

All disks are 100% separate (bar sharing the same Mini-SAS). No RAID in use at all here. All VMs running at the same time.

hardware or software RAID?? what are the brand and model of the drives??

No RAID. Listed Above.

[+BudMan MVC]

So you have your 3 disks setup as 3 datastores? So you have 4 different datastores - your SSD, and then 1 on each of your disks?

As to your SSD keeping up???

Where were you seeing 9.7MBps to and from what? Doing uploads to datastore are SLOW as hell yes, I have see this!!

So I sure an the hell was not going to create a 2TB vmfs for my file server, so I created a RDM to the 2TB disk and then assigned this to my 2k8r2 essentials vm. Then did the same for the extra 2 750GB drives I had laying around. So my file server has 3TB in a drive pool (drivepool from stablebit) - raw disk access.. Even have smart info from the disks inside the vm this way! And fs is native ntfs, so if need be can pull this disk and plug it into anything and get my files off, etc.

So from my workstation to this server here are the speeds I am getting -- Not too shabby to a drive pool running on a VM, on a cheap dual 1.5GHz box ;)

I am nothing but impressed with the performance I am getting out of this little box -- for the price you can not beat it!!!

[CPressland]

Where were you seeing 9.7MBps to and from what? Doing uploads to datastore are SLOW as hell yes, I have see this!!

Transfer was from Windows 2K8 R2 MicroServer N36L (Storage Server) to Server 2003 R2 (ESXi Virtual Machine) over Samba. I know it's not network drivers as pfSense is able to route at full gigabit speeds and thats sitting in a VM also now, this leads me to believe it's an I/O bottleneck.

[remixedcat]

Did u use any converters??

What networking gear u got??

[+BudMan MVC]

"over Samba"

Since when does windows OSes run SAMBA? ;)

You mean using SMB? Samba is a linux suite that brings the SMB/CIFS protocols to linux ;) Not the actual name of the protocol used in windows file copy.

As to the speed -- and is your N36L box connected at gig? Keep in mind as well, just because you have a gig interface on the hardware of your N40L, does not mean the vms inside of it are using gig.. Its possible that pfsense is gig, and your 2k3 virtual nic is set or only seeing 100.

[+John Teacake MVC]

Where did you get the RAM for this server, I have just got one!

[+BudMan MVC]

Here is the ram I ordered -- cheaper, and have had not issues with it.

20-148-347 MEM 4Gx2 | CRUC CT2KIT51264BA1339 R $42.99

http://www.newegg.co...N82E16820148347

Looks like the price went up a couple of bucks.

[CPressland]

Did u use any converters?? What networking gear u got??

No Converters, CAT5e

"over Samba" Since when does windows OSes run SAMBA? ;) You mean using SMB? Samba is a linux suite that brings the SMB/CIFS protocols to linux ;) Not the actual name of the protocol used in windows file copy. As to the speed -- and is your N36L box connected at gig? Keep in mind as well, just because you have a gig interface on the hardware of your N40L, does not mean the vms inside of it are using gig.. Its possible that pfsense is gig, and your 2k3 virtual nic is set or only seeing 100.

*slaps own head*

Dude you know what I meant by SMB/Samba/CIFS etc. All VMs are Gigabit.

Anyway, just to update thread, I resolved all issues, it was simply I/O being too high from multiple VMs being installed. It looks like ESXi cannot keep up with XEN / VMWare Workstation.

[remixedcat]

Glad to hear that you resolved the issue. I was expecting some issues to have resulted from a bad P2V conversion, etc..

ESXi should have been able to handle multi-vms with no problems...

[CPressland]

Glad to hear that you resolved the issue. I was expecting some issues to have resulted from a bad P2V conversion, etc..

ESXi should have been able to handle multi-vms with no problems...

Additionally these are budget Hard Drives - I will be replacing them with decent 15,000 RPM Drives in the next few months.

[remixedcat]

Good. ;-) Please let us know how that goes! I hope vmware esxi will work for yah really good.

[+BudMan MVC]

To be honest I think 15k drives are kind of waste of money for this sort of setup. But hey whatever floats your boat.

And I don't think it would help with any sort of multiple installs all at the same time I/0 issue either.

Did you answer my previous question about what data stores you were pulling the iso(s) from and what datastores you were installing too.. Were they all the same data store, or common storage for the ISO's?

Also if your looking to serve up data off these drives -- I would prob RDM them to your OS your going to serve the files from vs creating a VMFS on the disk and then mapping that vmfs. Your going to get better performance with RDM vs vmfs I pretty sure.

I am serving up movies off one of my VMs - which has 3 drives in a drive pool (drivepool from stablebits) And getting 80MB+ per second both write and read to this share from my workstation. I personally don't think a 15k drive is going add much to this sort of I/0 over a 7200rpm drive.

[CPressland]

And I don't think it would help with any sort of multiple installs all at the same time I/0 issue either.

Yeah, I think I/O is going to be fairly limited by the CPU.

Did you answer my previous question about what data stores you were pulling the iso(s) from and what datastores you were installing too.. Were they all the same data store, or common storage for the ISO's?

The ISO was stored on the 8GB DataStore, which is an SSD.

The OS was stored on the 500GB DataStore, which is a Samsung F1 7200 RPM

Also if your looking to serve up data off these drives -- I would prob RDM them to your OS your going to serve the files from vs creating a VMFS on the disk and then mapping that vmfs. Your going to get better performance with RDM vs vmfs I pretty sure.

Once I get some better hard drives, I'll be sure to go the RDM Method. Especially for the high-bandwidth VMs.

I am serving up movies off one of my VMs - which has 3 drives in a drive pool (drivepool from stablebits) And getting 80MB+ per second both write and read to this share from my workstation. I personally don't think a 15k drive is going add much to this sort of I/0 over a 7200rpm drive.

The limitation for serving up movies is I'm currently using PS3 Media Server for this, which yields fairly high CPU use, so I run this on the Storage Server as not to effect performance of the VMs

[+BudMan MVC]

I just have to serve up the files via smb.. I don't have to transcode them on the fly or anything.. Just a simple file server, my popcorn hour plays the files in whatever format they are in.

So you were writing 3 different installs to the same datastore at the same time - and reading all from the same iso?? WTF yeah that going to be slow as hell! Why not just install once and then copy the install if you were installing the same OS?

[ToneKnee]

I would think the NL40 is quite limited in regards to performance, it would only take a number of VM's for it to choke.

[+BudMan MVC]

"it would only take a number of VM's for it to choke."

No **** ;) But you can say that for any hardware - the number of VMs you can run at the same time is just going to vary. And what those VM's are doing.

What I can dell you is it runs my router, and is running my file server, and my test box w7 and my linux distro. The file server, the router and the linux box are on all the time..

It is more than capable of handling the number of VMs I would ever need to be running at the same time, and allow for many many more to be available when I need them for play or testing. Allowed me to combine my file server and my router boxes into a tiny little thing - that looking currently at the killawatt meter its connected to is drawing 55.5 watts ;) That is with 4 disks in it. Its been on for 366 hours and has drawn a total of 20.3KWH, so I pay like 12.5 cents per kwh so your looking at $2.50 to operate for 15 days. So about $5 a month to have it on running all the machines I need to run or play with.

And for the price you sure and the hell can not beat it for a home esxi solution or even a small business setup. No I wouldn't expect to run my enterprise VMs on it ;)