esxi ESXi - HP MicroServer N40L Performance

[+BudMan MVC]

so lets compare build -- I am using

VMware ESXi 5.0.0 build-623860 from VM, are you running the same or you using the HP builds?

BTW - question for you, do you have or do you know how to enable the other hardware listings in the health section.. I have tried installing the cim providers from HP.. But has not added anything.. Would like to see fan and hdd, etc.

[+John Teacake MVC]

From what I understand if you are copying files between VM's it doesn't even touch the wire i.e the NIC it uses shared memory? Thats why you would get impressive speeds VM to VM Am I right..

[+BudMan MVC]

yeah I agree the physical nic should not be used in a vm to vm copy, or iperf test.

I show 1.55Gbps with iperf between vms - since the nic is only 1Gb, my test kind of shows that the physical nic is not involved at all in this copy.

Now that you bring up that is only memory -- maybe his issue is some bad memory? 300mbps seems pretty bad!

[CPressland]

so lets compare build -- I am using

VMware ESXi 5.0.0 build-623860 from VM, are you running the same or you using the HP builds?

I'm using the Vanilla ESXi 5.0.0 build-469512. Didn't even realise patches would be out, I just assumed the build I downloaded would be up-to-date.

Am Updating Via the following method:

cd /scratch/downloads
wget http://hostupdate.vmware.com/software/VUM/OFFLINE/release-328-20120312-212851/update-from-esxi5.0-5.0_update01.zip
wget http://hostupdate.vmware.com/software/VUM/OFFLINE/release-331-20120408-206167/ESXi500-201204001.zip
vim-cmd hostsvc/maintenance_mode_enter
esxcli software vib install -d /scratch/downloads/update-from-esxi5.0-5.0_update01.zip
esxcli software vib install -d /scratch/downloads/ESXi500-201204001.zip
vim-cmd hostsvc/maintenance_mode_exit
reboot
rm /scratch/downloads/update-from-esxi5.0-5.0_update01.zip
rm /scratch/downloads/ESXi500-201204001.zip
[/CODE]

BTW - question for you, do you have or do you know how to enable the other hardware listings in the health section.. I have tried installing the cim providers from HP.. But has not added anything.. Would like to see fan and hdd, etc.

I get no health information display either. I too would like to be able to at very least monitor the HDD's SMART capabilities.

From what I understand if you are copying files between VM's it doesn't even touch the wire i.e the NIC it uses shared memory? Thats why you would get impressive speeds VM to VM Am I right..

Thats the general consensus, so why is mine going so slow... hmm? hopefully upgrading to the new build will help.

[CPressland]

Now that you bring up that is only memory -- maybe his issue is some bad memory? 300mbps seems pretty bad!

RAM was tested with memtest86+ for 72 hours before ESXi was installed.

[+BudMan MVC]

one thing to keep in mind on the u1, it seems to break the autostart feature. Work around is to just create add some commands in the rc file -- not sure if you autostart any of your vms -- but if you do, you will wonder why after you update it no longer works.

[CPressland]

one thing to keep in mind on the u1, it seems to break the autostart feature. Work around is to just create add some commands in the rc file -- not sure if you autostart any of your vms -- but if you do, you will wonder why after you update it no longer works.

Got those commands handy?

I auto-start my pfSense and Server 2008 Domain Controller VMs. The rest I don't really need. But without pfSense I cannot remote in, and without the Domain Controller, DNS doesn't work.

[Guest]

one thing to keep in mind on the u1, it seems to break the autostart feature. Work around is to just create add some commands in the rc file -- not sure if you autostart any of your vms -- but if you do, you will wonder why after you update it no longer works.

Ah, I noticed that. Nice that it's not just me.

[CrimsonBetrayal]

Problem I had on an HP MicroServer N36 was my network throughput on a Unix machine as an ESXi host was terrible (5MB/s up & 15MB/s down) compared to when I put just a base Unix system on (90MB/s up and 110MB/s down). I now run a Unix based server with VirtualBox VM's for my needs, it never needs a restart. So far I am loving it. Same config, 8GB ram, 120GB hdd for OS's and 4 1.5TB drives in raid Z1 for data.

[CPressland]

SK[' timestamp=1335508472' post='594828125]

Ah, I noticed that. Nice that it's not just me.

I'll let you know if I have the same issue this afternoon after the upgrade. I'm planning the upgrade at 2:30PM today, as thats when most of my household is offline.

Problem I had on an HP MicroServer N36 was my network throughput on a Unix machine as an ESXi host was terrible (5MB/s up & 15MB/s down) compared to when I put just a base Unix system on (90MB/s up and 110MB/s down). I now run a Unix based server with VirtualBox VM's for my needs, it never needs a restart. So far I am loving it. Same config, 8GB ram, 120GB hdd for OS's and 4 1.5TB drives in raid Z1 for data.

I'd go this route, but as the server has a WAN connection I'd rather go for something a little more closed source.

[n_K]

I guess something we never asked is; do all your VMs have vmware-tools installed on them, and are using hardware configuration version 8?

[CPressland]

I guess something we never asked is; do all your VMs have vmware-tools installed on them, and are using hardware configuration version 8?

All Version 8, All with VMWare-Tools installed except for pfSense, that's running Vanilla.

[+BudMan MVC]

Got those commands handy?

* Use a SSH-client like PuTTy to connect to your host.

* Once logged into the host, run the command "vim-cmd vmsvc/getallvms" to find all VMs on your host and theire IDs (first column). Note the IDs for the VMs you would like to autostart. Or if you would like to start everyone, theres no need to remember the ID.

* Now edit /etc/rc.local with the editor "vi" using this command "vi /etc/rc.local"

* To enter insert-mode in "vi" press "i"-key.

* Somewhere in that file (I put mine above the if-statement) write this if you want every VM to autostart:

* * for i in $(vim-cmd vmsvc/getallvms|cut -f1 -d" "|grep -v Vmid); do vim-cmd vmsvc/power.on $i; sleep 10; done

* And if you only would like to start a few of the VMs on the host, this will be enough:

* * vim-cmd vmsvc/power.on ID_OF_VM

* * sleep NUMBER_OF_SECONDS (10 in my case)

* * vim-cmd vmsvc/power.on ID_OF_VM2

* * etc...

* When you are done editing, press ESC-key to exit "insert"-mode and then write ":wq" to write and quit.

* Reboot your host and the VMs should autostart once again.

so here for example is mine

-----

~ # cat /etc/rc.local

#!/bin/sh

export PATH=/sbin:/bin

log() {

echo "${1}"

/bin/busybox logger init "${1}"

}

# execute all service retgistered in ${rcdir} ($1 or /etc/rc.local.d)

vim-cmd vmsvc/power.on 5

sleep 60

vim-cmd vmsvc/power.on 3

sleep 60

vim-cmd vmsvc/power.on 8

if [ -d "${1:-/etc/rc.local.d}" ] ; then

<snippped>

----

[CPressland]

Awesome, I'll give this a try when I get home - Vi, yuk, hopefully Nano is included in ESXi.

[+BudMan MVC]

nope -- no nano, I even looked around on how to add it. I could not find a way.

[CPressland]

nope -- no nano, I even looked around on how to add it. I could not find a way.

Small price to pay in functionality for an awesome product! (bar the issues I'm having, which I believe will go once I upgrade).

To further note, I've even moved VM SwapFiles to a separate Data Store to attempt to further improve speeds.

[n_K]

You should be able to add nano no problem! Download the source code to nano, configure it for statically linked, and move the resultant binary to ESX in /bin and you should be able to use it.

(Just compiled, I'll upload it later)

[CPressland]

All,

Unfortunately after the upgrade I'm still facing the same performance issues, both between VMs and out to other clients on the network. Tried with a dumb switch today also, no difference.

It's 100% an ESXi switch issue.

Can you guys run through how you setup your VSwitches?

[Guest]

Mines a standard switch, I also get the same issue. I still need to test the thick/thin disc theory yet though.

[+BudMan MVC]

looking forward to the nano binary - that would be sweet!

As to vswitch setup, there isn't much too it.. You have not done any advanced option changes have you?

One thing I would double check.. You mentioned you changed your MTU to 9000 at one point. From your last screen of your vswitch you show 1500. Did you remember to change the MTU/Jumbo Frame setting on the nic in your vms?

have you done any changes to the nic settings in the vms?

Mine are default settings, the only thing I do on my vms related to network settings is I do disable ipv6 if not going to be using it. I turn off the llmnr (link-layer topology discovery mapper/responder) crap

IPv6 is fully disabled, not just uncheck the binding - on the vm

reg add hklm\system\currentcontrolset\services\tcpip6\parameters /v DisabledComponents /t REG_DWORD /d 255

This for one makes my ipconfig /all much easier to read ;)

Other than that I don't recall doing anything else. So we are on the same hardware, same os even for vms -- WTF your vswitch only showing 300mbps with iperf while mine shows 1.55Gbps?? You are showing a 10gbps connection on the vm nic now that you changed to vmxnet3 right?

At a loss here to what could be so different in our setups? Same freaking hardware N40L, esxi 5 u1 from vmware. Are you running your esxi off a usb disk or anything? Mine is installed to the 250GB drive that came with the N40L

Have made any changes to the bios settings of the N40L? What bios version do you have - mine is dated 7/29/11 I believe - I know its 7/11 something.

"I still need to test the thick/thin disc theory yet though." - this might have some issues with your file copies to and from vms, but should have nothing to do with the speed he is seeing on his vswitch - have you run an Iperf test between vms, and to box outside your host. Do you also see bad performance? Cpressland was seeing like 355Mbps, I show like 800 between vm and box outside. And 1.55Gbps between vms connected to the same vswitch. Iperf would remove any disk bottlenecks from the test.

Have you taken sniff using one of your vms -- is it possible that your vswitch is swamped with other traffic, maybe a broadcast storm or something going on? Turn on all the networking items -- do you see anything weird.. Mine is pretty much zero -- not doing anything currently.

Have you done any tweaking of the power management setting? Mine is set to balanced.

Under your security profile - what services do you have running/stopped?

[n_K]

So mediafire requires an account now... unimpressed. *continues to look for an alternative host*

Alright done, http://www63.zippysh...76335/file.html

Damnit, segmentation fault when running on ESXi...

Damn. Tried a full static compile of ncurses AND nano and it still won't work :( shame

Edited April 27, 2012 by n_K

[CPressland]

As to vswitch setup, there isn't much too it.. You have not done any advanced option changes have you?

No Advanced settings at all, the only thing I changed from the defaults is the number of ports, which I brought down from 100+ to 24.

have you done any changes to the nic settings in the vms?

Nope, other than setting static IPs.

Are you running your esxi off a usb disk or anything?

Running off of an 8GB Kingston SSD mounted in the CD Bay area and connected directly to the sata connection on the mobo. While not the fastest SSD, it's a crap ton faster than a standard HDD.

Have made any changes to the bios settings of the N40L? What bios version do you have - mine is dated 7/29/11 I believe - I know its 7/11 something.

BIOS is stock - no upgrades done due to requiring to install Windows to update.

Have you done any tweaking of the power management setting? Mine is set to balanced.

Mine is set to 'High Power'

Under your security profile - what services do you have running/stopped?

All off, except:

tbtd

SSH

Direct Console UI

[CPressland]

So - Budman has been VPNed into my Network for the last few days and we've been trying to figure this out. We've covered BIOS updates, swapping NICs around, updating VMWare Tools, disabling DNS on Windows Server 2008 R2 due to it flooding the network with UDP Packets. We've tried everything. Any further ideas Neowin?!

[remixedcat]

Would you please post this in the WMWare forums. I would really appreciate this to be brought to thier attention and it may get resolved quicker! Thank you!!!

[CrimsonBetrayal]

I would also be interested in a solution for this. I dumped ESXi because of the poor network performance. I've gone with a straight Solaris solution, I would however love to use ESXi, hell I'd even pay for it if I was guaranteed that it would give me the full performance of my network. But without this, ESXi is not right for me.