Jump to content



Photo

Building a ESXi box...


  • Please log in to reply
61 replies to this topic

#1 +riahc3

riahc3

    Neowin's most indecisive member

  • Tech Issues Solved: 11
  • Joined: 09-April 03
  • Location: Spain
  • OS: Windows 7
  • Phone: HTC Desire Z

Posted 19 January 2014 - 22:48

Hello,

I problably wont get this now because I plan to buy this "part per month" to get costs down but Id like to build a ESXi box.

Now most will recommend me the HP microserver (this or previous gen). What pros and cons are there to this?

This ESXi box will run:

Windows Server 2012 R2 (my domain controller)
pfSense (my router)

The WS2012R2 will have StableBit DrivePool to setup some storage pools and divide 4 disks of 3TB into my personal space and two other family members so when they log on they will have their own personal space and will also load profiles, settings, etc from it. So in a sense, it will also act as a NAS to stream to them and to media device thru my home.

Id like to hear some opinions :)

Thank you


#2 The_Observer

The_Observer

    Apples, Bananas, Rhinoceros!

  • Joined: 12-April 05
  • Location: New Zealand
  • OS: OS X 10.9
  • Phone: iPhone5s

Posted 19 January 2014 - 23:23

I just moved from DrivePool to FlexRaid. Had DrivePool for years, but losing half my spaces wasn't cool. Reddit has a great VMware subreddit i recommend. I don't know how FlexRaid will work in VM. Good luck on this.



#3 Klownicle

Klownicle

    I get a custom title? Sweett.

  • Joined: 22-June 02

Posted 19 January 2014 - 23:58

If you plan on using ESXi 5.5 beware the latest limits on legacy/consumer hardware no longer in the compatibility list.  You can get around this by using 5.1 U1 first then upgrading to 5.5 as the drivers from 5.1 U1 are kept during the upgrade.  Also, any 5.5 ESXi VM10 cannot be managed with the VmSphere Client, it must use the Web Client which isn't free.  So if you create anything to use the 5.5 features you must have a vCenter server which has the web client to fully manage.  The VmSphere Client will only be able to power on or use the console for VM's that are version 10.

 

A lot of legacy/consumer Sata controllers and Nic's have been removed.



#4 OP +riahc3

riahc3

    Neowin's most indecisive member

  • Tech Issues Solved: 11
  • Joined: 09-April 03
  • Location: Spain
  • OS: Windows 7
  • Phone: HTC Desire Z

Posted 20 January 2014 - 08:03

Hello,

If you plan on using ESXi 5.5 beware the latest limits on legacy/consumer hardware no longer in the compatibility list.  You can get around this by using 5.1 U1 first then upgrading to 5.5 as the drivers from 5.1 U1 are kept during the upgrade.  Also, any 5.5 ESXi VM10 cannot be managed with the VmSphere Client, it must use the Web Client which isn't free.  So if you create anything to use the 5.5 features you must have a vCenter server which has the web client to fully manage.  The VmSphere Client will only be able to power on or use the console for VM's that are version 10.
 
A lot of legacy/consumer Sata controllers and Nic's have been removed.

Yup, Ive seen there are a lot of limits now put in place by ESXi 5.5

I plan on consulting the VMWare Compatability list before hand and see what hardware is fully compatible.

I read about 5.5 ESXi VM10 not being able to be manages without a paid client but can I manage it with VMWare Workstation 10? Or do I have to use the the client?

Also Im a bit lost on the "vCenter server" bit; What exactly is this? Isnt the ESXi the server and thats it? Excuse me for my ignorance.

#5 +Chicane-UK

Chicane-UK

    Neowinian Senior

  • Tech Issues Solved: 1
  • Joined: 02-November 01
  • Location: The UK!
  • OS: MacOS 10.9 Mavericks
  • Phone: Google Nexus 4

Posted 20 January 2014 - 08:19

vCenter server is what you use if you need to centrally manage more than one ESXi server, and is basically required if you want to start using any of the more advanced VMware features such as clustering, vMotion, Storage DRS, etc etc. All this is underpinned by vCenter Server. 

 

If however you only want to run a single ESXi server, with a couple of VM's on it, you won't need vCenter :)

 

I have a number of friends who run ESXi on HP MicroServers and they're all pretty happy with it. As has been said though, VMware and some of the vendors have been turds with ESXi 5.5 and really limited the amount of hardware it can run on.. I'm bitter because a number of not even 3 year old HP servers at work are no longer supported for 5.5 and it's really scuppered our plans as we were expecting to go right from ESXi 5.0 to 5.5! So definitely check that HCL - I'd imagine the Gen8 MicroServer will be supported though so that would be an ideal platform to build on for a small home ESXi box. 



#6 OP +riahc3

riahc3

    Neowin's most indecisive member

  • Tech Issues Solved: 11
  • Joined: 09-April 03
  • Location: Spain
  • OS: Windows 7
  • Phone: HTC Desire Z

Posted 20 January 2014 - 12:23

Hello,

vCenter server is what you use if you need to centrally manage more than one ESXi server, and is basically required if you want to start using any of the more advanced VMware features such as clustering, vMotion, Storage DRS, etc etc. All this is underpinned by vCenter Server. 
 
If however you only want to run a single ESXi server, with a couple of VM's on it, you won't need vCenter :)

Then I dont think I need it. Im not gonna do anything except deal with one ESXi server.
 

I have a number of friends who run ESXi on HP MicroServers and they're all pretty happy with it. As has been said though, VMware and some of the vendors have been turds with ESXi 5.5 and really limited the amount of hardware it can run on.. I'm bitter because a number of not even 3 year old HP servers at work are no longer supported for 5.5 and it's really scuppered our plans as we were expecting to go right from ESXi 5.0 to 5.5! So definitely check that HCL - I'd imagine the Gen8 MicroServer will be supported though so that would be an ideal platform to build on for a small home ESXi box.

Yes, Ive read all the hate towards 5.5; Its even gotten me to think about staying on 5.1 even if things like USB 3.0 passthrough isnt avaliable (AFAIK, VM guests have to be at version 10 to support this, right?)

Also, for the HP MicroServers, arent they ECC RAM? Forgive my ignorance...

I would love to build a ESXi box but hey if the MicroServer Gen8 gives me all I need that I couldnt get from a DIY (and then some), why complicate myself with builds? I have no idea so thats why Im asking opinions :)

#7 OP +riahc3

riahc3

    Neowin's most indecisive member

  • Tech Issues Solved: 11
  • Joined: 09-April 03
  • Location: Spain
  • OS: Windows 7
  • Phone: HTC Desire Z

Posted 20 January 2014 - 12:52

Hello,

For the record I see HP ProLiant MicroServer Gen8 G1610T (2GB RAM, Intel CeleronG1610T) for 438,36€

This excludes G.Skill Ripjaws X DDR3 1333 PC3-10666 8GB at 71.95€ each stick (143,90 €) (why gaming RAM? Its actually cheaper than some Kingston valueram....)

Im saying this because I was tempted to max it out but I just saw that WS2012R2 only asks for 512MB RAM (long time I havent read that). I might be able to get away with less but I did read BudMan that ESXi sucks a lot of RAM up (although he was able to get it down to 3GB). Excuse again my ignorance, but, besides processor, when streaming a 1080 video to a 802.11n device, I need a large ammount of RAM right?

#8 +BudMan

BudMan

    Neowinian Senior

  • Tech Issues Solved: 96
  • Joined: 04-July 02
  • Location: Schaumburg, IL
  • OS: Win7, Vista, 2k3, 2k8, XP, Linux, FreeBSD, OSX, etc. etc.

Posted 20 January 2014 - 14:55

NOT esxi -- vserver appliance.. NOT esxi itself.. Only if you plan on playing with the vcenter server appliance does it like to use up a bit of ram.. Which you can tweak down yes. If you plan on going with 5 or 5.1 you wouldn't never need the vcenter anything. If 5.5 and you upgrade your vms to 10 is the only time you would need vcenter or workstation to CHANGE their settings, ie add a nic or something.. If you create the machine how you like it - and then upgrade it to 10 you won't need vcenter unless you need to "change" something on it. You can use the client to access console, etc.

As to the HCL -- yeah mine was a upgrade.. I had heard 5.5 stripped some equipment, but its working fine on my N40L via an upgrade and my added nics, etc.

You only need large ram if you plan on running more than a couple of vms. I currently run 4 of them 24/7/365 -- if your only running 2 where pfsense only needs 512/1G unless you plan on playing with snort and squid, etcl

pfsensevm.png

If all you plan on doing with it is pfsense and another 512 box.. I highly doubt 2012 performs well with only 512 ;)

Taking them to 8GB is fairly cheap, taking them to 16 cost a bit more. You do not need to run ecc, I sure am not in my n40l.. Maybe that changed with the gen8? You would only really need that if machine doesn't run unless it is.. Or this was critical box, not a home setup.

#9 +ChuckFinley

ChuckFinley

    member_id=28229

  • Joined: 14-May 03

Posted 20 January 2014 - 16:53

Yeah my HP Microserver works just fine running 5.5 I havent updated the VM's though. It seems to all work OK for what I need. 



#10 OP +riahc3

riahc3

    Neowin's most indecisive member

  • Tech Issues Solved: 11
  • Joined: 09-April 03
  • Location: Spain
  • OS: Windows 7
  • Phone: HTC Desire Z

Posted 20 January 2014 - 19:17

Hello,

If 5.5 and you upgrade your vms to 10 is the only time you would need vcenter or workstation to CHANGE their settings, ie add a nic or something..

I had that doubt; Do I need vcenter OR could all modifications (such as add a nic, another HDD, passthrough, etc) be done with VMWare Workstation 10? Im used to Workstation 10 so I can get by with that :)

 

Only if you plan on playing with the vcenter server appliance does it like to use up a bit of ram

Im missing something because the vCenter Server I install on a client machine (regular laptop running regular Windows for example) so why would that take up RAM on a barebone ESXi server???
 

You can use the client to access console, etc.

Accessing the console is basically accessing a unix shell directly on the ESXi server right?

 

As to the HCL -- yeah mine was a upgrade.. I had heard 5.5 stripped some equipment, but its working fine on my N40L via an upgrade and my added nics, etc.

Ive read sources that 5.5 has dropped a lot of old hardware support.......old meaning 2-3 years. Yeah, in technology that's pretty old but imagine if a regular OS did it...


 

You only need large ram if you plan on running more than a couple of vms. I currently run 4 of them 24/7/365 -- if your only running 2 where pfsense only needs 512/1G unless you plan on playing with snort and squid, etcl

I only plan 2 right now 24/7/365 (cant think of another interesting VM to run always)

I don't think Ill be interested in major packages (always had a thing for Asterisk though).
 

If all you plan on doing with it is pfsense and another 512 box.. I highly doubt 2012 performs well with only 512 ;)

I know :p I was gonna put 4GB on it as that's what my Windows 7 box has.


 

Taking them to 8GB is fairly cheap, taking them to 16 cost a bit more. You do not need to run ecc, I sure am not in my n40l.. Maybe that changed with the gen8? You would only really need that if machine doesn't run unless it is.. Or this was critical box, not a home setup.

From Neowin's review, they look like standard non ECC RAM sticks....which I imagine ANY RAM can be used that is non HP.

Worst that could happen is that I see 8GB isn't enough and just get another stick I guess....

Also I thought it was pretty important but since no one has mentioned it, maybe its something stupid and pointless: the processor of the microserver doenst support VT-d ....

#11 OP +riahc3

riahc3

    Neowin's most indecisive member

  • Tech Issues Solved: 11
  • Joined: 09-April 03
  • Location: Spain
  • OS: Windows 7
  • Phone: HTC Desire Z

Posted 20 January 2014 - 19:58

Hello,

Well this caught me off guard; Installing VMWare vCenter, I can't install one of the main prerequisites which is vCenter Single Sign-On because it seems it requires a 64-bit operating system.

#12 +BudMan

BudMan

    Neowinian Senior

  • Tech Issues Solved: 96
  • Joined: 04-July 02
  • Location: Schaumburg, IL
  • OS: Win7, Vista, 2k3, 2k8, XP, Linux, FreeBSD, OSX, etc. etc.

Posted 20 January 2014 - 21:26

You would only need workstation if you upgraded the vm hardware to 10, if 8 or 9 or 7, etc. then the client works just fine. No its not console based. See the screenshot showing the pfsense vm with only 512. That is the Vclient. The vcenter appliance.. Looks for 8 gig on install, you can change it.. And then tweak it to actually run.. for example I have mine with 3GB currently..

vcenter.png

Here is guide on lowering its ram requirements
http://www.virtually...-memory-to.html

here is link to how to install the appliance
http://kb.vmware.com...ernalId=2007619

As to 64 bit..

What were you going to install vcenter on - thought you had
Windows Server 2012 R2 (my domain controller)

2012 does not come in 32bit versions..

As to VT-d -- what hardware do you want your VMs to directly connect too? In a home lab why would you need this.. So what if your vm sees virtual nic vs direct access to the physical nic. You can give your vm direct access to disk with a raw map.. It doesn't need direct access to the controller the disks are on.

I have connected wifi nics and camera to vms all via just usb passthru.. Its a home/lab setup not sure why you would need direct I/O access to specific hardware.

That being said I thought one of the features the gen8 models brought was VT-d?

#13 OP +riahc3

riahc3

    Neowin's most indecisive member

  • Tech Issues Solved: 11
  • Joined: 09-April 03
  • Location: Spain
  • OS: Windows 7
  • Phone: HTC Desire Z

Posted 20 January 2014 - 22:10

Hello,

You would only need workstation if you upgraded the vm hardware to 10, if 8 or 9 or 7, etc. then the client works just fine. No its not console based. See the screenshot showing the pfsense vm with only 512. That is the Vclient. The vcenter appliance.. Looks for 8 gig on install, you can change it.. And then tweak it to actually run.. for example I have mine with 3GB currently..

I understood console as a actual terminal console....sorry.


Here is guide on lowering its ram requirements
http://www.virtually...-memory-to.html

here is link to how to install the appliance
http://kb.vmware.com...ernalId=2007619

Ill refer to these later. Its obvious I am VERY lost sadly :(


As to 64 bit..

What were you going to install vcenter on - thought you had
Windows Server 2012 R2 (my domain controller)

I was installing the VMWare vSphere portion of this entire thing:

Untitled.png

It fails as soon as I try vCenter Single Sign-On, saying I need 64-bit. I believe like you mentioned that this is only necessary to MODIFY VM 10s I create, right?




As to VT-d -- what hardware do you want your VMs to directly connect too? In a home lab why would you need this.. So what if your vm sees virtual nic vs direct access to the physical nic. You can give your vm direct access to disk with a raw map.. It doesn't need direct access to the controller the disks are on.

Well, now that you mention NICs, wouldn't Backtrack need direct access to a 802.11g USB adapter? Im doubting that right now so sorry.....

I have connected wifi nics and camera to vms all via just usb passthru.. Its a home/lab setup not sure why you would need direct I/O access to specific hardware.

Does performance suffer a lot?

That being said I thought one of the features the gen8 models brought was VT-d?

Its a processor limitation, not a chipset one :)

#14 OP +riahc3

riahc3

    Neowin's most indecisive member

  • Tech Issues Solved: 11
  • Joined: 09-April 03
  • Location: Spain
  • OS: Windows 7
  • Phone: HTC Desire Z

Posted 21 January 2014 - 11:56

Hello,

I have another question:

Like I mentioned, I have 4 3TB disks. Obviously I need to install WS2012R2 and pfSense somewhere.

How exactly does this work out? Would a 5th 320GB disk be OK to install this?

#15 Fahim S.

Fahim S.

    Neowinian Senior

  • Tech Issues Solved: 2
  • Joined: 15-April 02
  • OS: Windows 8 - OG
  • Phone: Google Nexus 4 16GB by LG

Posted 21 January 2014 - 12:11

The Gen8 MicroServer doesn't have a place you can put a 5th disc unless you use a 2.5" disc (spinning or SSD) in the optical bay.

The Gen7 MicroServer has room for an extra 2 3.5" discs or up to 8 2.5" discs (you need a controller card for them to plug into).