Is ESXi for me

[neoraptor]

Recently I replaced my old htpc server with this :

 

Core i5 5200U / currently 4GB of ram / 750GB wdc black / dual lan / broadcom wifi.

My htpc (debian and recently ubuntu) was used for minidlna, youtube for the kid and manual backups via samba shares.

 

Background:

- I have some networking (ccna long ago) and virtualization (vmware workstation and virtualbox VMs for development) knowledge but never used baremetal virtualization.

- Currently using tp-link router with gigabit ports with stock OS (was thinking to put openwrt at some point). Main dev station, htpc and tv are wired, phones, tablets, wife's laptop - wireless; Don't use advanced functionality on the router (media sharing, vpn, qos etc)

 

So I was thinking to put esxi to have pfsense (or other router os) and my htpc on that box.

 

While i know this is possible here are my questions:

1. Are two nics enough (read somewhere it is suggested to have a separate nic for esxi admin, but adding one will be difficult as there aren't too many mini pcie lan cards)

nic1 - WAN port for router

nic2 - LAN -> switch --> main dev station, tv

 

2. If not using anything fancy on my current router what will I gain with pfsense  (was thinking for router based ad blocking)? I know it will be faster but i doubt that I'll see much difference for my wired devices (and i don't care for the rest too much)

 

3. I want to use the built in wifi card for access point - it is Broadcom but I can put Atheros as I read I have better chance for getting it to work. I know I can use my current router as AP but i want to have it untouched so i can just plug it in case of any issues. I also know there are great APs but I don't plan spending more.

 

4. How is the learning curve for esxi, I see there are plenty of resources and knowledgeable people here but I don't have too much spare time right now (having a 2 year old kid is kind of time consuming )?

 

5. Will the installation wipe my hard drive? I'm planing to install it on usb flash drive.

 

Sorry for the chaotic thread... I would post a potato but don't have a virtualized one

[binaryzero]

You won't gain anything but a headache. 

 

Leave the setup as it is lol. 

[shockz]

3 hours ago, Jared- said:

You won't gain anything but a headache. 

 

Leave the setup as it is lol. 

How so? Pfsense is pretty easy to install and configure, and there are plenty of official guides on doing just about anything.

 

I've been using pfsense for years, but recently moved to a lightweight firewall called ipFire. Another solid firewall is Sophos UTM 9, free for home users. Their XG appliance is garbage however.

 

The only thing I regret about my ESXi server is not setting one up sooner. Having full control over my network, with the ability to fine tune things more than I ever could with even tomato or DDWRT makes it worth it right there, let alone having my file server, web server, dhcp, dns, wireless, AD/domain, firewall all consolidated to one box at my home. The power savings alone makes it worth it. Having a lab environment isolated from everything else is wonderful too.

 

My advice, jump on it and install ESXi.

 

I find that using a USB drive or SD card is best practice when installing the hypervisor, that leaves your actual hard drives dedicated to VMs. It's also much easier to recover your infrastructure as your main drives can be left alone should something go wrong with the hypervisor. 

 

Two NICs is enough for a basic network setup that brings in your internet to the rest of your network.

 

 

The learning curve isn't hard at all, when I first installed it, I was up and running, logged in to vSphere in only a few minutes.

 

When you setup your datastore(s) for ESX, it will wipe any existing data/partitions on the drives you'll be placing VMs on. If your datastore will also be on your USB drive, this will happen automatically when ESXi is installed onto it. I'd recommend using something more permanent though for a datastore, not a USB drive. Aside from loading the hypervisor on USB, anything else will be pretty slow on it.

Edited January 5, 2016 by shockz

[binaryzero]

LOL! Good luck, I hope you're here to support him when he has a hundred and one questions. 

 

 

[shockz]

2 minutes ago, Jared- said:

LOL! Good luck, I hope you're here to support him when he has a hundred and one questions. 

It's a forum, of course people will be around. It was a wonderful learning experience, that's for sure. And having a lab now to play with just about anything lets me keep up on the latest and greatest. 

 

ESXi isn't hard, and pfsense with a simple IPS, Firewall & Port Forwarding took me a half hour to setup. The biggest quirk for me was figuring out a few NAT options for port forwarding. A google search fixed my issue in a few mins. I've not messed with adblocking on a firewall level, but there are guides on the pfsense wiki.

[+BudMan MVC]

While 2 nics is enough, actually with vlans you could do it with 1.

 

But sure more nics are better, while it is nice to breakout your vmkern to its own nic for performance in moving files to and from the datastore.  This is not something you do all that often so sharing your vmkern on the same nic as your lan vswitch is not that big of an issue.

 

As to AP... Why don't you just leverage your current router as your AP?  Or do you have a gateway? modem/router ??

 

So what else do you want to run on this hardware that you want to use virtual?  The 4GB might be a bit of a restriction if your planning on running multiple vms..  But pfsense does not need a lot of ram or cpu unless you plan on running stuff like snort and squid, etc.

 

As to learning curve.. To be honest esxi is pretty simple in a home setup, sure it can get complicated in an enterprise with clusters of hosts and vmotion and networked storage, etc. etc.  Think of it as machine running virtualbox or workstation.  Where your interface is just a client..

 

 

Vs running the software to manage your machines via interface on your OS on the machine running your VM like vb or workstation.  You just run the software client on a different machine is all.  But editing/setting up your vms is not much different than using vb or workstation.  If you have experience with these then your ahead of the game already.

 

There are plenty of people here running esxi, and many of them running pfsense on it.. I have been for years and years.. Be more than happy to help you with any question on either esxi or pfsense..

[+Fahim S. MVC]

1) Two NICs are plenty for the set-up you are suggesting. 3 would be better.  I run with 2 (LAN and WAN, as you have described).

2) pfSense is completely rock solid - which to me is as good a reason as any to use it.  I only have to reboot my pfSense VM when it is going through a software upgrade (or the hypervisor is).

3) It's possible, you just have to pass through the port to the VM.  I have never done pfSense & wifi on the same box (and in fact would recommend against it), so don't know about card compatibility.

4) ESXi is relatively easy - if I can learn it, so can you.

5) As part of the installation you will be asked to designate a drive as a datastore.  If I remember correctly, that drive, if it isn't already an ESXi datastore, will be wiped to create a datastore.

 

I agree with @BudMan though.  4GB of memory isn't very much, and will limit the number of VMs you can create.  For the record my pfSense VM is set up to use 512Mb of RAM and 4GB of disk.

[+BudMan MVC]

Yeah I normally run my pfsense at 512, I had bumped it to 2GB for playing with snort.. Now that I am done with my playing prob drop it back down to 512.. 

 

If your not running squid with caching disk usage of pfsense is going to be very light..

/ (ufs): 29% of 3.9G

 

Fahim brings up a good point about trying to use the boxes wifi in psfsense if pfsense is running in a vm..  I am with Famim here on against using pfsense as your actual AP..  I just don't see the point to this..  You can use any wifi router as just an AP..  You can pick up some $20 wifi router and have multiple AP around the house.  Or jump to the next level and get actual AP, and use those that support poe so it can be properly mounted for best coverage.  If your going to have a esxi host now, I would really suggest unifi - the new line of AC model AP start at $89 for the lite model.  You can then run the controller software as a vm, see that UC vm on my esxi screeny

 

And I will agree as well if Fahim can learn it, anybody can learn it heheheheh J/K buddy and congrats on your new mvc status..  You had my vote!  Check the mvc&staff section now that your mvc

 

 

[Haggis Veteran]

So I did exactly this

 

I had never used ESXi before, or pfsense before and went from have just a debian install on the local drive

 

So i now run, Server with ESXi, a vm for pfsense and a vm with my plex install

 

My server does have 3 NIC but as others have said just put your vmkern on the same as your LAN

 

As i said i have never used ESXi before but i have never had to do anything with it after the insital install, this was very easy

 

Then i installed vsphere on my laptop and connected to the ESXi host to setup the VM's

 

 

 

pfsense

 

There are loads of guides to setting this up, again i installed it following a guide and its been sat there ever since just working away

 

During the install you setup the NIC's you want to use for LAN and WAN

 

I found it easier to unplug them and then when it asked for LAN plug that one in so you know you were selecting the right one

 

and then on the WAN one do the same, this saves you having to work out what one is what

 

 

 

If you need help with anything just post here, i know roughly what i am doing LOL, @BudMan and @Fahim S. know a lot more than me

[+BudMan MVC]

Yeah figuring out which nic is which might be a issue depending on your cards in your host box.  If they are different brands/models its real easy.  Unless you happen to have multi port nics (See below)  And then once you setup a vm, you can set the mac on the vnics you want that you assign to pfsense or just look what mac esxi gives it so when you setup pfsense you know exactly which interface is which.

 

When you setup esxi you can leave the wan connection disconnected so your sure setting up vmkern/lan switch on the correct physical nic connected to your network.  Then once you have that setup, setup your wan vswitch connected to your modem or current wan modem/router.

 

 

As you can see it will show your physical nics, and then you can make sure the correct nic is assigned to the correct vswitch..  Then when you setup vms its real easy to assign the vnic of that vm to whatever network you want.  With just wan and lan is really simple, as you add physical nics it can become more complicated if you want to do load balancing, vlans over that physical network, etc. etc..  But in a single wan with vmkern/lan setup it pretty straight forward..

[neoraptor]

Big thanks to all who replied!

 

@Jared- why the long face ? Forums are for asking and helping each other. I'll do my best to not ask too stupid questions and will try to share whatever i find useful for the others.

 

@Fahim S. @BudMan :

1. RAM - I'm planing to have pfsense vm (512MB), htpc vm (ubuntu - 1-1.5GB), LAMP server vm (ubuntu server - 512MB, currently running locally on my dev laptop in virtualbox). So I should fit just fine in 4GB, I can always get another stick of 4/8 DDR3L ram if needed.

2. AP - was considering using the router (TL-WR1043ND, no modems) but want to have zero downtime during initial setup and while everything is working nicely. Another router is an option but don't want to spend extra cash right now (I'm aware of real APs but ... some other day).

 

Here is my current plan:

1. Try to switch the wifi cards with my laptop as Broadcom isn't supported in pfsense and will boot it from usb to see if it works.

2. If it works esxi install on usb/sd card. If not, will start looking for cheap and cheerful second hand router for AP.

3. Get a gigabit switch.

 

My main question was if it is worth to do it (with the limited free time i have) and by the looks of it (the replies above) it surely is.

[+Fahim S. MVC]

39 minutes ago, neoraptor said:

Big thanks to all who replied!

 

@Jared- why the long face ? Forums are for asking and helping each other. I'll do my best to not ask too stupid questions and will try to share whatever i find useful for the others.

 

@Fahim S. @BudMan :

1. RAM - I'm planing to have pfsense vm (512MB), htpc vm (ubuntu - 1-1.5GB), LAMP server vm (ubuntu server - 512MB, currently running locally on my dev laptop in virtualbox). So I should fit just fine in 4GB, I can always get another stick of 4/8 DDR3L ram if needed.

2. AP - was considering using the router (TL-WR1043ND, no modems) but want to have zero downtime during initial setup and while everything is working nicely. Another router is an option but don't want to spend extra cash right now (I'm aware of real APs but ... some other day).

 

Here is my current plan:

1. Try to switch the wifi cards with my laptop as Broadcom isn't supported in pfsense and will boot it from usb to see if it works.

2. If it works esxi install on usb/sd card. If not, will start looking for cheap and cheerful second hand router for AP.

3. Get a gigabit switch.

 

My main question was if it is worth to do it (with the limited free time i have) and by the looks of it (the replies above) it surely is.

When you say 'will it boot from usb', what do you mean by 'it'?

[neoraptor]

1 minute ago, Fahim S. said:

When you say 'will it boot from usb', what do you mean by 'it'?

After I switch the wifi cards I will use pfsense installed on usb (no esxi) to see if I can use the Atheros wireless adapter as AP.

[+Fahim S. MVC]

10 minutes ago, neoraptor said:

After I switch the wifi cards I will use pfsense installed on usb (no esxi) to see if I can use the Atheros wireless adapter as AP.

oh ok - that makes a lot of sense.

[+BudMan MVC]

you do understand that booting pfsense native to access the wireless card is a bit different than pfsense as a vm and using the wifi card..  Does this host support passthru?

 

"will start looking for cheap and cheerful second hand router for AP."

 

I really don't get the whole reluctance to spending money on an actual AP and proper placement...  Everyone wants wifi at max speed everywhere, but then when it comes to spending a few bucks to make that happen..  But then you see people dropping 200 without issue on the latest router with wifi that they stick in some corner of their house and then bitch that they don't get full speed AC or 5ghz N in the family room, etc..

 

I hope it works out for you, but tell you what if you want to be happy with your network and you use wifi -- your going to want to go with real AP.. with proper deployment for good coverage..

 

I am not even a big fan of wifi, if it doesn't move its wired in my house.  To be honest even AC is liking paint drying compared to gig wire..  My chromecast even went wire as soon as they came out with ethernet for it..  And I put in another AP and I have a small house because the 5ghz coverage out the patio wasn't very good    So I put in a unifi uap-ac-lr the new 2gen stuff in the kitchen right by the patio door..  Now my clients all max out speed of their nic out there, or speed of my internet connection if on ac with 80mbps down...

[tbarnett]

There are plenty of HP Proliant DL385 G6 & G7 servers on eBay.  I have several of each.  Most have additional NICs since they are most likely used enterprise equipment. 

[neoraptor]

16 hours ago, BudMan said:

you do understand that booting pfsense native to access the wireless card is a bit different than pfsense as a vm and using the wifi card..  Does this host support passthru?

If it doesn't work in native (non virtualized) installation it won't work for sure under esxi.

 

16 hours ago, BudMan said:

I really don't get the whole reluctance to spending money on an actual AP and proper placement...  Everyone wants wifi at max speed everywhere, but then when it comes to spending a few bucks to make that happen..  But then you see people dropping 200 without issue on the latest router with wifi that they stick in some corner of their house and then bitch that they don't get full speed AC or 5ghz N in the family room, etc..

 

As I said in my first post, everything important is wired and I don't care too much for the wireless devices (as long as youtube is fine). I live in apartment and don't have any issues with covering the area (even car parking spot has some connectivity).

 

15 hours ago, tbarnett said:

There are plenty of HP Proliant DL385 G6 & G7 servers on eBay.  I have several of each.  Most have additional NICs since they are most likely used enterprise equipment. 

Why would I want an expensive, noisy, heavy/bulky old server for my living room (especially when i got a fanless i5 with low power usage) ?

Edited January 6, 2016 by neoraptor

[binaryzero]

/me pets the 4 Meraki APs I have next to me.