Proxmox networking for a beginner, with virtualised router

[Biohead]

I'm trying to consolidate a lot of mini-PCs I have around the house running fairly simple tasks (Homeassistant, Arr stack, Jellyfin etc) as well as setting up a new pfsense instance as my home router - in VM form. My NAS will remain a separate machine. I have dual WAN connections (1000/1000 and 1000/100), but I don't do anything particularly heavy on the router side. I'd like to setup some decent ad-blocking capabilities, but I don't think that's particularly taxing.

I've opted for one of the Intel N305 mini-PCs with multiple (6) LAN ports running Proxmox and I'll have a pfSense VM with 1 or 2 cores and 2GB RAM, and everything else I'll try to use containers and have them speak to each other directly. I have some experience with Proxmox and pfsense, but the networking side of both together has made me scratch my head.

I'm trying to get my head around the network side of the actual proxmox machine.  I've been "advised" (i.e. told best practice) to use a PVE Management port, a PVE LAN port, then my two WAN ports and a normal LAN port are passthrough to pfsense. I understand the PVE Mgmt port is to access Proxmox if the router is down for whatever reason, but I'm struggling to see the difference between the PVE LAN port and the normal firewall LAN port. Would the PVE LAN port be bridged into pfsense and both the bridge and the passthrough port are assigned as LAN??

I'm also on the lookout for best practices... networking really isn't my thing. Should I give Proxmox an IP in my normal home networking range, or do I keep it well away from that? If I keep it separate, would I be able to access Proxmox from the LAN or would I have to plugin?

Sorry for what probably seems like some very simple questions I really should already know! I'm just a bit excited by how powerful these new little Alder Lake-N chips actually are!

 

[Nick H. Supervisor]

On 14/10/2023 at 13:36, Biohead said:

Sorry for what probably seems like some very simple questions I really should already know! I'm just a bit excited by how powerful these new little Alder Lake-N chips actually are!

<Moved to Smart Home, Network & Security>

[br_]

On 14/10/2023 at 08:36, Biohead said:

I'm trying to consolidate a lot of mini-PCs I have around the house running fairly simple tasks (Homeassistant, Arr stack, Jellyfin etc) as well as setting up a new pfsense instance as my home router - in VM form. My NAS will remain a separate machine. I have dual WAN connections (1000/1000 and 1000/100), but I don't do anything particularly heavy on the router side. I'd like to setup some decent ad-blocking capabilities, but I don't think that's particularly taxing.

I've opted for one of the Intel N305 mini-PCs with multiple (6) LAN ports running Proxmox and I'll have a pfSense VM with 1 or 2 cores and 2GB RAM, and everything else I'll try to use containers and have them speak to each other directly. I have some experience with Proxmox and pfsense, but the networking side of both together has made me scratch my head.

I'm trying to get my head around the network side of the actual proxmox machine.  I've been "advised" (i.e. told best practice) to use a PVE Management port, a PVE LAN port, then my two WAN ports and a normal LAN port are passthrough to pfsense. I understand the PVE Mgmt port is to access Proxmox if the router is down for whatever reason, but I'm struggling to see the difference between the PVE LAN port and the normal firewall LAN port. Would the PVE LAN port be bridged into pfsense and both the bridge and the passthrough port are assigned as LAN??

I'm also on the lookout for best practices... networking really isn't my thing. Should I give Proxmox an IP in my normal home networking range, or do I keep it well away from that? If I keep it separate, would I be able to access Proxmox from the LAN or would I have to plugin?

Sorry for what probably seems like some very simple questions I really should already know! I'm just a bit excited by how powerful these new little Alder Lake-N chips actually are!

 

First question is if this is a good idea for your setup — putting your router into a proxmox server that’s also running other things.

The simple reasons you might want to reconsider doing such a thing are because (1) the networking is much more complicated, even if you give the router VM its own Ethernet ports, and (2) if you have downtime with the hypervisor host (e.g., a reboot to apply a kernel update, or it crashes, or you mess something up) you’ll take down your whole network. 

With all of that in mind, if you want to proceed setting up a Proxmox host to run a router VM, there are a few options.

You mentioned passing through individual Ethernet ports. If your host supports PCIe passthrough of individual Ethernet ports, that’s perfect. You just add the two or three Ethernet ports as PCIe devices in the Proxmox UI.

 

A lot of platforms don’t support that, though. For example, one older dual Xeon server I use has two Intel 82574 gigabit Ethernet ports, and I can only pass through both ports or neither. Or the older Atom platform I use with 6 NICs for my own home router running on Proxmox doesn’t support PCIe passthrough at all.

Also, does your Ethernet switch support VLANs? You’ll want to take advantage of that if you want to separate the Proxmox management connection from its internet connection. If this is just a home network without a managed Ethernet switch you’re probably better off either just putting everything on vmbr0 and setting an IP on that (to expose the Proxmox web console to your whole LAN), or setting an IP in a completely different range only on another Ethernet port.

so like, let’s say you have eno1 through eno4 as your ethernet ports: use proxmox to give it an IP in a range like 192.168.150.1/24 on eno1 and call that your management port. You can plug directly into this to access the console.
 

Make a Linux bridge vmbr0 and put eno2 into it. That’s what you’ll assign to VMs so they can access the internet.
 

Also make another bridge, vmbr200, don’t give it any physical ports, and assign it the IP 192.168.200.2/24 in Proxmox with default gateway 192.168.200.1. This is how you’ll be able to get updates for the host on the internet.

Your router VM should have two virtual NIC connected to vmbr0 and vmbr200 in this example, and then the other two physical NICs passed through.

Then plug eno3 and eno4 into your two WAN connections. Or just use one, whatever fits your setup.

And in the router VM, your physical ports are the WAN ports, and set them to DHCP. the vmbr0 vnic is the one you’ll plug into your Ethernet switch, and proxmox doesn’t have its own IP on it in this example. Vmbr200 is the internal vnic that proxmox will get an internet connection from — set that on the router VM to IP 192.168.200.1/24, and set the vmbr0 port into your current LAN IP range. Set up DHCP and DNS as you see fit/need for your setup. 
 

something like that…