Splitting an ISP allocated range to multiple networks with a single public IP each (Layer 3 Switch)

[sc302 Veteran]

it would be the ip of the firewall on the /29 network.  the firewall would route all unknown traffic to the isp modem being that one of the interfaces on the firewall would be the unsecured internet/isp modem subnet. 

 

In that diagram the firewall would have multiple ips (one for each network that it is a member of). 

[swappedsr]

Okay so when they say an isp is routing another subnet over the point to point link subnet, it  really just means the ISP has put in in static route/s  or using a dynamic protocol for the additional subnet on all their downstream equipment to the transit equipment, assuming this is how we are be able to use that additional subnet, correct?  And then on the transit router you would have to add a route for each of the subnets or advertised route to the firewall so you can use the subents you are breaking up on the firewall interfaces.  Am I right?   

[+BudMan MVC]

Why would you add routes?  You do not need to add routes to networks that are directly attached to a router.  You add routes to networks that you are NOT directly attached too.

 

The router or firewall that has the /29's attached in my previous drawing would know how to talk to those networks because it has an IP in those networks.  The fact that you put in interface in a network would tell the router/firewall how to get to that network.

 

You only need routes to networks that are not directly attached to you.

[sc302 Veteran]

the router/switch/firewall/etc will automatically add routes that are hosted on itself (if it is layer 3 or better, layer 2 cannot route). 

 

 

[swappedsr]

Sorry I was confused there.  I thought that transit icon was actually a device between isp and firewall.  Am I correct in assuming how the isp would do the routing on their side and how we would set this up.  I have never had to get an additional ip block before.

 

[+BudMan MVC]

no that is not how I would do it..  Why would you eat up public space for transit inside your network?

 

You could use rfc1918 for this transit to the tenant equipment  What is on the other side of the tenant equipment.. Is that public space?

 

Also /31 is special use case - does your equipment support its use?  Yes it makes a good transit but to be honest you don't see them very often.  But sure an ISP might use that as their transit to your network.  Once the networks are routed to you, how you route them inside your network is up to you.  If you have multiple tenant devices you might use a /29 as your transit and connect all those routers on the same transit.  Instead of eating up /30's especially if you were going use public space.

 

But your transit network between your device that has a public interface and downstream routers can use rfc1918 as the transit.. There are never any hosts on this network, and don't have to be natted - they are just a way for you to logically look at the network and know what IP your going to in your next hop.  To be honest in point to point connections you don't even need to use an IP..  But it does help keep it logical in understanding what is going on.

 

keep in mind if your isp gave you say a /24 or a /25 you could break that up into smaller networks to give to your tenants and just route to their routers that will have those subnets attached on their side of their routers.  The use of rfc1918 as transit sometimes makes for odd looking traceroutes, but you can go public to private and then back to public... You can not not route those private outside your network, but you for sure can route them inside your network to other public space.

[swappedsr]

If you have multiple tenant devices you might use a /29 as your transit and connect all those routers on the same transit.  Instead of eating up /30's especially if you were going use public space.

 

If you put tenants on the same network (/29), which is the same network between the isp and firewall, you would need a layer 2 device between the isp and my firewall in order to have tenant equipment to join that network.  Might be fine if I want tenants bypassing the firewall.  Second issue I see is how can you limit tenants from misconfiguring their public ip's with an ip used by another tenant, which would be a plus when subnetting the public block like above, because it will add segmentation between networks.

 

But your transit network between your device that has a public interface and downstream routers can use rfc1918 as the transit.. There are never any hosts on this network, and don't have to be natted - they are just a way for you to logically look at the network and know what IP your going to in your next hop.  To be honest in point to point connections you don't even need to use an IP..  But it does help keep it logical in understanding what is going on.

 

So if we use private addresses between the firewall and the tenant routers then how would it work if the tenant routers  were the only devices they had?  The outside interfaces of their routers would have to have the public addresses and not private.  If the tenants had a firewall after the router then I could see using private addressing and then routing them the public subnets.  Assuming I am right in my thinking.  

 

 

Edited March 9, 2016 by swappedsr

[+BudMan MVC]

What??  Your not understanding what a transit network is it seems - it would be a layer 3 network, yes on the same layer2..  simple enough to put a switch between your router and their router.. there you go layer 2, with a layer 3 on it.  Who would have access to this layer 2, only YOU... 

 

But sure if you want to run cable from your interface on your firewall/router and their router.. Now you hae to have as many interfaces on your firewall as you have tenants.. If you have a switch between and put all your tenants transit on 1 then you really only need to have 1 interface lan facing on your router.

 

What do you mean how it would work??? Its a transit network - why do think it has to be natted???  It only has to be natted when there is no way to route rfc1918 on the public internet, but we are not talking the public internet, we are talking your NETWORK.. And the their public IP space is routed to your public wan interface on your router... You can do really anything you want inside your network..

 

Let me draw up an example..

[swappedsr]

So you were saying that you should use private addressing between the firewall and the tenant routers.  What if the router was the only device they had, no firewall or anything behind it that need a public address, maybe just some computers that need to be natted.  The outside interface of their router would need to be using public addresses.  That is what I meant.

 

Edited March 9, 2016 by swappedsr

[+BudMan MVC]

Well if they do not have router, then you would have to hang that network off your firewall.. 

 

See my drawing.

 

 

So lets say the ISP has given you a /24 lets call it 6.6.6.0/24

 

And you want to give your tenants /29's

 

pubicA 6.6.6.0/29

publcB 6.6.6.8/29

publicC 6.6.6.16/29

publicD 6.6.6.24/29

 

So the .1 in that drawing would be .1 and .9 and .17 and .25 for those specific /29s

 

So in your firewall/router route to public A would be

172.16.0.2

Public B would be 172.16.0.3

etc.

 

So you have say 6.6.6.2 in public A, and he wants to go to google at say 8.8.8.8 -- his first hop would be 6.6.6.1, That router says oh you want to go to 8.8.8.8.. I don't have route there let me send you to my gateway 172.16.0.1..  Your firewall says hey this traffic wants to go to 8.8.8.8 that is not on my networks, I don't have any specific route to it.. So I send it to my ISP at publictransit.1  your isp gets it and routes it to 8.8.8.8

 

Google say syn,ack back towards 6.6.6.2, that routes to your ISP, your ISP says oh that goes to publicTransit.2  -- your firewall gets it ans says oh that goes to publicA and sends it to 172.16.0.2

 

No natting anywhere.. even though you used rfc1918 space.  But pubic internet never sees that space.

 

Lets say publicA wants to talk to publicB..  He ends up sending it to your firewall, your firewall says oh wait I have a firewall rule - publicA can not talk to publicB.. And does not send the traffic.  But lets say you allow it then he just sends the traffic to 172.16.0.3,

 

Walk through any scenario you want of anything talking to anything..  And walk through the hops.  If your tenants have their own networks that they want to be rfc1918 space they would need another router that gets an IP in their public space..  To you you would only ever see their public space, so you never have to nat it.

 

You can do this with 1 interface on your firewall, and even could hang that publicD off the same interface with a vlan tag and vlan switch that your transit network runs on.  Now depending on how many clients you have and what bandwidth you have and how much they are suppose to have you might need to lagg some interfaces or use multiple interfaces on your firewall for more tenants, and yes then once you break it out to a different interface you would want to use a different transit.  Say 172.16.0.8/29

 

If you have any question just ask..  Comes down to this if your tenants have their own router and they want the space routed to them, you route it over a transit.  If they don't have a router and still want public space then you have to hang it off your firewall directly.  If they have more machines then their public space you want to assign them or they can pay for then sure you nat at your firewall  You put say their /29 or give them only a /30 and put those 2 address on your firewalls wan IP.. And then give them say a 192.168.0.0/24 network that you nat for them... then you would have to forward any inbound traffic to their 2 IPs on your wan to their private IPs..

 

Remember that whole 6.6.6.0/24 is routed to your publictransit.2 IP.. So either you pass it on, or you end at your wan and nat it to something behind your firewall.

[swappedsr]

Awesome explanation, I got it.  One question.  In the last part, say just one tenant out of the three list above didn't have enough ip's and we decided to use a public /30 and NAT for them like you said above.  How would that be configured.  Obviously, the firewall inside interface would hand them the private addresses, would we use like a sub-interface on the WAN port (Same WAN port as transit.2) for the public /30 and then NAT accordingly to the private subnet given to this tenant?  Thanks for helping clear this up.  

[+BudMan MVC]

Would depend on the actual firewall you were using, but normally when doing subinterfaces there is a vlan associated.. Normally you would just create a VIP or might be called MIP or even a DIP..  What firewall are you using?  Juniper, Cisco, Palo Alto, Fortinet, Pfsense (would be impressed), Other?

 

But yes this VIP or MIP would be on the same interface as your WAN..  Keep in mind your whole /24 in our example is routed to your wan IP so that interface is going to see traffic for lots of different IPs.. So putting an IP in range on it is no big deal.

 

Its pretty much the same as when your ISP gives you multiple IPs or a /?? but they hang you off their router like in example publicD above.. How do you use those multiple IPs they give.. You normally just assign them to your router via VIP..

 

So I am curious how many tenants are you going to have to start with?  What is the main pipe you have.. Large??  What sort of netblock do you have /24 or bigger?  Do you have multiple ISPs?  Has the netblock been delegated to you, do you have a ASN?  Quite often you can advertise your network to the public internet when you actually own the space.  For example I manage a /16 that my company owns, so we can advertise any subnet of that out whatever specific connection we want so say a /22 of it goes to one location while a different /22 goes to another location.  Working with your different ISP you can allow for if one site goes down, to advertise the networks that work using ISP A to now be carried by ISP B, etc.

 

I would assume your going to put in some sort of HA pair on your firewalls be it with CARP or HSRP, etc. You wouldn't want all your tenants to be offline if your firewall/router fails.

[sc302 Veteran]

Also something to consider, put a switch between your isp interface and your firewall...assign each tenant their own firewall appliance or virtual appliance with their own ip address.  This way they are completely segregated from each other, and can create their own rules as needed for their individual sites.   Much less of a configuration headache on your end and a more secure network on their end...you want to remove their access, simply unplug their router/firewall. 

[+BudMan MVC]

^valid approach sure.  But as the owner of the isp connection I would more than likely like to have my firewall between them and internet where you could restrict specific things if you so desired, would be able to keep easy ie on what kind of traffic they are doing and amount.   Monitoring could be done off the switch with some span ports or flows.

 

But having the firewall between them and the internet would give you more control - if that is what you desire.  Maybe you want to restrict them from providing specific services to the internet that are prone to attacks like dns and or ntp when misconfigured, or even smtp.

[sc302 Veteran]

being that you are in control of the switch you could put a per port acl and monitor bw on the switch port, so there is the capability if you so choose. 

[+BudMan MVC]

Agreed there are always multiple ways to skin the cat, once you know the breed of cat choosing the skinning method is much easier

[swappedsr]

This is actually all theoretical and I don't have actual clients I need this for.  I just wanted to make sure I understood everything as I have only ever needed to keep public ip's on the WAN side of my firewall.  You really made things clear here, thank you for that.  I have worked with Cisco ASA's and Pix firewalls in the past, I unfortunately inherited a Cisco ISA570, which will be replaced soon with enterprise level equipment.  I am still doing research on the equipment, and I have seen the open source pfsense and really want to try it.  Though, I really need something that will do IPS (Behavioral and signature based), SSL inspection, layer 7 application control, etc.  I haven't done too much research, but will be doing that soon enough.  

 

One of the reason I asked this question is I have a couple of users in a remote site and they essentially are tenants of a large complex.  I asked for some static ip's and then gave me a public ip address with a subnet mask of /29.  I immediately thought, they just gave me 30 usable addresses, what is stopping me from misconfiguring my ip address or someone else for that matter within that network?   I asked if there was anything in place to prevent this, but they never responded.  This is what ultimately led me to research an example of how the routing of public space should work behind perimeter equipment.  

 

I am assuming you are using BGP at both your sites with that /16 assignment?  I don't know much about BGP, other than it is exterior gateway protocol where you can advertise public blocks over different providers for redundancy.  How does the redundancy work when a site fails, assuming you have it set where it automatically advertises the subnet/s on the backup site during a site outage?  Are there any negatives using BGP?  I haven't read much into BGP and at some point will figure out if it is something that could be beneficial.  Right now we are load balancing along with failover for the two isp's we have.  

[+BudMan MVC]

  Quote

gave me a public ip address with a subnet mask of /29.  I immediately thought, they just gave me 30 usable addresses

 

/29 with 30 usable address??  You might want rethink that   /29 is only 6 host addresses.. Its fairly typical transit network when something like harp is used because you have the 3 address on each side of the connection.  Did you mean /27?

 

Yes using BGP, and there are way more than 2 sites ;)..   I would have to double check but I think we are only using that /16 in 4 sites currently.. Off the top I don't think we have anything using that space elsewhere - some of it is delegated to other customers, etc.

 

BGP is its own thing that is for sure.. It not all that complicated - but make sure you don't suck down the whole routing table for the internet   Easy way to crash your router, hehehe

 

[swappedsr]

Woops....Yes, I meant to put /27.  So I am a part of this /27 and I am sure I am not the only one in this network, that would be just silly on their part.  I am just not exactly sure what they could be doing to prevent someone on that network from misconfiguring their ip's, not sure they really can do anything, right?

 

As far as BPG, I will be definitely be researching it soon, seems like something we should look into.      

[+BudMan MVC]

I am in a /21 from my isp at home, you don't see these users messing up their IP..

 

What good wold it do - your just going to step on someone and not work anyway.  So how many IPs do you pay for - did they say you could use the whole /27 space?

[swappedsr]

They didn't say we can use the whole space, just gave us the ip and subnet.  I tried asking them if there were others on the network but never got an answer, I just never followed up with them either.  

 

I am not sure what you mean when you say what good would it do.  If you can prevent duplicate ip address this would prevent network issues.  For example,  If in this /27 we were given one ip address and another tenant in the same /27 given a different ip address.  What is preventing the other tenant from using our ip address and causing a duplicate ip address in this /27 network?  The last complex we were at before this one we were given a /30, 2 addresses, so they obviously did it right since nobody else could screw up our network as we were the only ones in this assigned subnet.  

[+BudMan MVC]

If user A is given 192.168.1.2/27

User B is given 192.168.1.3/27

User c is 192.168.1.4/27

etc..

 

why would user B use .4 or .5 or .6???

 

he was given .3, and wasn't given anything else - yes if he tries to use .4 he is going to mess up user C, but he is also not going to work very well either.  So what is the point of doing it was my point.

 

Even when you were given a /30 doesn't mean its any better than using a /27 -- keep in mine that if they subnet up their space into a bunch of /30 they loose a bunch of their address to wire and broadcast.  If they use larger subnets for customers in the same area they get more actual use out of their address space.

 

What if you want to buy some more addresses?  They no can say hey use .3-5 maybe on your same /27...

 

You only have 1 IP so what does it matter, do you have any routed networks to you, if not then this is not a transit network and just a network hanging off their router.  Have you tried pinging any of their ips?  Maybe they are open??  if you don't get a ping try using one

 

But its quite possible even if open and nobody using it, will not work.  Its quite possible they only have that 1 IP they gave you enabled to go anywhere, etc.

 

[swappedsr]

Yeah, I was just saying if another tenant fat fingered the address and ended up screwing up someone else.  

 

So yeah seems like a give and take.  If on a /27 with other tenants you get more usable addresses but potential for another tenant to fat finger address.  You can get around this by subnetting the larger network and avoid this problem, but you lose addresses to network and broadcast addresses.  

 

I just don't know if there was a way to keep everyone on the /27 so you don't lose addresses to network/broadcast but still prevent a tenant from fat fingering the wrong address on their interface.  Maybe you can through access lists on the switchport for each tenant?  For example, each tenant's switchport only allows the ip address they were assigned?  

 

 

[sc302 Veteran]

there is not, subnetting requires every subnet to have a network id and a broadcast number  (regardless if it is an internet facing/addressable subnet or a internal subnet).  in a /24, x.x.x.0 is your network id and your broadcast is x.x.x.255 so you have 1-254 as addresses that can be used by client pc's or other devices (routers/switches/firewalls/wap/refrigerators/toasters/etc).   even subnetting at an isp level you loose addresses but it allows them to control user access to other ip's....this is where dhcp plays a pivitol part in isp networks...they can give out addresses on a /24 or larger and not have to worry so much about losing addresses and they can make it so that it has to be a dhcp lease, someone can't go into the modem/router and assign whatever ip they want (they control the modem firmware or that section of the modem..it is off limits to the end user)

[+BudMan MVC]

unless your talking about the special use case /31 there is no wire or broadcast in that special case.

 

I agree many an ISP when handing out specific IP will do so via dhcp, so this is tied to a specific mac of the device in question.  So that customer always gets their specific IP... If they are going to change out the device then they would have to contact the isp with the new mac.

 

But when you create subnets you loose viable addresses, if you subnet a /24 down to /29 you loose 2 address per subnet, you can create 32 /29 out of 1 /24 so you have 32 * 6 for actual host addresses so you went from 254 hosts to 192 hosts.. A loss of 62 address that could of been used if you just used /24 vs creating all the little subnets.