Disclaimer: It is a highly recommended practice to employ a system of peer review for any changes you make that effect data plane traffic. This practice is especially important on systems managed via CLI. CLI is not always consistent between software versions or device types. Reviewing documentation and getting a second set of eyes always helps. CLI configuration had been the de facto method for configuring network equipment up until a few years ago. The only way to do it accurately and consistently is to ensure multiple experienced engineers sign off on the candidate configurations. Without peer review configurations tend to be riddled with typos, artifacts from cut-and-paste, and inconsistent conventions. Always have someone check your work.
In this blog post I will review how to implement dual hub Cisco Meraki MX’s into an existing Cisco infrastructure that is running EIGRP as the dynamic routing protocol.
As of June 2018 – MXs allow for OSPF peering when in vpn concentrator mode or in NAT mode with a single VLAN(there is also a beta for BGP but that is for another day). This OSPF peering however only does route injection and does not learn routes. This is to allow for upstream/downstream devices to be aware of VPN peer subnets and also to allow for us to have redundant dynamic routes to VPN peers in the scenario where we have dual hub MXs in a hub and spoke topology.
For a breakdown in understanding MX 1-arm concentrator mode, please refer to the following document:
VPN Concentrator Deployment Guide
and
Now for the fun stuff.
Both of these guides outline in pretty decent detail how to deploy the technologies from a meraki perspective. The one bit of data lacking is integration into existing topologies with EIGRP.
Summary of the problem
In the graphic below, we have 2x MXs that are operating as Hubs in VPN concentrator mode. They are both peering to an upstream L3 routing appliance via OSPF. The L3 appliances are then using EIGRP for dynamic routing within the organization.
The spoke MX is configured to connect to DC1 as the primary VPN path, and DC2 as the secondary VPN path. If we were to just redistribute the 10.10.10.0/24 subnet into EIGRP both MXs would show a more or less equal cost to the destination network of 10.10.10.0/24. This could potentially cause asynchronous routing if the spoke MX sent traffic to DC1 for a service upstream, and then the upstream router chose to send the return traffic to DC2. This is not the desired behavior typically as that means the return path may be less desirable (higher latency, loss, etc). When you add dual hubs to a spoke MX, there is no way on the hubs to prune the routes to the spoke as the hubs will always advertise all spoke networks connected (unless the spoke VPN is down).
To fix this potential problem we need to deploy some configurations to make sure that a spoke’s traffic that goes to a primary hub, returns on the same path.
Diagram
As you can see in the diagram the configuration is not terribly complex however we should break down each piece.
EIGRP
Without going into an entire blog post of how EIGRP works, remember that EIGRP does not use a simple cost setting like you would use in OSPF to weight a route. Instead EIGRP uses {Bandwidth}{Delay}{Reliability}{Load}{MTU}. We will be taking advantage of modifying the {delay} attribute when redistributing to make one set of redistributed routes appear more desirable than others.
A little light reading: Introduction to EIGRP
Prefix Lists
When redistributing routes you can take a number of different approaches. My least favorite is crossing your fingers and just redistributing the entire protocol. This can be useful in some circumstances but can cause more harm than good if you are not 100% on what routes you could possibly be injecting. That is why I recommend using prefix lists to more or less create filters for your redistribution statements. There is a potential if you redistribute the entire protocol that you could cause asynchronous routing as well which we want to avoid as we want the traffic to take the same return path. An example of a prefix list would be:
“ip prefix-list {Name} seq 10 permit {subnet in CIDR notation}”
E.g. “ip prefix-list PRIMARY seq 10 permit 10.10.10.0/24”
As you can see here we can use the sequence {XX} to place a statement before or after. I recommending if your list is not going to be too large to skip a few in between. E.g. Seq 10, seq 20, seq 30. This can be useful when you are building large prefix-lists and need to slide a prefix in between two others.
You can also use the suffix of “le XX” or “ge XX” as greater than or less than the prefix listed. This can be useful if you are trying to match a number of smaller prefixes.
Example:“ip prefix-list PRIMARY seq 20 permit 10.0.0.0/8 ge 16 le 30”
This prefix list would match any prefixes in the 10.0.0.0/8 that are greater than 16 bits and less than 30 bits. So any prefix that is 10.X.X.X/16 – 10.X.X.X/30 would be matched. However 10.X.X.X/31 would not be matched and 10.X.X.X/12 would not be matched due to being too small and too large respectively.
How could we use prefix lists though?
In this case we want to make the DC1 advertisement look much better than DC2, and only for the spoke MXs that are using DC1 as their primary. To accomplish this we would use prefix-lists to only match those spoke sites, and set a metric that is desirable on DC1 and less desirable on DC2.
This configuration on the DC1 IOS L3 appliance would be:
“ip prefix-list PRIMARY seq 10 permit 10.10.10.0/24”
On the DC2 IOS L3 appliance it would be:
“ip prefix-list SECONDARY seq 10 permit 10.10.10.0/24”
For more on prefix lists please read the following great blog article: PacketLife – Understanding IP Prefix Lists
Route Maps
Now to inject the prefix-lists we created, we need to utilize route maps. Route Maps are extremely versatile in function and can perform anything from route filtering to policy based routing and beyond. In this case we are just going to use route maps for route matching and injection.
On DC1 the route map would be:
Route-map HUB-Primary permit 10
Match IP address prefix-list PRIMARY
On DC2 the route map would be:
Route-map HUB-Secondary permit 10
Match IP address prefix-list SECONDARY
Route Map light reading: Route-Maps for IP Routing Protocol Redistribution Configuration
OSPF Configuration
One thing I always try to do is avoid making changes to a routing process when I have a multi-step configuration I am working on. This is in part to preserve the existing routing table and to make sure we do not make any config changes that would be potentially outage causing. To get OSPF up and running there are a number of configs to keep in mind. In this case the OSPF config is contained between the L3 first hop and the MX so it is not as paramount to add in all the little configuration tidbits that you would in a larger scale OSPF deployment. That being said if you are a big OSPF fan and want to build out your configuration with router-ids and other fun things have at it. In my example I will not be tuning the OSPF config.
DC1 & 2 OSPF configuration:
Router OSPF 10
Network X.X.X.X X.X.X.X area 0 #where the Xs represent the P2P network and wildcard mask between your MX and L3
E.G. “network 10.255.255.0 0.0.0.3 area 0”
Passive-interface default # Don’t advertise OSPF on any interface
No passive-interface {interface name} #ok advertise on this interface
E.g. “no passive-interface gigabitethernet 1/0/48”
Redistribution into EIGRP
Now what we all have been waiting for…. Let’s get some routes into EIGRP!
To redistribute the routes from our VPN topology into EIGRP we are going to tie our previous configurations together in 1 nice lengthy statement.
For DC1:
redistribute ospf {process} route-map {route-map we created} metric {bandwidth | delay | reliability | load | MTU}
E.G. “redistribute ospf 10 route-map HUB-Primary metric 10000 10 255 5 1500”
For DC2:
E.G. “redistribute ospf 10 route-map HUB-Secondary metric 10000 1000 255 5 1500″
Notice in the above example the underlined delay value is highly increased from the primary hub redistribution. This will make it more or less a backup route for access to the VPN subnet.
What if I have different spokes using DC2 as primary and DC1 as secondary?
In a lot of situations you have some spokes terminating on DC1 or DC2 as their primary hub. Which is to be expected as deployments grow and bandwidth isn’t always easily upgraded. To deploy we can take advantage of what we have already built, and add a little more config. So in the event that we have a spoke site with 10.10.10.0/24 terminating on DC1 as a primary, and 10.10.20.0/24 terminating on DC2 as primary, we would do the following:
For DC1:
ip prefix-list PRIMARY seq 10 permit 10.10.10.0/24
ip prefix-list SECONDARY seq 10 permit 10.10.20.0/24
Route-map HUB-Primary permit 10
Match IP address prefix-list PRIMARY
Route-map HUB-Secondary permit 10
Match IP address prefix-list SECONDARY
router ospf 10
redistribute ospf 10 route-map HUB-Primary metric 10000 10 255 5 1500
redistribute ospf 10 route-map HUB-Secondary metric 10000 1000 255 5 1500
For DC2:
ip prefix-list PRIMARY seq 10 permit 10.10.20.0/24
ip prefix-list SECONDARY seq 10 permit 10.10.10.0/24
Route-map HUB-Primary permit 10
Match IP address prefix-list PRIMARY
Route-map HUB-Secondary permit 10
Match IP address prefix-list SECONDARY
router ospf 10
redistribute ospf 10 route-map HUB-Primary metric 10000 10 255 5 1500
redistribute ospf 10 route-map HUB-Secondary metric 10000 1000 255 5 1500
In the above examples, what we did is make sure the primary spokes have priority from the primary hubs, and the secondary spokes are heavily weighted to be backup routes in the event that their primary hub goes down.
In Closing
This entire post came about after a number of situations I have had with customers needing redistribution and not having a clear path on how to do so. I included some links throughout the document that I urge you to read as you are configuring either in a lab or production. If you find any errors in my configurations or recommendations please let me know either in the comments or via DM on twitter/linkedin. Thank you for reading and I hope this was informative for you!
