![\"\"](\"https:\/\/blog.thomarite.uk\/wp-content\/uploads\/2020\/04\/Screenshot-from-2020-04-27-23-27-35-1024x973.png\")
<\/figure>\n\n\n\nThis is our scenario:<\/p>\n\n\n\n
- We have one customer vrf “CUST-A” with three locations: TY, LD and NY.<\/li>
- We are using BGP for PE-CE routing. Each site will use a different private ASN. Our SP is ASN 100.<\/li>
- TY has two connection to our SP so we want to make use of both of them.<\/li>
- We have a RR SP2 that is in line. So we need a full-mesh iBGP from all PE to SP2.<\/li>
- Our SP IGP is OSFP.<\/li>
- The goal is to make all other PE connected to CUST-A sites to be able to load-balance to TY site prefixes 192.168.11.0\/24 and 192.168.12.0\/24 using TY-SP1 and TY-SP3.<\/li><\/ul>\n\n\n\n
We start building the whole network as standard. This is very similar as stated in our first lab<\/a>:<\/p>\n\n\n\n
This is RR SP2 config:<\/p>\n\n\n\n
!\nip vrf CUST-A\n rd 100:1 \n route-target export 1:100\n route-target import 1:100\n!\ninterface Loopback0\n ip address 10.0.2.1 255.255.255.255\n! \ninterface GigabitEthernet1\/0\n description to SP1-PE\n ip address 10.0.12.2 255.255.255.0\n negotiation auto\n mpls ip\n!\ninterface GigabitEthernet2\/0\n description to SP3-PE\n ip address 10.0.23.2 255.255.255.0\n negotiation auto\n mpls ip\n!\ninterface FastEthernet3\/0\n description TO-LD-SP4\n ip address 10.0.24.2 255.255.255.0\n duplex auto\n speed auto\n mpls ip\n!\nrouter ospf 1\n log-adjacency-changes\n network 10.0.2.0 0.0.0.255 area 0\n network 10.0.12.0 0.0.0.255 area 0\n network 10.0.23.0 0.0.0.255 area 0\n network 10.0.24.0 0.0.0.255 area 0\n!\nrouter bgp 100\n no synchronization\n bgp log-neighbor-changes\n neighbor 10.0.1.1 remote-as 100\n neighbor 10.0.1.1 update-source Loopback0\n neighbor 10.0.1.1 route-reflector-client\n neighbor 10.0.3.1 remote-as 100\n neighbor 10.0.3.1 update-source Loopback0\n neighbor 10.0.3.1 route-reflector-client\n neighbor 10.0.4.1 remote-as 100\n neighbor 10.0.4.1 update-source Loopback0\n neighbor 10.0.4.1 route-reflector-client\n neighbor 10.0.5.1 remote-as 100\n neighbor 10.0.5.1 update-source Loopback0\n neighbor 10.0.5.1 route-reflector-client\n no auto-summary\n !\n address-family vpnv4\n neighbor 10.0.1.1 activate\n neighbor 10.0.1.1 send-community both\n neighbor 10.0.1.1 route-reflector-client\n neighbor 10.0.3.1 activate\n neighbor 10.0.3.1 send-community both\n neighbor 10.0.3.1 route-reflector-client\n neighbor 10.0.4.1 activate\n neighbor 10.0.4.1 send-community both\n neighbor 10.0.4.1 route-reflector-client\n neighbor 10.0.5.1 activate\n neighbor 10.0.5.1 send-community both\n neighbor 10.0.5.1 route-reflector-client\n exit-address-family\n !\n address-family ipv4 vrf CUST-A\n no synchronization\n exit-address-family\n!\n!\nmpls ldp router-id Loopback0 force\n<\/pre>\n\n\n\n
The configs for the SP PE follow the same patern, this is TY-SP1:<\/p>\n\n\n\n
!\nip vrf CUST-A\n rd 100:1 \n route-target export 1:100\n route-target import 1:100\n!\ninterface Loopback0\n ip address 10.0.1.1 255.255.255.255\n!\ninterface FastEthernet0\/0\n description to HQ\n ip vrf forwarding CUST-A\n ip address 172.16.100.254 255.255.255.0\n duplex half\n!\ninterface GigabitEthernet1\/0\n description to SP2-P\n ip address 10.0.12.1 255.255.255.0\n negotiation auto\n mpls ip\n!\nrouter ospf 1\n log-adjacency-changes\n network 10.0.1.0 0.0.0.255 area 0\n network 10.0.12.0 0.0.0.255 area 0\n!\nrouter bgp 100\n no synchronization\n bgp log-neighbor-changes\n neighbor 10.0.2.1 remote-as 100\n neighbor 10.0.2.1 update-source Loopback0\n no auto-summary\n !\n address-family vpnv4\n neighbor 10.0.2.1 activate\n neighbor 10.0.2.1 send-community both\n exit-address-family\n !\n address-family ipv4 vrf CUST-A\n neighbor 172.16.100.1 remote-as 65001\n neighbor 172.16.100.1 activate\n neighbor 172.16.100.1 soft-reconfiguration inbound\n no synchronization\n exit-address-family\n!\nmpls ldp router-id Loopback0 force\n!\n<\/pre>\n\n\n\n
Let’ see if LD-CE1 can ping our TY-C1<\/p>\n\n\n\n
LD-CE1#traceroute 192.168.12.1 source 172.16.30.1 \n\nType escape sequence to abort.\nTracing the route to 192.168.12.1\n\n 1 172.16.101.254 8 msec 20 msec 8 msec\n 2 10.0.24.2 [MPLS: Labels 18\/23 Exp 0] 40 msec 40 msec 36 msec\n 3 172.16.200.254 [MPLS: Label 23 Exp 0] 12 msec 32 msec 28 msec\n 4 172.16.200.1 60 msec 40 msec 40 msec\n 5 192.168.12.1 [AS 65001] 40 msec 60 msec 60 msec\nLD-CE1#\nLD-CE1#\nLD-CE1#ping 192.168.11.1 source 172.16.30.1 \n\nType escape sequence to abort.\nSending 5, 100-byte ICMP Echos to 192.168.11.1, timeout is 2 seconds:\nPacket sent with a source address of 172.16.30.1 \n!!!!!\nSuccess rate is 100 percent (5\/5), round-trip min\/avg\/max = 44\/54\/72 ms\nLD-CE1#\nLD-CE1#\nLD-CE1#\nLD-CE1#sh\nLD-CE1#show ip rou\nLD-CE1#show ip route \nCodes: C - connected, S - static, R - RIP, M - mobile, B - BGP\n D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area \n N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2\n E1 - OSPF external type 1, E2 - OSPF external type 2\n i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2\n ia - IS-IS inter area, * - candidate default, U - per-user static route\n o - ODR, P - periodic downloaded static route\n\nGateway of last resort is not set\n\nB 192.168.12.0\/24 [20\/0] via 172.16.101.254, 01:19:31\n 172.16.0.0\/16 is variably subnetted, 2 subnets, 2 masks\nC 172.16.30.1\/32 is directly connected, Loopback0\nC 172.16.101.0\/24 is directly connected, FastEthernet0\/0\nB 192.168.11.0\/24 [20\/0] via 172.16.101.254, 01:19:31\nLD-CE1#\n<\/pre>\n\n\n\n
So, what do we see when everything is configured?<\/p>\n\n\n\n
From SP2-RR, we see all BGP peers up to PEs and in the vpnv4 table we can see the TY prefixes 192.168.11.0\/24 and 192.168.12.0\/24. But only the path from TY-SP1 is preferred….<\/p>\n\n\n\n
SP2#show ip ospf neighbor \n\nNeighbor ID Pri State Dead Time Address Interface\n10.0.4.1 1 FULL\/DR 00:00:39 10.0.24.1 FastEthernet3\/0\n10.0.3.1 1 FULL\/DR 00:00:39 10.0.23.1 GigabitEthernet2\/0\n10.0.1.1 1 FULL\/BDR 00:00:37 10.0.12.1 GigabitEthernet1\/0\nSP2#\nSP2#\nSP2#show ip bgp summary \nBGP router identifier 10.0.2.1, local AS number 100\nBGP table version is 1, main routing table version 1\n\nNeighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up\/Down State\/PfxRcd\n10.0.1.1 4 100 98 111 1 0 0 01:25:16 0\n10.0.3.1 4 100 93 108 1 0 0 01:25:05 0\n10.0.4.1 4 100 96 114 1 0 0 00:55:06 0\n10.0.5.1 4 100 29 32 1 0 0 00:28:02 0\nSP2#\nSP2#show ip bgp vpnv4 all \nBGP table version is 9, local router ID is 10.0.2.1\nStatus codes: s suppressed, d damped, h history, * valid, > best, i - internal,\n r RIB-failure, S Stale\nOrigin codes: i - IGP, e - EGP, ? - incomplete\n\n Network Next Hop Metric LocPrf Weight Path\nRoute Distinguisher: 100:1 (default for vrf CUST-A)\n*>i172.16.30.1\/32 10.0.4.1 0 100 0 65002 i\n*>i192.168.11.0 10.0.1.1 0 100 0 65001 i\n* i 10.0.3.1 0 100 0 65001 i\n*>i192.168.12.0 10.0.1.1 0 100 0 65001 i\n* i 10.0.3.1 0 100 0 65001 i\nSP2#\n<\/pre>\n\n\n\n
Let confirm that the PE only receive the best prefix from the RR. So, from LD-SP4, we can see the paths to TY 192.168.11\/12 via TY-SP1 only:<\/p>\n\n\n\n
LD-SP4#show ip bgp vpnv4 all \nBGP table version is 18, local router ID is 10.0.4.1\nStatus codes: s suppressed, d damped, h history, * valid, > best, i - internal,\n r RIB-failure, S Stale\nOrigin codes: i - IGP, e - EGP, ? - incomplete\n\n Network Next Hop Metric LocPrf Weight Path\nRoute Distinguisher: 100:1 (default for vrf CUST-A)\n*> 172.16.30.1\/32 172.16.101.1 0 0 65002 i\n*>i192.168.11.0 10.0.1.1 0 100 0 65001 i\n*>i192.168.12.0 10.0.1.1 0 100 0 65001 i\nLD-SP4#\n<\/pre>\n\n\n\n
How do we make RR-SP2 to learn and advertise TY-SP1 and TY-SP3 paths. We need to use different RD in TY-SP1\/3 respectively.<\/p>\n\n\n\n
We have RD 100:1 assigned to CUST-A in all PEs. We are going to change that in TY-SP1\/3 so RR will see two different VPNv4 prefixes for the same destination.<\/p>\n\n\n\n
Let’s change TY-SP1 RD 100:1 to 100:101 and TY-SP3 to 100:102. Watch out as all routing config related to VRF CUST-A will disappear. <\/p>\n\n\n\n
And what about the RT config? Do we have to change anything? Actually, we need to keep it the same (we need to retype it), nothing changes here. Keep in mind that RT is used to import\/export vpnv4 prefixes into the VRF. The RD is not used to import\/export so for that reason (as we are going to see) we could actually use any RD for a VRF in a PE.<\/p>\n\n\n\n
Let’s see the changes for TY-SP1:<\/p>\n\n\n\n
TY-SP1(config)#ip vrf CUST-A\nTY-SP1(config-vrf)#no rd 100:1\n% \"rd 100:1\" for VRF CUST-A scheduled for deletion\nTY-SP1(config-vrf)#\n*Apr 27 22:28:48.347: %BGP-5-ADJCHANGE: neighbor 172.16.100.1 vpn vrf CUST-A Down Neighbor deleted\nTY-SP1(config-vrf)#rd 100:101\n% Deletion of \"rd\" in progress; wait for it to complete\nTY-SP1(config-vrf)#\nTY-SP1(config-vrf)#rd 100:101\nTY-SP1(config-vrf)#route-target export 100:1\nTY-SP1(config-vrf)#route-target import 100:1\nTY-SP1(config-vrf)#exit\nTY-SP1(config)#router bgp 100\nTY-SP1(config-router)#address-family ipv4 vrf CUST-A \nTY-SP1(config-router-af)# neighbor 172.16.100.1 remote-as 65001\nTY-SP1(config-router-af)# neighbor 172.16.100.1 activate\nTY-SP1(config-router-af)# neighbor 172.16.100.1 soft-reconfiguration inbound\nTY-SP1(config-router-af)#\n*Apr 27 22:33:50.571: %BGP-5-ADJCHANGE: neighbor 172.16.100.1 vpn vrf CUST-A Up \nTY-SP1(config-router-af)#\n<\/pre>\n\n\n\n
So after repeating the same step in TY-SP3 (using RD 100:102), let’s see what happens in RR-SP2:<\/p>\n\n\n\n
SP2#show ip bgp vpnv4 all \nBGP table version is 51, local router ID is 10.0.2.1\nStatus codes: s suppressed, d damped, h history, * valid, > best, i - internal,\n r RIB-failure, S Stale\nOrigin codes: i - IGP, e - EGP, ? - incomplete\n\n Network Next Hop Metric LocPrf Weight Path\nRoute Distinguisher: 100:1 (default for vrf CUST-A)\n*>i172.16.30.1\/32 10.0.4.1 0 100 0 65002 i\n* i192.168.11.0 10.0.3.1 0 100 0 65001 i\n*>i 10.0.1.1 0 100 0 65001 i\n* i192.168.12.0 10.0.3.1 0 100 0 65001 i\n*>i 10.0.1.1 0 100 0 65001 i\nRoute Distinguisher: 100:101\n*>i192.168.11.0 10.0.1.1 0 100 0 65001 i\n*>i192.168.12.0 10.0.1.1 0 100 0 65001 i\nRoute Distinguisher: 100:102\n*>i192.168.11.0 10.0.3.1 0 100 0 65001 i\n*>i192.168.12.0 10.0.3.1 0 100 0 65001 i\nSP2#\n<\/pre>\n\n\n\n
Now we can see VPNv4 for 100:101 (TY-SP1) and 100:102 (TY-SP2)!!!<\/p>\n\n\n\n
Ok, let’s what the other PE are seeing. In our case, let’s check LD-SP4:<\/p>\n\n\n\n
LD-SP4#show ip bgp vpnv4 all \nBGP table version is 18, local router ID is 10.0.4.1\nStatus codes: s suppressed, d damped, h history, * valid, > best, i - internal,\n r RIB-failure, S Stale\nOrigin codes: i - IGP, e - EGP, ? - incomplete\n\n Network Next Hop Metric LocPrf Weight Path\nRoute Distinguisher: 100:1 (default for vrf CUST-A)\n*> 172.16.30.1\/32 172.16.101.1 0 0 65002 i\n* i192.168.11.0 10.0.3.1 0 100 0 65001 i\n*>i 10.0.1.1 0 100 0 65001 i\n* i192.168.12.0 10.0.3.1 0 100 0 65001 i\n*>i 10.0.1.1 0 100 0 65001 i\nRoute Distinguisher: 100:101\n*>i192.168.11.0 10.0.1.1 0 100 0 65001 i\n*>i192.168.12.0 10.0.1.1 0 100 0 65001 i\nRoute Distinguisher: 100:102\n*>i192.168.11.0 10.0.3.1 0 100 0 65001 i\n*>i192.168.12.0 10.0.3.1 0 100 0 65001 i\nLD-SP4#\nLD-SP4#\nLD-SP4#show ip route vrf CUST-A\n\nRouting Table: CUST-A\nCodes: C - connected, S - static, R - RIP, M - mobile, B - BGP\n D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area \n N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2\n E1 - OSPF external type 1, E2 - OSPF external type 2\n i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2\n ia - IS-IS inter area, * - candidate default, U - per-user static route\n o - ODR, P - periodic downloaded static route\n\nGateway of last resort is not set\n\nB 192.168.12.0\/24 [200\/0] via 10.0.1.1, 00:01:05\n 172.16.0.0\/16 is variably subnetted, 2 subnets, 2 masks\nB 172.16.30.1\/32 [20\/0] via 172.16.101.1, 00:33:47\nC 172.16.101.0\/24 is directly connected, FastEthernet0\/0\nB 192.168.11.0\/24 [200\/0] via 10.0.1.1, 00:01:05\nLD-SP4#\n<\/pre>\n\n\n\n
So, LD-SP4 is receiving the VPNv4 100:101 and 100:102 from RR-SP2!!! That’s good, but we are still seeing the path to TY 192.168.11\/12 prefixes via TY-SP1 (10.0.1.1) only.<\/p>\n\n\n\n
So why BGP ECMP is not working? Because we have to enable it.<\/p>\n\n\n\n
LD-SP4(config)#router bgp 100\nLD-SP4(config-router)#address-family ipv4 vrf CUST-A\nLD-SP4(config-router-af)#maximum-paths eibgp 2\nLD-SP4(config-router-af)#\n*Apr 27 22:58:25.447: BGP: VPNv4 Unicast multipath configuration changed\n*Apr 27 22:58:25.447: BGP-VPN(4): MPLS label changed for prefix 100:1:192.168.11.0\/24\n*Apr 27 22:58:25.447: BGP-VPN(4): multipath from neighbor 10.0.2.1 nexthop 10.0.3.1 new outlabel 24\n*Apr 27 22:58:25.447: vpn: free local label 1048577 for remote prefix CUST-A:192.168.11.0\/24\n*Apr 27 22:58:25.447: vpn: get path labels: 100:1:192.168.11.0\/255.255.255.0\n*Apr 27 22:58:25.451: vpn(4): inlabel=nolabel, outlabel=22, outlabel owner=BGP\n*Apr 27 22:58:25.451: vpn(4): Announce labels to IPRM CUST-A:192.168.11.0\/24 gw 10.0.1.1 inlabel=nolabel, outlabel=22\n*Apr 27 22:58:25.451: BGP-VPN(4): MPLS label changed for prefix 100:1:192.168.12.0\/24\n*Apr 27 22:58:25.451: BGP-VPN(4): multipath from neighbor 10.0.2.1 nexthop 10.0.3.1 new outlabel 23\n*Apr 27 22:58:25.451: vpn: free local label 1048577 for remote prefix CUST-A:192.168.12.0\/24\n*Apr 27 22:58:25.451: vpn: get path labels: 100:1:192.168.12.0\/255.255.255.0\n*\nLD-SP4(config-router-af)#endApr 27 22:58:25.451: vpn(4): inlabel=nolabel, outlabel=21, outlabel owner=BGP\n*Apr 27 22:58:25.451: vpn(4): Announce labels to IPRM CUST-A:192.168.12.0\/24 gw 10.0.1.1 inlabel=nolabel, outlabel=21\n*Apr 27 22:58:25.455: vpn: get path labels: 100:1:192.168.11.0\/255.255.255.0\n*Apr 27 22:58:25.459: vpn(4): inlabel=nolabel, outlabel=24, outlabel owner=BGP\n*Apr 27 22:58:25.459: vpn(4): Announce labels to IPRM CUST-A:192.168.11.0\/24 gw 10.0.3.1 inlabel=nolabel, outlabel=24\n*Apr 27 22:58:25.459: vpn(4): get path labels; 100:1:192.168.11.0\/24 nexthop 10.0.3.1, not bestpath\n*Apr 27 22:58:25.475: vpn: get path labels: 100:1:192.168.12.0\/255.255.255.0\n*Apr 27 22:58:25.475: vpn(4): inlabel=nolabel, outlabel=23, outlabel owner=BGP\n*Apr 27 22:58:25.475: vpn(4): Announce labels to IPRM CUST-A:192.168.12.0\/24 gw 10.0.3.1 inlabel=nolabel, outlabel=23\n*Apr 27 22:58:25.479: vpn(4): get path labels; 100:1:192.168.12.0\/24 nexthop 10.0.3.1, not bestpath\nLD-SP4(config-router-af)#end\nLD-SP4#\n*Apr 27 22:58:27.411: %SYS-5-CONFIG_I: Configured from console by console\nLD-SP4#\nLD-SP4#\nLD-SP4#show ip route vrf CUST-A\n\nRouting Table: CUST-A\nCodes: C - connected, S - static, R - RIP, M - mobile, B - BGP\n D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area \n N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2\n E1 - OSPF external type 1, E2 - OSPF external type 2\n i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2\n ia - IS-IS inter area, * - candidate default, U - per-user static route\n o - ODR, P - periodic downloaded static route\n\nGateway of last resort is not set\n\nB 192.168.12.0\/24 [200\/0] via 10.0.3.1, 00:00:07\n [200\/0] via 10.0.1.1, 00:02:18\n 172.16.0.0\/16 is variably subnetted, 2 subnets, 2 masks\nB 172.16.30.1\/32 [20\/0] via 172.16.101.1, 00:35:00\nC 172.16.101.0\/24 is directly connected, FastEthernet0\/0\nB 192.168.11.0\/24 [200\/0] via 10.0.3.1, 00:00:07\n [200\/0] via 10.0.1.1, 00:02:18\nLD-SP4#\n<\/pre>\n\n\n\n
We finally got it! Our PE LD-SP4 is able to see two paths to TY prefixes!<\/p>\n\n\n\n
In summary:<\/p>\n\n\n\n
- We need to change the VRF RD in the PE we want to be participant in load-balancing<\/li>
- We need to enable EIBGP ECMP <\/li><\/ul>\n","protected":false},"excerpt":{"rendered":"
I read once about how to do load-balancing when using Route-Reflectors (RR) in a MPLS L3VPN network. It is a insteresting topic because RRs only reflect the best prefixes to the its clients. So how we make the RR to send more than one? So I built a GNS3 lab to work on this subject: … Continue reading “GNS3: Load-Balancing with Route Reflectors in a MPLS L3VPN network”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[],"class_list":["post-100","post","type-post","status-publish","format-standard","hentry","category-networks"],"_links":{"self":[{"href":"https:\/\/blog.thomarite.uk\/index.php\/wp-json\/wp\/v2\/posts\/100","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.thomarite.uk\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.thomarite.uk\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.thomarite.uk\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.thomarite.uk\/index.php\/wp-json\/wp\/v2\/comments?post=100"}],"version-history":[{"count":3,"href":"https:\/\/blog.thomarite.uk\/index.php\/wp-json\/wp\/v2\/posts\/100\/revisions"}],"predecessor-version":[{"id":132,"href":"https:\/\/blog.thomarite.uk\/index.php\/wp-json\/wp\/v2\/posts\/100\/revisions\/132"}],"wp:attachment":[{"href":"https:\/\/blog.thomarite.uk\/index.php\/wp-json\/wp\/v2\/media?parent=100"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.thomarite.uk\/index.php\/wp-json\/wp\/v2\/categories?post=100"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.thomarite.uk\/index.php\/wp-json\/wp\/v2\/tags?post=100"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}