This is the diagram for the lab:<\/p>\n\n\n\n <\/p>\n\n\n\n Difference from lab3 and lab2. We have P1, that is a pure P router, only handling labels, it doesnt do any BGP.<\/p>\n\n\n\n This time all devices FRR config are generated automatically via gen_frr_config.py<\/a> (in lab2 all config was manual).<\/p>\n\n\n\n Again the environment is configured via Vagrant file + l3vpn_provisioning<\/a> script. This is mix of lab2 (install FRR), lab3 (define VRFs) and lab1 (configure MPLS at linux level).<\/p>\n\n\n\n So after some tuning, everything is installed, routing looks correct (although<\/strong> I dont know why but I have to reload FRR to get the proper generated BGP config in PE1 and PE2. P1 is fine).<\/p>\n\n\n\n So let’s see PE1:<\/p>\n\n\n\n IGP (IS-IS) is up:<\/p>\n\n\n\n BGP is up to PE2 and we can see routes received in AF IPv4VPN:<\/p>\n\n\n\n Check routing tables, we can see prefixes in both VRFs, so that’s good. And the labels needed.<\/p>\n\n\n\n Now check LDP and MPLS labels. Everything looks sane. We have LDP labels for P1 (17) and PE2 (18). And labels for each VFR. <\/p>\n\n\n\n Similar view happens in PE2.<\/p>\n\n\n\n From P1 that is our P router. We only care about LDP and ISIS<\/p>\n\n\n\n So as usual, let’s try to test connectivity. Will ping from CE1 (connected to PE1) to CE3 (connected to PE2) that belong to the same VRF vrf_cust1.<\/p>\n\n\n\n First of all, I had to modify iptables in my host to avoid unnecessary NAT (iptables masquerade) between CE1 and CE3.<\/p>\n\n\n\n Ok, staring pinging from CE1 to CE3:<\/p>\n\n\n\n No good. Let’s check what the next hop, PE1, is doing. It seem it is sending the traffic double encapsulated to P1 as expected <\/p>\n\n\n\n Let’s check next hop, P1. I can see it is sending the traffic to PE2 doing PHP, so removing the top label (LDP) and only leaving the BGP label:<\/p>\n\n\n\n But then PE2 is not sending anything to CE3. I can’t see anything in the links:<\/p>\n\n\n\n I have double-checked the configs. All routing and config looks sane in PE2:<\/p>\n\n\n\n So I am a bit puzzled the last couple of weeks about this issue. I was thinking that iptables was fooling me again and was dropping the traffic somehow but as far as I can see. PE2 is not sending anything and I dont really know how to troubleshoot FRR in this case. I have asked for help in the FRR list. Let’s see how it goes. I think I am doing something wrong because I am not doing anything new.<\/p>\n","protected":false},"excerpt":{"rendered":" Finally I am trying to setup MPLS L3VPN. Again, I am following the\u00a0author post\u00a0but adapting it to my environment using libvirt instead of VirtualBox and Debian10 as VM. All my data is here. This is the diagram for the lab: Difference from lab3 and lab2. We have P1, that is a pure P router, only … Continue reading “Linux+MPLS-Part4”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[],"class_list":["post-617","post","type-post","status-publish","format-standard","hentry","category-networks"],"_links":{"self":[{"href":"https:\/\/blog.thomarite.uk\/index.php\/wp-json\/wp\/v2\/posts\/617","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.thomarite.uk\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.thomarite.uk\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.thomarite.uk\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.thomarite.uk\/index.php\/wp-json\/wp\/v2\/comments?post=617"}],"version-history":[{"count":3,"href":"https:\/\/blog.thomarite.uk\/index.php\/wp-json\/wp\/v2\/posts\/617\/revisions"}],"predecessor-version":[{"id":634,"href":"https:\/\/blog.thomarite.uk\/index.php\/wp-json\/wp\/v2\/posts\/617\/revisions\/634"}],"wp:attachment":[{"href":"https:\/\/blog.thomarite.uk\/index.php\/wp-json\/wp\/v2\/media?parent=617"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.thomarite.uk\/index.php\/wp-json\/wp\/v2\/categories?post=617"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.thomarite.uk\/index.php\/wp-json\/wp\/v2\/tags?post=617"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}![\"\"](\"https:\/\/blog.thomarite.uk\/wp-content\/uploads\/2021\/02\/linux-mpls-lab4-v0-1024x709.png\")
<\/figure>\n\n\n\nPE1# show isis neighbor \n Area ISIS:\n System Id Interface L State Holdtime SNPA\n P1 ens8 2 Up 30 2020.2020.2020\n PE1# \n PE1# exit\n root@PE1:\/home\/vagrant# <\/pre>\n\n\n\n
PE1# \n PE1# show bgp summary \n IPv4 Unicast Summary:\n BGP router identifier 172.20.5.1, local AS number 65010 vrf-id 0\n BGP table version 0\n RIB entries 0, using 0 bytes of memory\n Peers 1, using 21 KiB of memory\n Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up\/Down State\/PfxRcd PfxSnt\n 172.20.5.2 4 65010 111 105 0 0 0 01:39:14 0 0\n Total number of neighbors 1\n IPv4 VPN Summary<\/strong>:\n BGP router identifier 172.20.5.1, local AS number 65010 vrf-id 0\n BGP table version 0\n RIB entries 11, using 2112 bytes of memory\n Peers 1, using 21 KiB of memory\n Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up\/Down State\/PfxRcd<\/strong> PfxSnt\n 172.20.5.2<\/strong> 4 65010 111 105 0 0 0 01:39:14 2<\/strong> 2\n Total number of neighbors 1\n PE1# <\/pre>\n\n\n\n
PE1# show ip route vrf all \n Codes: K - kernel route, C - connected, S - static, R - RIP,\n O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,\n T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,\n F - PBR, f - OpenFabric,\n > - selected route, * - FIB route, q - queued, r - rejected, b - backup\n VRF default:\n C>* 172.20.5.1\/32 is directly connected, lo, 02:19:16\n I>* 172.20.5.2\/32 [115\/30] via 192.168.66.102, ens8, label 17, weight 1, 02:16:10\n I>* 172.20.5.5\/32 [115\/20] via 192.168.66.102, ens8, label implicit-null, weight 1, 02:18:34\n I 192.168.66.0\/24 [115\/20] via 192.168.66.102, ens8 inactive, weight 1, 02:18:34\n C>* 192.168.66.0\/24 is directly connected, ens8, 02:19:16\n I>* 192.168.77.0\/24 [115\/20] via 192.168.66.102, ens8, label implicit-null, weight 1, 02:18:34\n C>* 192.168.121.0\/24 is directly connected, ens5, 02:19:16\n K>* 192.168.121.1\/32 [0\/1024] is directly connected, ens5, 02:19:16\n VRF vrf_cust1<\/strong>:\n C>* 192.168.11.0\/24 is directly connected, ens6, 02:19:05\n B> 192.168.23.0\/24 [200\/0] via 172.20.5.2 (vrf default) (recursive), label 80<\/strong>, weight 1, 02:13:32\n via 192.168.66.102, ens8 (vrf default), label 17\/80<\/strong>, weight 1, 02:13:32 \n VRF vrf_cust2<\/strong>:\n C>* 192.168.12.0\/24 is directly connected, ens7, 02:19:05\n B> 192.168.24.0\/24 [200\/0] via 172.20.5.2 (vrf default) (recursive), label 81<\/strong>, weight 1, 02:13:32\n via 192.168.66.102, ens8 (vrf default), label 17\/81<\/strong>, weight 1, 02:13:32\n PE1# <\/pre>\n\n\n\n
PE1# show mpls table \n Inbound Label Type Nexthop Outbound Label \n \n 16 LDP 192.168.66.102 implicit-null \n 17<\/strong> LDP 192.168.66.102 implicit-null \n 18<\/strong> LDP 192.168.66.102 17 \n 80<\/strong> BGP<\/strong> vrf_cust1<\/strong> - \n 81<\/strong> BGP<\/strong> vrf_cust2<\/strong> - \n PE1# \n PE1# show mpls ldp neighbor \n AF ID State Remote Address Uptime\n ipv4 172.20.5.5 OPERATIONAL 172.20.5.5 02:20:20\n PE1# \n PE1# \n PE1# show mpls ldp binding \n AF Destination Nexthop Local Label Remote Label In Use\n ipv4 172.20.5.1\/32 172.20.5.5 imp-null 16 no\n ipv4 172.20.5.2\/32 172.20.5.5 18 17 yes<\/strong>\n ipv4 172.20.5.5\/32 172.20.5.5 16 imp-null yes<\/strong>\n ipv4 192.168.11.0\/24 0.0.0.0 imp-null - no\n ipv4 192.168.12.0\/24 0.0.0.0 imp-null - no\n ipv4 192.168.66.0\/24 172.20.5.5 imp-null imp-null no\n ipv4 192.168.77.0\/24 172.20.5.5 17 imp-null yes<\/strong>\n ipv4 192.168.121.0\/24 172.20.5.5 imp-null imp-null no\n PE1# <\/pre>\n\n\n\n
P1# \n P1# show mpls table<\/strong> \n Inbound Label Type Nexthop Outbound Label \n \n 16 LDP 192.168.66.101 implicit-null \n 17 LDP 192.168.77.101 implicit-null \n P1# show mpls ldp neighbor \n AF ID State Remote Address Uptime\n ipv4 172.20.5.1 OPERATIONAL 172.20.5.1 02:23:55\n ipv4 172.20.5.2 OPERATIONAL 172.20.5.2 02:21:01\n P1# \n P1# show isis neighbor<\/strong> \n Area ISIS:\n System Id Interface L State Holdtime SNPA\n PE1 ens6 2 Up 28 2020.2020.2020\n PE2 ens7 2 Up 29 2020.2020.2020\n P1# \n P1# show ip route<\/strong>\n Codes: K - kernel route, C - connected, S - static, R - RIP,\n O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,\n T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,\n F - PBR, f - OpenFabric,\n > - selected route, * - FIB route, q - queued, r - rejected, b - backup\n K>* 0.0.0.0\/0 [0\/1024] via 192.168.121.1, ens5, src 192.168.121.253, 02:24:45\n I>* 172.20.5.1\/32 [115\/20] via 192.168.66.101, ens6, label implicit-null, weight 1, 02:24:04\n I>* 172.20.5.2\/32 [115\/20] via 192.168.77.101, ens7, label implicit-null, weight 1, 02:21:39\n C>* 172.20.5.5\/32 is directly connected, lo, 02:24:45\n I 192.168.66.0\/24 [115\/20] via 192.168.66.101, ens6 inactive, weight 1, 02:24:04\n C>* 192.168.66.0\/24 is directly connected, ens6, 02:24:45\n I 192.168.77.0\/24 [115\/20] via 192.168.77.101, ens7 inactive, weight 1, 02:21:39\n C>* 192.168.77.0\/24 is directly connected, ens7, 02:24:45\n C>* 192.168.121.0\/24 is directly connected, ens5, 02:24:45\n K>* 192.168.121.1\/32 [0\/1024] is directly connected, ens5, 02:24:45\n P1# <\/pre>\n\n\n\n
# iptables -t nat -vnL LIBVIRT_PRT --line-numbers\n Chain LIBVIRT_PRT (1 references)\n num pkts bytes target prot opt in out source destination \n 1 15 1451 RETURN all -- * * 192.168.77.0\/24 224.0.0.0\/24 \n 2 0 0 RETURN all -- * * 192.168.77.0\/24 255.255.255.255 \n 3 0 0 MASQUERADE tcp -- * * 192.168.77.0\/24 !192.168.77.0\/24 masq ports: 1024-65535\n 4 18 3476 MASQUERADE udp -- * * 192.168.77.0\/24 !192.168.77.0\/24 masq ports: 1024-65535\n 5 0 0 MASQUERADE all -- * * 192.168.77.0\/24 !192.168.77.0\/24 \n 6 13 1754 RETURN all -- * * 192.168.122.0\/24 224.0.0.0\/24 \n 7 0 0 RETURN all -- * * 192.168.122.0\/24 255.255.255.255 \n 8 0 0 MASQUERADE tcp -- * * 192.168.122.0\/24 !192.168.122.0\/24 masq ports: 1024-65535\n 9 0 0 MASQUERADE udp -- * * 192.168.122.0\/24 !192.168.122.0\/24 masq ports: 1024-65535\n 10 0 0 MASQUERADE all -- * * 192.168.122.0\/24 !192.168.122.0\/24 \n 11 24 2301 RETURN all -- * * 192.168.11.0\/24 224.0.0.0\/24 \n 12 0 0 RETURN all -- * * 192.168.11.0\/24 255.255.255.255 \n 13 0 0 MASQUERADE tcp -- * * 192.168.11.0\/24 !192.168.11.0\/24 masq ports: 1024-65535\n 14 23 4476 MASQUERADE udp -- * * 192.168.11.0\/24 !192.168.11.0\/24 masq ports: 1024-65535\n 15 1 84 MASQUERADE all -- * * 192.168.11.0\/24 !192.168.11.0\/24 <\/strong>\n 16 29 2541 RETURN all -- * * 192.168.121.0\/24 224.0.0.0\/24 \n 17 0 0 RETURN all -- * * 192.168.121.0\/24 255.255.255.255 \n 18 36 2160 MASQUERADE tcp -- * * 192.168.121.0\/24 !192.168.121.0\/24 masq ports: 1024-65535\n 19 65 7792 MASQUERADE udp -- * * 192.168.121.0\/24 !192.168.121.0\/24 masq ports: 1024-65535\n 20 0 0 MASQUERADE all -- * * 192.168.121.0\/24 !192.168.121.0\/24 \n 21 20 2119 RETURN all -- * * 192.168.24.0\/24 224.0.0.0\/24 \n 22 0 0 RETURN all -- * * 192.168.24.0\/24 255.255.255.255 \n 23 0 0 MASQUERADE tcp -- * * 192.168.24.0\/24 !192.168.24.0\/24 masq ports: 1024-65535\n 24 21 4076 MASQUERADE udp -- * * 192.168.24.0\/24 !192.168.24.0\/24 masq ports: 1024-65535\n 25 0 0 MASQUERADE all -- * * 192.168.24.0\/24 !192.168.24.0\/24 \n 26 20 2119 RETURN all -- * * 192.168.23.0\/24 224.0.0.0\/24 \n 27 0 0 RETURN all -- * * 192.168.23.0\/24 255.255.255.255 \n 28 1 60 MASQUERADE tcp -- * * 192.168.23.0\/24 !192.168.23.0\/24 masq ports: 1024-65535\n 29 20 3876 MASQUERADE udp -- * * 192.168.23.0\/24 !192.168.23.0\/24 masq ports: 1024-65535\n 30 1 84 MASQUERADE all -- * * 192.168.23.0\/24 !192.168.23.0\/24 <\/strong>\n 31 25 2389 RETURN all -- * * 192.168.66.0\/24 224.0.0.0\/24 \n 32 0 0 RETURN all -- * * 192.168.66.0\/24 255.255.255.255 \n 33 0 0 MASQUERADE tcp -- * * 192.168.66.0\/24 !192.168.66.0\/24 masq ports: 1024-65535\n 34 23 4476 MASQUERADE udp -- * * 192.168.66.0\/24 !192.168.66.0\/24 masq ports: 1024-65535\n 35 0 0 MASQUERADE all -- * * 192.168.66.0\/24 !192.168.66.0\/24 \n 36 24 2298 RETURN all -- * * 192.168.12.0\/24 224.0.0.0\/24 \n 37 0 0 RETURN all -- * * 192.168.12.0\/24 255.255.255.255 \n 38 0 0 MASQUERADE tcp -- * * 192.168.12.0\/24 !192.168.12.0\/24 masq ports: 1024-65535\n 39 23 4476 MASQUERADE udp -- * * 192.168.12.0\/24 !192.168.12.0\/24 masq ports: 1024-65535\n 40 0 0 MASQUERADE all -- * * 192.168.12.0\/24 !192.168.12.0\/24 \n#\n\n\n# iptables -t nat -I LIBVIRT_PRT 13 -s 192.168.11.0\/24 -d 192.168.23.0\/24 -j RETURN\n# iptables -t nat -I LIBVIRT_PRT 29 -s 192.168.23.0\/24 -d 192.168.11.0\/24 -j RETURN<\/pre>\n\n\n\n
vagrant@CE1:~$ ping 192.168.23.102\n PING 192.168.23.102 (192.168.23.102) 56(84) bytes of data.<\/pre>\n\n\n\n
root@PE1:\/home\/vagrant# tcpdump -i ens8\n...\n20:29:16.648325 MPLS (label 17<\/strong>, exp 0, ttl 63) (label 80<\/strong>, exp 0, [S], ttl 63) IP 192.168.11.102 > 192.168.23.102: ICMP echo request, id 2298, seq 2627, length 64\n20:29:17.672287 MPLS (label 17<\/strong>, exp 0, ttl 63) (label 80<\/strong>, exp 0, [S], ttl 63) IP 192.168.11.102 > 192.168.23.102: ICMP echo request, id 2298, seq 2628, length 64\n...<\/pre>\n\n\n\n
root@PE2:\/home\/vagrant# tcpdump -i ens8\n...\n20:29:16.648176 MPLS (label 80<\/strong>, exp 0, [S], ttl 63) IP 192.168.11.102 > 192.168.23.102: ICMP echo request, id 2298, seq 2627, length 64\n20:29:17.671968 MPLS (label 80<\/strong>, exp 0, [S], ttl 63) IP 192.168.11.102 > 192.168.23.102: ICMP echo request, id 2298, seq 2628, length 64\n...<\/pre>\n\n\n\n
root@CE3:\/home\/vagrant# tcpdump -i ens6\n tcpdump: verbose output suppressed, use -v or -vv for full protocol decode\n listening on ens6, link-type EN10MB (Ethernet), capture size 262144 bytes\n 20:32:03.174796 STP 802.1d, Config, Flags [none], bridge-id 8000.52:54:00:e2:cb:54.8001, length 35\n 20:32:05.158761 STP 802.1d, Config, Flags [none], bridge-id 8000.52:54:00:e2:cb:54.8001, length 35\n 20:32:07.174742 STP 802.1d, Config, Flags [none], bridge-id 8000.52:54:00:e2:cb:54.8001, length 35<\/pre>\n\n\n\n
vagrant@PE2:~$ ip route\n default via 192.168.121.1 dev ens5 proto dhcp src 192.168.121.31 metric 1024 \n 172.20.5.1 encap mpls 16 via 192.168.77.102 dev ens8 proto isis metric 20 \n 172.20.5.5 via 192.168.77.102 dev ens8 proto isis metric 20 \n 192.168.66.0\/24 via 192.168.77.102 dev ens8 proto isis metric 20 \n 192.168.77.0\/24 dev ens8 proto kernel scope link src 192.168.77.101 \n 192.168.121.0\/24 dev ens5 proto kernel scope link src 192.168.121.31 \n 192.168.121.1 dev ens5 proto dhcp scope link src 192.168.121.31 metric 1024 \n vagrant@PE2:~$ \n vagrant@PE2:~$ ip -4 a\n 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000\n inet 127.0.0.1\/8 scope host lo\n valid_lft forever preferred_lft forever\n inet 172.20.5.2\/32 scope global lo\n valid_lft forever preferred_lft forever\n 2: ens5: mtu 1500 qdisc pfifo_fast state UP group default qlen 1000\n inet 192.168.121.31\/24 brd 192.168.121.255 scope global dynamic ens5\n valid_lft 2524sec preferred_lft 2524sec\n 3: ens6: mtu 1500 qdisc pfifo_fast master vrf_cust1 state UP group default qlen 1000\n inet 192.168.23.101\/24 brd 192.168.23.255 scope global ens6\n valid_lft forever preferred_lft forever\n 4: ens7: mtu 1500 qdisc pfifo_fast master vrf_cust2 state UP group default qlen 1000\n inet 192.168.24.101\/24 brd 192.168.24.255 scope global ens7\n valid_lft forever preferred_lft forever\n 5: ens8: mtu 1500 qdisc pfifo_fast state UP group default qlen 1000\n inet 192.168.77.101\/24 brd 192.168.77.255 scope global ens8\n valid_lft forever preferred_lft forever\n vagrant@PE2:~$ \n vagrant@PE2:~$ \n vagrant@PE2:~$ \n vagrant@PE2:~$ \n vagrant@PE2:~$ ip -M route\n 16 as to 16 via inet 192.168.77.102 dev ens8 proto ldp \n 17 via inet 192.168.77.102 dev ens8 proto ldp \n 18 via inet 192.168.77.102 dev ens8 proto ldp \n vagrant@PE2:~$ \n vagrant@PE2:~$ ip route show table 10\n blackhole default \n 192.168.11.0\/24 encap mpls 16\/80 via 192.168.77.102 dev ens8 proto bgp metric 20 \n broadcast 192.168.23.0 dev ens6 proto kernel scope link src 192.168.23.101 \n 192.168.23.0\/24 dev ens6 proto kernel scope link src 192.168.23.101 \n local 192.168.23.101 dev ens6 proto kernel scope host src 192.168.23.101 \n broadcast 192.168.23.255 dev ens6 proto kernel scope link src 192.168.23.101 \n vagrant@PE2:~$ \n vagrant@PE2:~$ \n vagrant@PE2:~$ ip vrf \n Name Table\n vrf_cust1 10\n vrf_cust2 20\n vagrant@PE2:~$ \n\nroot@PE2:\/home\/vagrant# sysctl -a | grep mpls\n net.mpls.conf.ens5.input = 0\n net.mpls.conf.ens6.input = 0\n net.mpls.conf.ens7.input = 0\n net.mpls.conf.ens8.input = 1\n net.mpls.conf.lo.input = 0\n net.mpls.conf.vrf_cust1.input = 0\n net.mpls.conf.vrf_cust2.input = 0\n net.mpls.default_ttl = 255\n net.mpls.ip_ttl_propagate = 1\n net.mpls.platform_labels = 100000\nroot@PE2:\/home\/vagrant# \nroot@PE2:\/home\/vagrant# lsmod | grep mpls\n mpls_iptunnel 16384 3\n mpls_router 36864 1 mpls_iptunnel\n ip_tunnel 24576 1 mpls_router\nroot@PE2:\/home\/vagrant# <\/pre>\n\n\n\n