I had quite a lot of troubles with Nexus 9000v on GNS 1.5 with VXLAN. In fact, after finishing the config (with the same Layout used that you see above) it always ended up with the loopback source interface for the NVE Interfaces getting shutdown by the switch. I didn’t had the time to investigate it (beside the fact that i tried to get it work a lot of times …), but this week i upgraded the environment to GNS 2.0. And there was a very welcomed surprise -> It works 🙂
Update: Ariel Liguori mentioned that PIM-bidir is not official supported as a multicast underlay with VXLAN on the Nexus 9k:
- For 7.0(3)I2(1) and later, bidirectional PIM is not supported for underlay multicast.
At the End of this posting there is the original PIM configuration for PIM bidir for all 4 Switches.
So, here is the configuration for Leaf_1&2 and Spine_1&2.
feature ospf feature pim feature interface-vlan feature vn-segment-vlan-based feature nv overlay vlan 1,192 ip pim rp-address 10.0.0.255 group-list 224.0.0.0/4 ip pim anycast-rp 10.0.0.255 10.0.255.1 ip pim anycast-rp 10.0.0.255 10.0.255.2 vlan 192 vn-segment 192000 interface nve1 no shutdown source-interface loopback255 member vni 192000 mcast-group 224.192.16.0 interface Ethernet1/1 no switchport medium p2p no ip redirects ip unnumbered loopback255 ip router ospf 255 area 0.0.0.1 ip pim sparse-mode no shutdown interface Ethernet1/2 no switchport medium p2p no ip redirects ip unnumbered loopback255 ip router ospf 255 area 0.0.0.1 ip pim sparse-mode no shutdown interface Ethernet1/9 switchport access vlan 192 spanning-tree port type edge interface loopback255 ip address 10.0.255.3/32 ip ospf network point-to-point ip router ospf 255 area 0.0.0.1 ip pim sparse-mode ip igmp join-group 224.1.1.1 boot nxos bootflash:/nxos.7.0.3.I5.2.bin router ospf 255 router-id 10.0.255.3 log-adjacency-changes
feature ospf feature pim feature interface-vlan feature vn-segment-vlan-based feature nv overlay vlan 1,192 ip pim rp-address 10.0.0.255 group-list 224.0.0.0/4 ip pim anycast-rp 10.0.0.255 10.0.255.1 ip pim anycast-rp 10.0.0.255 10.0.255.2 vlan 192 vn-segment 192000 interface nve1 no shutdown source-interface loopback255 member vni 192000 mcast-group 224.192.16.0 interface Ethernet1/1 no switchport medium p2p no ip redirects ip unnumbered loopback255 ip router ospf 255 area 0.0.0.1 ip pim sparse-mode no shutdown interface Ethernet1/2 no switchport medium p2p no ip redirects ip unnumbered loopback255 ip router ospf 255 area 0.0.0.1 ip pim sparse-mode no shutdown interface Ethernet1/9 switchport access vlan 192 spanning-tree port type edge interface loopback255 ip address 10.0.255.4/32 ip ospf network point-to-point ip router ospf 255 area 0.0.0.1 ip pim sparse-mode ip igmp join-group 224.1.1.1 boot nxos bootflash:/nxos.7.0.3.I5.2.bin router ospf 255 router-id 10.0.255.4 log-adjacency-changes
feature ospf feature pim ip pim rp-address 10.0.0.255 group-list 224.0.0.0/4 ip pim anycast-rp 10.0.0.255 10.0.255.1 ip pim anycast-rp 10.0.0.255 10.0.255.2 vlan 1 interface Ethernet1/1 no switchport medium p2p no ip redirects ip unnumbered loopback255 ip router ospf 255 area 0.0.0.1 ip pim sparse-mode no shutdown interface Ethernet1/2 no switchport medium p2p no ip redirects ip unnumbered loopback255 ip router ospf 255 area 0.0.0.1 ip pim sparse-mode no shutdown interface loopback1 ip address 10.0.0.255/32 ip ospf network point-to-point ip router ospf 255 area 0.0.0.1 ip pim sparse-mode interface loopback255 ip address 10.0.255.1/32 ip ospf network point-to-point ip router ospf 255 area 0.0.0.1 ip pim sparse-mode ip igmp join-group 224.1.1.1 boot nxos bootflash:/nxos.7.0.3.I5.2.bin router ospf 255 router-id 10.0.255.1 log-adjacency-changes
feature ospf feature pim ip pim rp-address 10.0.0.255 group-list 224.0.0.0/4 ip pim anycast-rp 10.0.0.255 10.0.255.1 ip pim anycast-rp 10.0.0.255 10.0.255.2 vlan 1 interface Ethernet1/1 no switchport medium p2p no ip redirects ip unnumbered loopback255 ip router ospf 255 area 0.0.0.1 ip pim sparse-mode no shutdown interface Ethernet1/2 no switchport medium p2p no ip redirects ip unnumbered loopback255 ip router ospf 255 area 0.0.0.1 ip pim sparse-mode no shutdown interface loopback1 ip address 10.0.0.255/32 ip ospf network point-to-point ip router ospf 255 area 0.0.0.1 ip pim sparse-mode interface loopback255 ip address 10.0.255.2/32 ip ospf network point-to-point ip router ospf 255 area 0.0.0.1 ip pim sparse-mode ip igmp join-group 224.1.1.1 boot nxos bootflash:/nxos.7.0.3.I5.2.bin router ospf 255 router-id 10.0.255.2 log-adjacency-changes
And finally the test from PC1->PC2 and PC2->PC1
PC1> ping 192.168.1.2 -c 3 84 bytes from 192.168.1.2 icmp_seq=1 ttl=64 time=19.514 ms 84 bytes from 192.168.1.2 icmp_seq=2 ttl=64 time=41.914 ms 84 bytes from 192.168.1.2 icmp_seq=3 ttl=64 time=50.067 ms PC2> ping 192.168.1.1 -c 3 84 bytes from 192.168.1.1 icmp_seq=1 ttl=64 time=63.936 ms 84 bytes from 192.168.1.1 icmp_seq=2 ttl=64 time=36.437 ms 84 bytes from 192.168.1.1 icmp_seq=3 ttl=64 time=72.507 ms
This is the (currently on the 9k not supported) relevant bidir configuration
Leaf_1: ip pim rp-address 10.0.0.255 group-list 224.0.0.0/4 bidir Leaf_2: ip pim rp-address 10.0.0.255 group-list 224.0.0.0/4 bidir Spine_1: ip pim rp-address 10.0.0.255 group-list 224.0.0.0/4 bidir interface loopback1 ip address 10.0.0.254/31 ip pim sparse-mode Spine_2: ip pim rp-address 10.0.0.255 group-list 224.0.0.0/4 bidir interface loopback1 ip address 10.0.0.254/30 ip pim sparse-mode
Hi Martin,
After long troubleshooting session I just find your blog, Many thanks for the great illustration