VXLAN FLOOD&LEARN ON CSR1000v

After getting VXLAN F&L to work on the Nexus 9000v. I thought why not trying the “easier one”, which was already tested a lot of times over the last 1-2 years. And well, i works out of the box.

How is it done?

  • CSR1000v as Leafs
  • IOU L3 Router (you can basically use any router that is capable of OSPF/ISIS/EIGRP and PIM)
  • VPCS als clients to ping
  • GNS3 to capture the Ethernet frames for a deeper understanding of what’s going on

At first you need basic connectivity and dynamic routing in your fabric. Since i played a lot with OSPF lately, it was a welcomed chance to freshen up the command syntax of ISIS:

interface Loopback0
 ip address 10.0.255.11 255.255.255.255

interface GigabitEthernetx
 description to Spine_x
 ip address 10.x.x.2 255.255.255.0
 ip router isis 255

router isis 255
 net 49.0001.0000.0000.0011.00
 is-type level-1
 log-adjacency-changes
 passive-interface Loopback0
interface Loopback0
 ip address 10.0.255.1 255.255.255.255

interface Ethernet0/x
 description to Spine_x
 ip address 10.x.x.1 255.255.255.0
 ip router isis 255

router isis 255
 net 49.0001.0000.0000.0001.00
 is-type level-1
 log-adjacency-changes
 passive-interface Loopback0
Leaf_1#sh isis neighbors 

Tag 255:
System Id       Type Interface     IP Address      State Holdtime Circuit Id
Spine_1         L1   Gi1           10.1.1.1        UP    9        Spine_1.01         
Spine_2         L1   Gi2           10.2.1.1        UP    7        Spine_2.01         
Spine_3         L1   Gi3           10.3.1.1        UP    8        Spine_3.01         
Spine_4         L1   Gi4           10.4.1.1        UP    9        Spine_4.01         

Leaf_1#sh ip route

Gateway of last resort is not set

      10.0.0.0/8 is variably subnetted, 28 subnets, 2 masks
i L1     10.0.255.1/32 [115/10] via 10.1.1.1, 00:00:36, GigabitEthernet1
i L1     10.0.255.2/32 [115/10] via 10.2.1.1, 00:00:36, GigabitEthernet2
i L1     10.0.255.3/32 [115/10] via 10.3.1.1, 00:00:36, GigabitEthernet3
i L1     10.0.255.4/32 [115/10] via 10.4.1.1, 00:00:36, GigabitEthernet4
C        10.0.255.11/32 is directly connected, Loopback0
i L1     10.0.255.12/32 [115/20] via 10.4.1.1, 00:00:36, GigabitEthernet4
                        [115/20] via 10.3.1.1, 00:00:36, GigabitEthernet3
                        [115/20] via 10.2.1.1, 00:00:36, GigabitEthernet2
                        [115/20] via 10.1.1.1, 00:00:36, GigabitEthernet1
i L1     10.0.255.13/32 [115/20] via 10.4.1.1, 00:00:36, GigabitEthernet4
                        [115/20] via 10.3.1.1, 00:00:36, GigabitEthernet3
                        [115/20] via 10.2.1.1, 00:00:36, GigabitEthernet2
                        [115/20] via 10.1.1.1, 00:00:36, GigabitEthernet1
i L1     10.0.255.14/32 [115/20] via 10.4.1.1, 00:00:36, GigabitEthernet4
                        [115/20] via 10.3.1.1, 00:00:36, GigabitEthernet3
                        [115/20] via 10.2.1.1, 00:00:36, GigabitEthernet2
                        [115/20] via 10.1.1.1, 00:00:36, GigabitEthernet1
C        10.1.1.0/24 is directly connected, GigabitEthernet1
L        10.1.1.2/32 is directly connected, GigabitEthernet1
i L1     10.1.2.0/24 [115/20] via 10.1.1.1, 00:00:36, GigabitEthernet1
i L1     10.1.3.0/24 [115/20] via 10.1.1.1, 00:00:36, GigabitEthernet1
i L1     10.1.4.0/24 [115/20] via 10.1.1.1, 00:00:36, GigabitEthernet1
C        10.2.1.0/24 is directly connected, GigabitEthernet2
L        10.2.1.2/32 is directly connected, GigabitEthernet2
i L1     10.2.2.0/24 [115/20] via 10.2.1.1, 00:00:36, GigabitEthernet2
i L1     10.2.3.0/24 [115/20] via 10.2.1.1, 00:00:36, GigabitEthernet2
i L1     10.2.4.0/24 [115/20] via 10.2.1.1, 00:00:36, GigabitEthernet2
C        10.3.1.0/24 is directly connected, GigabitEthernet3
L        10.3.1.2/32 is directly connected, GigabitEthernet3
i L1     10.3.2.0/24 [115/20] via 10.3.1.1, 00:00:36, GigabitEthernet3
i L1     10.3.3.0/24 [115/20] via 10.3.1.1, 00:00:36, GigabitEthernet3
i L1     10.3.4.0/24 [115/20] via 10.3.1.1, 00:00:36, GigabitEthernet3
C        10.4.1.0/24 is directly connected, GigabitEthernet4
L        10.4.1.2/32 is directly connected, GigabitEthernet4
i L1     10.4.2.0/24 [115/20] via 10.4.1.1, 00:00:36, GigabitEthernet4
i L1     10.4.3.0/24 [115/20] via 10.4.1.1, 00:00:36, GigabitEthernet4
i L1     10.4.4.0/24 [115/20] via 10.4.1.1, 00:00:36, GigabitEthernet4

Then it’s time for the pretty forward Multicast part.

ip multicast-routing distributed

interface Loopback0
 ip pim sparse-mode

interface GigabitEthernetx
 ip pim sparse-mode

ip pim bidir-enable
ip multicast-routing 

interface Loopback0
 ip pim sparse-mode

interface Ethernet0/x
 ip pim sparse-mode

ip pim bidir-enable
ip pim bsr-candidate Loopback0 0
ip pim rp-candidate Loopback0 group-list Multicast bidir
!
ip access-list standard Multicast
 permit 224.0.0.0 0.255.255.255
Leaf_1#sh ip pim neighbor 
PIM Neighbor Table
Mode: B - Bidir Capable, DR - Designated Router, N - Default DR Priority,
      P - Proxy Capable, S - State Refresh Capable, G - GenID Capable,
      L - DR Load-balancing Capable
Neighbor          Interface                Uptime/Expires    Ver   DR
Address                                                            Prio/Mode
10.1.1.1          GigabitEthernet1         06:38:11/00:01:38 v2    1 / B S P G
10.2.1.1          GigabitEthernet2         00:01:18/00:01:25 v2    1 / B S P G
10.3.1.1          GigabitEthernet3         00:01:18/00:01:25 v2    1 / B S P G
10.4.1.1          GigabitEthernet4         00:01:16/00:01:26 v2    1 / B S P G

Leaf_1#sh ip mroute 
IP Multicast Routing Table

(*,224.0.0.0/8), 06:28:39/-, RP 10.0.255.1, flags: B
  Bidir-Upstream: GigabitEthernet1, RPF nbr: 10.1.1.1
  Incoming interface list:
    GigabitEthernet4, Accepting/Sparse
    GigabitEthernet3, Accepting/Sparse
    GigabitEthernet2, Accepting/Sparse
    Loopback0, Accepting/Sparse
    GigabitEthernet1, Accepting/Sparse

## 224.0.1.4 = AUTO-RP-DISCOVERY
(*, 224.0.1.40), 06:34:41/00:02:19, RP 0.0.0.0, flags: DCL
  Incoming interface: Null, RPF nbr 0.0.0.0
  Outgoing interface list:
    GigabitEthernet4, Forward/Sparse, 00:02:08/00:02:09
    GigabitEthernet2, Forward/Sparse, 00:02:08/00:02:05
    GigabitEthernet3, Forward/Sparse, 00:02:09/00:02:05
    GigabitEthernet1, Forward/Sparse, 06:34:41/00:02:19

Now it’s time for the VXLAN part

bridge-domain 1 
 member vni 74656
 member GigabitEthernet6 service-instance 1

interface GigabitEthernet6
 description to PC1
 no ip address
 service instance 1 ethernet
  encapsulation untagged

interface nve1
 no ip address
 source-interface Loopback0
 member vni 74656 mcast-group 224.1.2.3
nothing to do here ;)

The spines are just used for IGP routing, multicast and forwarding the traffic from the leafs.
Leaf_1#show nve peers 
Interface  VNI      Type Peer-IP          Router-RMAC    eVNI     state flags UP time
nve1       74656    L2DP 10.0.255.13     
nve1       74656    L2DP 10.0.255.12     
nve1       74656    L2DP 10.0.255.14     

Leaf_1#show bridge-domain 1
Bridge-domain 1 (2 ports in all)
State: UP                    Mac learning: Enabled
Aging-Timer: 300 second(s)
    GigabitEthernet6 service instance 1
    vni 74656
   AED MAC address    Policy  Tag       Age  Pseudoport
   0   0050.7966.6801 forward dynamic   267  nve1.VNI74656, VxLAN 
                                             src: 10.0.255.11 dst: 10.0.255.13
   0   0050.7966.6800 forward dynamic   276  GigabitEthernet6.EFP1
   0   0050.7966.6803 forward dynamic   271  nve1.VNI74656, VxLAN 
                                             src: 10.0.255.11 dst: 10.0.255.12
   0   0050.7966.6802 forward dynamic   276  nve1.VNI74656, VxLAN 
                                             src: 10.0.255.11 dst: 10.0.255.14

To get a good packet capture via Wireshark i did disable the links from Leaf_1 to Spine2-4. Heres the screenshot from Wireshark:

And that’s it, VXLAN F&L on the CSR1000v is working!

Thanks here to Jon Major and his great Blogpost about VXLAN and CSR1kv

Related posts

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.