Ethernet Fabric (VDX, CNA)

Reply
Occasional Contributor
Posts: 5
Registered: ‎02-15-2017
Accepted Solution

Cannot ping VDX6740s from the internal network

Hello. I am hoping someone could help me out. I'm still a noob with Brocade, but I have prior experience with Cisco catalyst switches. I am doing my best reading manuals, forums and best practice guides, but I cannot find the answer I am looking for. Please bear with me.

 

Here's my question.

 

I connected two VDX6740s to the network – they’re setup in logical-chassis mode (VCS fabric). I assigned the following management IPs/virtual IP on the switches, but I cannot ping (nor SSH) them from the internal network.

 

interface Management 1/0

no tcp burstrate

ip icmp echo-reply

no ip address dhcp

ip address 10.0.0.200/24

ipv6 icmpv6 echo-reply

no ipv6 address autoconfig

no ipv6 address dhcp

vrf forwarding mgmt-vrf

 

interface Management 2/0

no tcp burstrate

ip icmp echo-reply

no ip address dhcp

ip address 10.0.0.201/24

ipv6 icmpv6 echo-reply

no ipv6 address autoconfig

no ipv6 address dhcp

vrf forwarding mgmt-vrf

 

rbridge-id 1

ip route 0.0.0.0/0 10.0.0.254

switch-attributes chassis-name MW_VDX6740

switch-attributes host-name EDM-DC01-R03-SW1A

vrf mgmt-vrf

  address-family ipv4 unicast

 

rbridge-id 2

ip route 0.0.0.0/0 10.0.0.254

switch-attributes chassis-name MW_VDX6740

switch-attributes host-name EDM-DC01-R03-SW1B

vrf mgmt-vrf

  address-family ipv4 unicast

 

show vcs detail

Config Mode    : Distributed

VCS Mode        : Logical Chassis

VCS ID              : 10

Virtual IP           : 10.0.0.17/20

Associated rbridge-id        : 2

Total Number of Nodes     : 2

Nodes Disconnected from Cluster : 0

Cluster Condition               : Good

Cluster Status                     : All Nodes Present in the Cluster

 

I can ping my management interfaces and virtual-ip address while consoled in to the switches, but not from the internal network. What am I doing wrong? Is there another command I need to put in place?

 

EDM-DC01-R03-SW1B# ping 10.0.0.117 vrf mgmt-vrf

Type Control-c to abort

PING 10.0.0.117 (10.0.0.117): 56 data bytes

64 bytes from 10.0.0.117: icmp_seq=0 ttl=64 time=0.123 ms

64 bytes from 10.0.0.117: icmp_seq=1 ttl=64 time=0.117 ms

64 bytes from 10.0.0.117: icmp_seq=2 ttl=64 time=0.131 ms

^C--- 10.0.0.117 ping statistics ---

3 packets transmitted, 3 packets received, 0% packet loss

round-trip min/avg/max/stddev = 0.117/0.124/0.131/0.000 ms

 

Any feedback is greatly appreciated. Hoping to hear from y'all soon. 

Highlighted
Brocade Moderator
Posts: 206
Registered: ‎06-30-2010

Re: Cannot ping VDX6740s from the internal network

Hi jenyx,

 

You need to make sure that your management default route is defined under the management VRF, you currently have this defined under the default VRF which means that the management interface has now route.

 

A show ip route vrf mgmt-vrf should show no routes (apart from connected), whereas a show ip route should yield 0.0.0.0/0 10.0.0.254

 

To correct this add route as following under the management VRF

 

vrf mgmt-vrf
 address-family ipv4 unicast
  ip route 0.0.0.0/0 10.0.0.254

 

Regards

Mick

 

 


If my response has solved your query please click the "Accept as Solution" button.

Any and all information provided by me is not reviewed, approved or endorsed by Brocade and is provided solely as a convenience for Brocade customers.

All systems and all networks are different and unique. If you have a service affecting network problem, please open a TAC service request for service through Brocade, or through your OEM equipment provider.
Occasional Contributor
Posts: 5
Registered: ‎02-15-2017

Re: Cannot ping VDX6740s from the internal network

Hello, thanks for your response.

 

I added the route in the mgmt-vrf as suggested. I can now ping from the VDXs to my core switch. I cannot, however, ping from my core switch to the VCS cluster and I still cannot ping the VDXs from the internal network.

 

Note that my VCS cluster is physically connected to ICX7750 cluster using 2-QSFP+ modules that are link aggregated.

 

Having said so, I cannot enable the mgmt-vrf on both interfaces because they're configured as LAGs. Sorry for the noob question, but I feel like I have to ask anyway.. Does this mean I have to have another physical (mgmt) connection from my VDX to ICX so I can ping from my internal network to my VDXs? 

 

SW1A(config)# do ping 10.0.0.254
Type Control-c to abort
PING 10.0.0.254 (10.0.0.254): 56 data bytes
64 bytes from 10.0.0.254: icmp_seq=0 ttl=64 time=0.144 ms
64 bytes from 10.0.0.254: icmp_seq=1 ttl=64 time=0.110 ms
64 bytes from 10.0.0.254: icmp_seq=2 ttl=64 time=0.106 ms
^C^C--- 128.144.50.250 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.106/0.120/0.144/0.000 ms

Brocade Moderator
Posts: 206
Registered: ‎06-30-2010

Re: Cannot ping VDX6740s from the internal network

Hi jenyx,

 

Just looking at your configuration again, I see that the Management interfaces are /24 and Virtual IP is /20.  Is this deliberate?

 

May need additional information to understand exactly what you mean by internal network etc.  A diagram would be useful to better understand your topology.

 

You may need to create a Ve interface under each of the Rbridges on a VLAN which is configured on the LAG then this Ve could be part of the mgmt-vrf if that's what you require

 

Not 100% sure exactly what you are trying to achieve, as mentioned a diagram may help

 

Regards

Mick


If my response has solved your query please click the "Accept as Solution" button.

Any and all information provided by me is not reviewed, approved or endorsed by Brocade and is provided solely as a convenience for Brocade customers.

All systems and all networks are different and unique. If you have a service affecting network problem, please open a TAC service request for service through Brocade, or through your OEM equipment provider.
Occasional Contributor
Posts: 5
Registered: ‎02-15-2017

Re: Cannot ping VDX6740s from the internal network

[ Edited ]

Hello Mick,

 

Please see attached diagram. I added some notes in the diagram to better explain what I have done. 

 

 

"Just looking at your configuration again, I see that the Management interfaces are /24 and Virtual IP is /20.  Is this deliberate?"

-No, it was typo. Sorry.

 

"You may need to create a Ve interface under each of the Rbridges on a VLAN which is configured on the LAG then this Ve could be part of the mgmt-vrf if that's what you require"

-I figured the problem could be with the VLAN tagging on the ICXs.. So, I tagged the link-aggregated ports to allow VLAN 1 traffic. (see diagram for clarity).

-I created an interface Ve 1  under each Rbridges and bound it to mgmt-vrf, but I still cannot ping the ICX (VLAN1) default-gateway.

 

rbridge-id 1
switch-attributes chassis-name MW_VDX6740
switch-attributes host-name SW1A
vrf mgmt-vrf
address-family ipv4 unicast
max-route 3600
ip route 0.0.0.0/0 10.0.0.254
!
address-family ipv6 unicast

!
system-monitor fan threshold marginal-threshold 1 down-threshold 2
system-monitor fan alert state removed action raslog
system-monitor power threshold marginal-threshold 1 down-threshold 2
system-monitor power alert state removed action raslog
system-monitor temp threshold marginal-threshold 1 down-threshold 2
system-monitor cid-card threshold marginal-threshold 1 down-threshold 2
system-monitor cid-card alert state none action none
system-monitor sfp alert state none action none
system-monitor compact-flash threshold marginal-threshold 1 down-threshold 0
system-monitor MM threshold marginal-threshold 1 down-threshold 0
system-monitor LineCard threshold marginal-threshold 1 down-threshold 2
system-monitor LineCard alert state none action none
system-monitor SFM threshold marginal-threshold 1 down-threshold 2
no protocol vrrp
no protocol vrrp-extended
hardware-profile tcam default
hardware-profile route-table default
clock timezone Etc/GMT
ag
pg 0
modes lb
rename pg0
!
timeout fnm 120
counter reliability 25
!
telnet server shutdown
ssh server key rsa 2048
ssh server key ecdsa 256
ssh server key dsa
ssh client cipher non-cbc
fcoe
fcoe-enodes 0
!
interface Ve 1

vrf forwarding mgmt-vrf

no shutdown
!

rbridge-id 2
switch-attributes chassis-name MW_VDX6740
switch-attributes host-name SW1B
vrf mgmt-vrf
address-family ipv4 unicast

max-route 3600
ip route 0.0.0.0/0 10.0.0.254
!
address-family ipv6 unicast
!
!
system-monitor fan threshold marginal-threshold 1 down-threshold 2
system-monitor fan alert state removed action raslog
system-monitor power threshold marginal-threshold 1 down-threshold 2
system-monitor power alert state removed action raslog
system-monitor temp threshold marginal-threshold 1 down-threshold 2
system-monitor cid-card threshold marginal-threshold 1 down-threshold 2
system-monitor cid-card alert state none action none
system-monitor sfp alert state none action none
system-monitor compact-flash threshold marginal-threshold 1 down-threshold 0
system-monitor MM threshold marginal-threshold 1 down-threshold 2
system-monitor LineCard threshold marginal-threshold 1 down-threshold 2
system-monitor LineCard alert state none action none
system-monitor SFM threshold marginal-threshold 1 down-threshold 2
no protocol vrrp
no protocol vrrp-extended
hardware-profile tcam default
hardware-profile route-table default
clock timezone Etc/GMT
ag
pg 0
modes lb
rename pg0
!
timeout fnm 120
counter reliability 25
!
telnet server shutdown
ssh server key rsa 2048
ssh server key ecdsa 256
ssh server key dsa
fcoe
fcoe-enodes 0
!
interface Ve 1
vrf forwarding mgmt-vrf
no shutdown

Here's my config for VDX's link-aggregated ports: 

 

interface Port-channel 10
vlag ignore-split
speed 40000
switchport
switchport mode trunk
switchport trunk allowed vlan all
switchport trunk tag native-vlan
spanning-tree shutdown
no shutdown

 

interface FortyGigabitEthernet 1/0/51
no fabric isl enable
no fabric trunk enable
channel-group 10 mode active type standard
lacp timeout long
sflow enable
no shutdown

 

interface FortyGigabitEthernet 2/0/51
description LAG2_CORE_CS1B
no fabric isl enable
no fabric trunk enable
channel-group 10 mode active type standard
lacp timeout long
sflow enable
no shutdown


Port-channel 10 is up, line protocol is up
Hardware is AGGREGATE, address is c4f5.7c80.5637
Current address is c4f5.7c80.5637
Interface index (ifindex) is 671088650
Minimum number of links to bring Port-channel up is 1
MTU 2500 bytes
LineSpeed Actual : 80000 Mbit
Allowed Member Speed : 40000 Mbit
Priority Tag disable
IPv6 RA Guard disable
Last clearing of show interface counters: 4d04h34m
Queueing strategy: fifo
Receive Statistics:
1507535 packets, 153393479 bytes
Unicasts: 412018, Multicasts: 495026, Broadcasts: 600491
64-byte pkts: 171628, Over 64-byte pkts: 1134329, Over 127-byte pkts: 157997
Over 255-byte pkts: 9393, Over 511-byte pkts: 23221, Over 1023-byte pkts: 10638
Over 1518-byte pkts(Jumbo): 329
Runts: 0, Jabbers: 0, CRC: 0, Overruns: 0
Errors: 0, Discards: 0
Transmit Statistics:
308628 packets, 38108098 bytes
Unicasts: 764, Multicasts: 125533, Broadcasts: 182331
Underruns: 0
Errors: 0, Discards: 0
Rate info:
Input 0.122748 Mbits/sec, 119 packets/sec, 0.00% of line-rate
Output 0.000000 Mbits/sec, 0 packets/sec, 0.00% of line-rate
Time since last interface status change: 01:58:53

 

On ICX's end:

=== LAG "Core2_VDXSW" ID 10 (dynamic Deployed) ===
LAG Configuration:
Ports: e 1/2/3 e 2/2/3
Port Count: 2
Primary Port: 1/2/3
Trunk Type: hash-based
LACP Key: 20010
Deployment: HW Trunk ID 7
Port Link State Dupl Speed Trunk Tag Pvid Pri MAC Name
1/2/3 Up Forward Full 40G 10 Yes N/A 0 cc4e.2438.9300
2/2/3 Up Forward Full 40G 10 Yes N/A 0 cc4e.2438.9300

Port [Sys P] [Port P] [ Key ] [Act][Tio][Agg][Syn][Col][Dis][Def][Exp][Ope]
1/2/3 1 1 20010 Yes L Agg Syn Col Dis No No Ope
2/2/3 1 1 20010 Yes L Agg Syn Col Dis No No Ope


Partner Info and PDU Statistics
Port Partner Partner LACP LACP
System ID Key Rx Count Tx Count
1/2/3 32768-01e0.5200.000a 10 11668 11644
2/2/3 32768-01e0.5200.000a 10 11682 11653

 

When I do 'show ip route vrf mgmt-vrf', it only shows below..

Total number of IP routes: 0
10.0.0.200/32 DIRECT mgmt 1 0/0 D 4d5h

 

What am I doing wrong? Please email me at jenny.solomon@albertainnovates.ca if you need more information. Much appreciated! Smiley Happy

Occasional Contributor
Posts: 5
Registered: ‎02-15-2017

Re: Cannot ping VDX6740s from the internal network

I don't know what happened to my response to this post yesterday..sigh.

But anyway, I have attached the diagram on this post. 

 

 

'Just looking at your configuration again, I see that the Management interfaces are /24 and Virtual IP is /20.  Is this deliberate?'

-No, it was typo. I meant to write /24.

 

'You may need to create a Ve interface under each of the Rbridges on a VLAN which is configured on the LAG then this Ve could be part of the mgmt-vrf if that's what you require'

-I tried this as suggested, but I may be doing it wrong, because I still cannot ping my existing network's Default gateway. Anyway, I added a virtual interface to each Rbridge-id, and bound it to mgmt-vrf. Still cannot ping my ICX. I tried assigning an IP address to int ve but looks like it cannot the same IP subnet as the management interface. See diagram, I wrote notes on there too. 

 

SW1B# show ip route vrf mgmt-vrf
Total number of IP routes: 0
128.144.50.113/32 DIRECT mgmt 1 0/0 D 4d20h

 

### VDX LAG 1/0/51&2/0/51

SW1B# show running-config interface Port-channel 10
interface Port-channel 10
vlag ignore-split
speed 40000
switchport
switchport mode trunk
switchport trunk allowed vlan all
no switchport trunk tag native-vlan
spanning-tree shutdown
no shutdown

 

SW1B# show running-config interface FortyGigabitEthernet 1/0/51
interface FortyGigabitEthernet 1/0/51
no fabric isl enable
no fabric trunk enable
channel-group 10 mode active type standard
lacp timeout long
sflow enable
no shutdown

 

SW1B# show running-config interface FortyGigabitEthernet 2/0/51
interface FortyGigabitEthernet 2/0/51
description LAG2_CORE_CS1B
no fabric isl enable
no fabric trunk enable
channel-group 10 mode active type standard
lacp timeout long
sflow enable
no shutdown
!

 

SW1B# show interface port-channel 10 switchport
Interface name : Port-channel 10
Switchport mode : trunk
Fcoeport enabled : no
Ingress filter : enable
Acceptable frame types : vlan-tagged only
Native Vlan : 1
Active Vlans : 1-2,11,100-115,117-119,125,128-132,135-139,222-226,999
Inactive Vlans : -
MAC learn disable Vlans : -

 

ON ICX's side ports 1/2/3&2/2/3 are link-aggregated: (see diagram for clarity)

 

=== LAG "Core2_VDXSW1A" ID 10 (dynamic Deployed) ===
LAG Configuration:
Ports: e 1/2/3 e 2/2/3
Port Count: 2
Primary Port: 1/2/3
Trunk Type: hash-based
LACP Key: 20010
Deployment: HW Trunk ID 7
Port Link State Dupl Speed Trunk Tag Pvid Pri MAC Name
1/2/3 Up Forward Full 40G 10 Yes N/A 0 cc4e.2438.9300
2/2/3 Up Forward Full 40G 10 Yes N/A 0 cc4e.2438.9300

Port [Sys P] [Port P] [ Key ] [Act][Tio][Agg][Syn][Col][Dis][Def][Exp][Ope]
1/2/3 1 1 20010 Yes L Agg Syn Col Dis No No Ope
2/2/3 1 1 20010 Yes L Agg Syn Col Dis No No Ope


Partner Info and PDU Statistics
Port Partner Partner LACP LACP
System ID Key Rx Count Tx Count
1/2/3 32768-01e0.5200.000a 10 13587 13560
2/2/3 32768-01e0.5200.000a 10 13601 13569

 

Do note that I have tagged ports 1/2/3&2/2/3 to allow all VLANs (because I thought this could be why I cannot ping the ICX). 

 

Please email me at jenny.solomon@albertainnovates.ca if you need more details. Again, I appreciate your time. Smiley Happy 

 

Join the Community

Get quick and easy access to valuable resource designed to help you manage your Brocade Network.