Cisco Cisco Nexus 2224TP GE Fabric Extender White Paper

Page of 70
 
 
 
© 2014 Cisco | IBM. All rights reserved. 
Page 1 
Cisco Nexus B22 Blade Fabric 
Extender for IBM 
Design and Deployment Guide 
November 2013 
 
 
 
 
 
 
 
Int 
 
 

Summary of Contents of white paper for Cisco Cisco Nexus 2224TP GE Fabric Extender

  • Page 1 Cisco Nexus B22 Blade Fabric Extender for IBM Design and Deployment Guide November 2013 Int © 2014 Cisco |...
  • Page 2Contents Introduction .............................................................................................................................................................. 3 Network Diagram ...................................................................................................................................................... 3 Hardware Installation ............................................................................................................................................... 5 Fabric Extender Management Model ..................................................................................................................... 6 Fabric Connectivity...
  • Page 3: Network DiagramIntroduction ® ® The Cisco Nexus B22 Blade Fabric Extender for IBM extends the Cisco Nexus switch fabric to the...
  • Page 4Figure 1: Cisco Nexus Virtual Chassis Topology The Cisco Nexus switches, along with the Cisco Nexus 2000 Series and B22...
  • Page 5: Hardware InstallationThe fourth blade chassis from the left in Figure 1 contains Cisco Nexus B22 for IBM fabric extenders that connect...
  • Page 6: Fabric Extender Management ModelNo configuration is required from the chassis MMB. Only the minimum CMM firmware is required to properly detect and enable...
  • Page 7: Statically Pinned Fabric Interface ConnectionFabric Connectivity Options The Cisco Nexus B22 for IBM creates a distributed, modular chassis with the Cisco Nexus parent switch...
  • Page 8Table 6 summarizes the assignment with four fabric links: With the max link parameter set to 4, the interfaces are...
  • Page 9: Port Channel Fabric Interface ConnectionThe relationship of host-side ports to parent switch fabric ports is static. If a fabric interface fails, all its associated...
  • Page 10: Configuring a Fabric Port ChannelFigure 4 shows Port Channel designs. Note: A fabric interface that fails in the Port Channel does not trigger a...
  • Page 11 N5548-Bottom(config)# 2. Enable the fabric extender feature: N5548-Bottom(config)# N5548-Bottom(config)# feature fex N5548-Bottom(config)# 3. Logically create the fabric extender: N5548-Bottom(config)#...
  • Page 126. Repeat the steps on the second Cisco Nexus 5000 Series Switch connected to the fabric extender in interconnect bay...
  • Page 13: Virtual Port Channel ConnectionVirtual Port Channel Connection vPCs allow links that are physically connected to two different Cisco Nexus switches to form a...
  • Page 14: Configuring a vPCConfiguring a vPC The high-level steps for enabling vPC are listed here. This configuration should be implemented on both switches...
  • Page 154. Configure the vPC peer link: N5548-Bottom# interface port-channel 20 N5548-Bottom(config-if)# interface ethernet 1/9 N5548-Bottom(config-if)# channel-group 20 N5548-Bottom(config-if)# interface ethernet...
  • Page 16: Server Network Teaming vPC role : primary Number of vPCs configured : 0 Peer Gateway : Disabled Dual-active excluded VLANs : -...
  • Page 17: Creating Host-Side vPC for Server Links with LACPFigure 7: Enhanced vPC Configuration Creating Host-Side vPC for Server Links with LACP 1. Enable LACP on both parent switches....
  • Page 183. Define and configure the right diagram FEX 192 for enhanced vPC on the left Nexus parent 5548-Bottom(config)# fex 192...
  • Page 19 5548-Bottom#(config-if)# no shutdown 5548-Top# configure terminal 5548-Top#(config)# interface port-channel 201 5548-Top#(config)# switchport mode trunk 5548-Top#(config)# switchport trunk native vlan...
  • Page 20Note: With EvPC when you configure a Port Channel from the Cisco Nexus 2000 Series to the server, do not...
  • Page 21: Configuring FCoE Group Port- Type Protocol Member Ports Channel -------------------------------------------------------------------------------- 10 Po10(SD) Eth NONE -- 20 Po20(SU) Eth NONE Eth1/9(P) Eth1/10(D)...
  • Page 22: Configuring the Cisco Nexus 5000 Series and B22 for IBM for FCoEFigure 8 shows the ports configured for FCoE and the drivers loaded. Figure 8: OneCommand FCoE Utility Showing Ports Configured...
  • Page 23 FC plugin loaded successfully FCoE manager enabled successfully FC enabled on all modules successfully Warning: Ensure class-fcoe is included...
  • Page 24Create the virtual Fibre Channel interface (Port Channel) on both switches: N5548-Bottom(config)# N5548-Bottom(config)# interface vfc 1032 N5548-Bottom(config-if)# switchport mode F...
  • Page 25 N5548-Top(config)# N5548-Top(config)# vsan database N5548-Top(config-vsan-db)# vsan 201 N5548-Top(config-vsan-db)# vsan 201 interface vfc1032 N5548-Top(config-vsan-db)# Note: The VLAN and VSAN numbers...
  • Page 269. Create the necessary zone and zone sets with appropriate members: N5548-Bottom(config)# zoneset name zoneset1 vsan 200 N5548-Bottom(config-zoneset)#zone name zone1...
  • Page 27: iSCSI ConfigurationYou can run these commands on a second Cisco Nexus Switches to verify the fabric. Figure 9 shows a server...
  • Page 281. Define a class map of QoS policies on the first switch to identify the iSCSI traffic (here, iSCSI traffic...
  • Page 297. Configure the no-drop policy maps: N5548-Top(config)# class-map type network-qos iSCSI-C1 N5548-Top(config-cmap-nq)# match qos-group 2 N5548-Top(config-cmap-nq)# exit N5548-Top(config)# policy-map type...
  • Page 30: Virtual Network Adapter PartitioningThe storage array should then be visible as shown in Figure 11. Figure 11: IBM Flex System X440+10GB Fabric Blade...
  • Page 311. Install the license: a. Capture the application-specific integrated circuit (ASIC) ID from the LoM by opening the unified extensible...
  • Page 322. Configure the virtual network adapter: a. During the boot cycle, press F1 to open the UEFI menu. b. Select...
  • Page 33 e. Make sure that the multichannel mode is set to Switch Independent and select Controller Configuration. f. Select the...
  • Page 34 h. Select the Configure LPVID option. 3. Configure unique VLANs as necessary for each Ethernet vNIC. a. This feature...
  • Page 35: show fexDebug Commands show fex This command displays the status of the fabric extenders that are powered on and connected. Cisco...
  • Page 36: show fex detail FEX FEX FEX FEX Fex Number Description State Model Serial 191 FEX0191 Online N2K-B22IBM-P FOC1730R0XQ 192 FEX0192 Online N2K-B22IBM-P...
  • Page 37: show interface brief Eth191/1/13 Down Po191 Eth191/1/14 Up Po191 Logs: 09/25/2013 11:09:20.990637: Module register received 09/25/2013 11:09:20.993278: Registration response sent 09/25/2013 11:09:21.98103:...
  • Page 38 Eth1/12 1 eth access down SFP not inserted 10G(D) -- Eth1/13 1 eth access down SFP not inserted 10G(D)...
  • Page 39 -------------------------------------------------------------------------------- Port-channel VLAN Type Mode Status Reason Speed Protocol Interface -------------------------------------------------------------------------------- Po10 1 eth trunk down No operational members...
  • Page 40 Eth160/1/6 1 eth access down Link not connected auto(D) -- Eth160/1/7 1 eth access down Link not connected auto(D)...
  • Page 41 Eth192/1/6 1 eth access down Link not connected auto(D) -- Eth192/1/7 182 eth trunk up none 10G(D) 207 Eth192/1/8...
  • Page 42: show interface ethernet 191/1/1show interface ethernet 191/1/1 This command displays detailed statistics for Cisco Nexus B22 for IBM fabric extender port 1. N5548-Bottom#...
  • Page 43: show vlanshow vlan This command displays the VLAN and the ports associated with the VLAN. N5548-Bottom(config-if)# show vlan N5548-Bottom(config-if)# N5548-Bottom# show...
  • Page 44: show interface fex-fabric Eth194/1/1, Eth194/1/2 Eth194/1/3, Eth194/1/4 Eth194/1/5, Eth194/1/6 Eth194/1/7, Eth194/1/8 Eth194/1/9, Eth194/1/10 Eth194/1/11, Eth194/1/12 Eth194/1/13, Eth194/1/14 50 iSCSI-VL50 active Po20, Eth1/9,...
  • Page 45: Cisco Nexus 5000 Series Switch 1 Configuration 194 Eth1/20 Active 1 N2K-B22IBM-P FOC1720R0VV N5596-1-B22# Cisco Nexus Configurations Cisco Nexus 5000 Series Switch 1 Configuration N5548-Bottom(config)# show...
  • Page 46 class type network-qos class-default mtu 9216 multicast-optimize system qos service-policy type queuing input fcoe-default-in-policy service-policy type queuing output fcoe-default-out-policy...
  • Page 47 fcoe vsan 200 name FCoE-VL200 spanning-tree vlan 1-3967 priority 4096 vpc domain 5 role priority 1 system-priority 1 peer-keepalive...
  • Page 48 interface port-channel201 switchport mode trunk switchport trunk allowed vlan 182-184,200 interface port-channel202 switchport access vlan 182 interface port-channel203 switchport...
  • Page 49 vsan 200 interface vfc1037 vsan 200 interface vfc1044 vsan 200 interface fc1/48 interface fc1/41 interface fc1/42 interface fc1/43 interface...
  • Page 50 switchport mode trunk channel-group 20 interface Ethernet1/10 switchport mode trunk channel-group 20 interface Ethernet1/11 interface Ethernet1/12 interface Ethernet1/13 interface...
  • Page 51 channel-group 192 interface Ethernet1/24 switchport mode fex-fabric fex associate 192 channel-group 192 interface Ethernet1/25 interface Ethernet1/26 interface Ethernet1/27 interface...
  • Page 52 interface Ethernet2/4 interface Ethernet2/5 interface Ethernet2/6 interface Ethernet2/7 interface Ethernet2/8 interface Ethernet2/9 interface Ethernet2/10 interface Ethernet2/11 interface Ethernet2/12 interface...
  • Page 53 interface Ethernet191/1/6 interface Ethernet191/1/7 switchport mode trunk switchport trunk native vlan 182 switchport trunk allowed vlan 182-184,200 channel-group 201...
  • Page 54 switchport trunk allowed vlan 182-184,200 channel-group 201 mode active interface Ethernet192/1/8 interface Ethernet192/1/9 interface Ethernet192/1/10 interface Ethernet192/1/11 interface Ethernet192/1/12...
  • Page 55 interface Ethernet193/1/11 interface Ethernet193/1/12 interface Ethernet193/1/13 interface Ethernet193/1/14 clock timezone pst -7 0 system default zone default-zone permit system...
  • Page 56: Cisco Nexus 5000 Series Switch 2 ConfigurationCisco Nexus 5000 Series Switch 2 Configuration N5548-Top(config)# show run !Command: show running-config !Time: Mon Oct 14 16:21:29 2013 version...
  • Page 57 service-policy type queuing output fcoe-default-out-policy service-policy type qos input iSCSI-C1 service-policy type network-qos iSCSI-C1 control-plane service-policy input copp-system-policy-customized fex...
  • Page 58 vlan 201 fcoe vsan 201 name FCoE-VL201 spanning-tree vlan 1-3967 priority 8192 vpc domain 5 role priority 2 system-priority...
  • Page 59 interface port-channel193 switchport mode fex-fabric fex associate 193 vpc 193 interface port-channel194 switchport mode fex-fabric fex associate 194 vpc...
  • Page 60 switchport allowed vsan 201 no shutdown interface vfc1037 bind interface Ethernet192/1/7 switchport trunk allowed vsan 201 no shutdown interface...
  • Page 61 interface Ethernet1/4 interface Ethernet1/5 switchport mode fex-fabric fex associate 161 channel-group 61 interface Ethernet1/6 switchport mode fex-fabric fex associate...
  • Page 62 interface Ethernet1/19 switchport mode fex-fabric fex associate 193 channel-group 193 interface Ethernet1/20 switchport mode fex-fabric fex associate 194 channel-group...
  • Page 63 interface Ethernet1/28 switchport mode fex-fabric fex associate 196 interface Ethernet1/29 switchport mode fex-fabric fex associate 196 interface Ethernet1/30 switchport...
  • Page 64 interface Ethernet2/6 interface Ethernet2/7 interface Ethernet2/8 interface Ethernet2/9 interface Ethernet2/10 interface Ethernet2/11 interface Ethernet2/12 interface Ethernet2/13 interface Ethernet2/14 interface...
  • Page 65 interface Ethernet160/1/11 interface Ethernet160/1/12 interface Ethernet160/1/13 interface Ethernet160/1/14 interface Ethernet160/1/15 interface Ethernet160/1/16 interface Ethernet161/1/1 interface Ethernet161/1/2 interface Ethernet161/1/3 interface...
  • Page 66 switchport trunk allowed vlan 182-184,201 interface Ethernet191/1/2 interface Ethernet191/1/3 switchport access vlan 182 channel-group 203 interface Ethernet191/1/4 switchport access...
  • Page 67 interface Ethernet192/1/3 switchport access vlan 182 channel-group 203 interface Ethernet192/1/4 switchport access vlan 182 interface Ethernet192/1/5 interface Ethernet192/1/6 interface...
  • Page 68 interface Ethernet193/1/6 interface Ethernet193/1/7 switchport mode trunk spanning-tree port type edge trunk channel-group 317 interface Ethernet193/1/8 interface Ethernet193/1/9 interface...
  • Page 69 interface Ethernet196/1/13 interface Ethernet196/1/14 clock timezone pst -7 0 system default zone default-zone permit system default zone distribute full...
  • Page 70: For More InformationConclusion The advent of Cisco Nexus 2000 Series Fabric Extenders has enabled customers to benefit from both top-of-rack (ToR) and...