Fibre Channel SAN Topologies - Data Storage, …

1 Fibre Channel SAN TopologiesVersion Fibre Channel Topology Overview Simple and Complex Fibre Channel SAN Topologies Brocade Virtual Fabrics Case Studies FICON TopologiesErik SmithRichard HultmanDennis KloeppingFibre Channel SAN Topologies TechBook2 Copyright 2011- 2015 EMC Corporation. All rights believes the information in this publication is accurate as of its publication date. The information issubject to change without INFORMATION IN THIS PUBLICATION IS PROVIDED AS IS. EMC CORPORATION MAKES NOREPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THISPUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY ORFITNESS FOR A PARTICULAR , copying, and distribution of any EMC software described in this publication requires an applicablesoftware , EMC, and the EMC logo are registered trademarks or trademarks of EMC Corporation in the UnitedState and other countries. All other trademarks used herein are the property of their respective the most up-to-date regulator document for your product line, go to EMC Online Support( ).

2 Part number Channel SAN Topologies 9 Chapter 1 Fibre Channel SAN TopologiesFibre Channel topology 16 Instructions for using this 17 General best 19 Switch-specific best 24 Connectrix 24 Connectrix 27 Host and storage 27 Switch and fabric management .. 29 Connectrix 29 Connectrix 31 Connectrix 31 Connectrix 32 Chapter 2 Simple Fibre Channel SAN TopologiesSingle switch fabrics .. 34 Overview of fabric design 34 Connectrix B example .. 35 Connectrix MDS-Series 38 Two switch fabrics .. 46 Overview of fabric design 46 Connectrix B example .. 47 Connectrix MDS 59 Blade switch with direct attached 69 General steps to set up a blade server .. 70 ContentsFibre Channel SAN Topologies TechBook4 ContentsBest 73 Host and storage 73 Switch and fabric 74 IBM Brocade 75 Chapter 3 Complex Fibre Channel SAN TopologiesBest practices .. 88 ISL subscription .. 88 Host and storage 88 Switch and fabric 88 Four switch full 89 Overview of fabric design considerations.

3 89 Connectrix B 90 Connectrix MDS 101 Compound core edge 113 Overview of fabric design considerations .. 113 Connectrix B 115 Connectrix MDS 131 Heterogeneous switch interoperability .. 148 Interoperability 149 Heterogeneous SAN 151 How to set up an interoperable switched fabrictopology .. 151 Cisco Inter VSAN Routing (IVR) in a heterogeneousenvironment .. 180 Vendor-specific switch settings for 184 Heterogeneous interoperability test information .. 190 Chapter 4 Monitoring your 202 Switch-based error types .. 204 Fabric resiliency features and 210 Brocade SAN resiliency 210 Fabric resiliency thresholds .. 212 Quick reference for steps to address switch-based errorsissues .. 213 Brocade fabric resiliency 218 Latency and congestion bottleneck conditions .. 218 Latency severities .. 2215 Fibre Channel SAN Topologies TechBookContentsLatency detection, notification, isolation, andmitigation .. 222 Configuring FS features case 225 Case study: Brocade CLI and CMDCE.

4 225 Configuring Bottleneck Detection .. 229 Enabling port 233 Configuring Edge Hold 238 Summary .. 240 Chapter 5 Brocade Virtual Fabrics Case StudyBrocade Virtual Fabrics case study overview .. 242 Objectives of Virtual Fabrics 243 Logical Switch capability .. 244 Virtual Fabrics and inter-switch links .. 247 How to configure Brocade Virtual Fabrics case 250 Brocade Virtual Fabrics versus traditional Cisco VirtualSANs .. 268 Domain IDs must be allocated to Logical Switcheswhile partitioning an existing fabric into 268 Zoning information may be shared between LogicalSwitches, even when FIDs may not 269 Switch name must be configured on the 270 Deleting a VF .. 270 Brocade Base Switches 271 FCR with Brocade VFs .. 271 Chapter 6 FICON TopologiesOverview .. 274 Topology 276 Zoning practices .. 277 Cascading .. 278 Terminology .. 279 IOCP considerations .. 281 FICON and EMC 283 CUP (fabric management server).. 284 Connectrix B 285 Supported 285 OCP considerations.

5 286 Fibre Channel SAN Topologies TechBook6 ContentsCUP 286 Switch node 287 EMC 287 Connectrix MDS 288 Supported products .. 288 Requirements .. 288 Configuring .. 289 OCP 290 CUP 290 FICON configuration 290 Switch node 291 FICON port 295 Fibre Channel SAN Topologies TechBook7 TitlePage1 Single switch topology example .. 342 Single switch SAN using DS-5100B and EZSwitchSetup with zoning .. 353 Connectrix MDS 9506 using VSANs .. 384 Two switch fabric example .. 465 Two switch SAN with DS-5100B and DS-5300B using CLI toconfigure SAN ..486 Port settings .. 557 Two Connectrix MDS 9506s using VSANs .. 598(a) Blade server direct attached to storage (b) two-tier FC SAN .. 709 IBM blade server directly attached to storage .. 7510 IBM Brocade 4 GB SAN switch module (32R1812) directly attachedto storage ..7611 Four switch full mesh fabric .. 8912 Four switch fabric with ED-48000Bs .. 9113 Four Connectrix MDS 9506s full mesh configuration.

6 10114 Four switch compound core edge switches .. 11315 Four ED-48000Bs in full mesh configuration with edge MDS 9513s in full mesh configuration with edge 1: Basic configuration .. 15518 Phase 2: Adding Connectrix MDS 9513 .. 15719 Phase 3: Moving half the host and storage ports .. 16020 Phase 4: Completely moving host and storage ports .. 16121 Phase 5: Adding Connectrix MDS 9216 .. 16222 Phase 6: Moving hosts and storage to a new edge .. 16423 Phase 7: Adding Connectrix MDS switch to core example .. 16524 Phase 1: Basic configuration .. 16825 Phase 2: Adding Connectrix MDS 9513 .. 17126 Phase 3: Moving half the host and storage ports .. 174 FiguresFibre Channel SAN Topologies TechBook8 Figures27 Phase 4: Completely moving host and storage ports .. 17528 Phase 5: Adding Connectrix MDS 9216 .. 17629 Phase 6: Moving hosts and storage to a new edge .. 17830 Phase 7: Adding Connectrix M switch to the core .. 17931 Fabric wide effects of a latency bottleneck condition.

7 21932 Fabric wide effects of a congestion bottleneck condition .. 22033 Uncongested environment .. 22634 Impact of a slow drain port .. 22735 Buffer Queue for port 7 continues to grow .. 22836 Buffer Queue for port 7 on Switch A, port 1 .. 22937 Port fencing dialog box .. 23538 Port Fencing dialog box .. 23639 Create a customized threshold .. 23740 Apply customized threshold .. 23741 Logical Switches .. 24442 DISL connections between Logical Switches .. 24743 ISL connection between Logical Switches and connection between Base Switches (BS) .. 24945 Topology A example .. 25046 Topology B example .. 25147 Block diagram of fabric topology .. 26148 Cascaded IOCP .. 28149 Cascaded FICON IOCP .. 281 Fibre Channel SAN Topologies TechBook9 PrefaceThis EMC Engineering TechBook provides a high-level overview of FibreChannel SAN Topologies , discusses simple and complex Fibre Channel SANtopologies, shows how to monitor your SAN, and provides case studies forBrocade Virtual Fabrics.

8 FICON connectivity is also would like to thank all the contributors to this document, includingEMC engineers, EMC field personnel, and partners. Your contributions part of an effort to improve and enhance the performance and capabilitiesof its product lines, EMC periodically releases revisions of its hardware andsoftware. Therefore, some functions described in this document may not besupported by all versions of the software or hardware currently in use. Forthe most up-to-date information on product features, refer to your productrelease notes. If a product does not function properly or does not function asdescribed in this document, please contact your EMC :This document was accurate at publication time. New versions of EMCdocuments might be released on EMC Online Support Check to ensure that you are using the latestversion of this TechBook is intended for EMC field personnel, includingtechnology consultants, and for the storage architect, administrator,and operator involved in acquiring, managing, operating, ordesigning a networked storage environment that contains EMC andhost Channel SAN Topologies TechBookPrefaceEMC Support Matrixand E-LabInteroperabilityNavigatorFor the most up-to-date information, always consult theEMC SupportMatrix(ESM), available through E-Lab Interoperability Navigator(ELN) documents include: The following documents, including this one, are availablethrough the E-Lab Interoperability Navigator,Documents >Topology Resource Center, documents are also available at the following location.

9 Backup and Recovery in a SAN TechBook Building Secure SANs TechBook Extended Distance Technologies TechBook Fibre Channel over Ethernet (FCoE): data Center Bridging (DCB)Concepts and Protocols TechBook Fibre Channel over Ethernet (FCoE): data Center Bridging (DCB)Case Studies TechBook iSCSI SAN Topologies TechBook Networked Storage Concepts and Protocols TechBook Networking for Storage Virtualization and RecoverPoint TechBook WAN Optimization Controller Technologies TechBook EMC Connectrix SAN Products data Reference Manual Legacy SAN Technologies Reference Manual Non-EMC SAN Products data Reference Manual RSA security solutions documentation, which can be found >Content LibraryAll of the following documentation and release notes can be found atEMC Online Support hardware documents and release notes include: Connectrix B series Connectrix MDS (release notes only) VNX series and CLARiiON Symmetrix VMAXEMC software documents include those on: RecoverPoint TimeFinder PowerPathFibre Channel SAN Topologies TechBook11 PrefaceThe following E-Lab documentation is also available: Host Connectivity Guides HBA GuidesFor Cisco and Brocade documentation, refer to the vendor s website.

10 Of thisTechBookThis TechBook was authored by Erik Smith, Aditya Nadkarni,Richard Hultman, and Dennis Kloepping, with contributions fromEMC employees, EMC engineers, EMC field personnel, and Smithis a Consulting Technologist for the Connectrix businessunit within EMC Engineering. Over the past 16 years, Erik has heldvarious technical roles in both EMC Engineering and TechnicalSupport. Erik has authored and coauthored several EMC is also a member of Hultmanis a Consultant Systems Integration Engineer andhas been with EMC for over 16 years. For the past 13 years, Rich hasworked in the E-Lab qualifying switch firmware and hardware. Richhas over 30 years of experience including designing personalcomputer hardware, symmetric multi-processing systems, and diskadapters, as well as developing firmware for disk Kloeppingis a a Principal Integration Engineer in EMC'sE-Lab Product Certification and Test group. Dennis has been withEMC for over 14 years, focusing on IBM zSeries FICON interconnectivity with EMC products.