QFX5200 25GbE Fiber Optic Interface Interoperability Testing

1 2017 Juniper Networks, Inc. QFX5200 25 GbE Fiber Optic Interface Interoperability Testing Tech Note October 2017 Juniper Networks, Inc. 1133 Innovation Way Sunnyvale, California 94089 USA 408-745-2000 Copyright 2017, Juniper Networks, Inc. All rights reserved. QFX5200 25 GbE Fiber Optic Interface Interoperability Testing 2 2017 Juniper Networks, Inc. Table of Contents Introduction .. 3 Importance of Interoperability Testing .. 3 Forward Error Correction (FEC).. 4 Technical Overview .. 4 Implementation .. 5 Configuration .. 6 Verification .. 6 Interoperability Testing Methodology .. 7 Internal Test Results .. 7 Optics/Cable Testing .. 8 Interoperability Test Matrix .. 8 University of New Hampshire Interoperability Lab Test Results .. 9 Summary .. 10 Appendix .. 10 Test Methodology for Internal Testing .. 10 QFX5200 25 GbE Fiber Optic Interface Interoperability Testing 2017 Juniper Networks, Inc.

2 3 Introduction The explosive growth of cloud applications, video and mobile traffic is driving the demand for faster and faster server uplink speeds. The 10 GbE sever access connectivity is getting saturated as operators add more workloads or VMs per server. Increasingly network architects are looking to the emerging 25 GbE standard as a solution for their increased bandwidth requirements. A number of factors make 25 GbE attractive to datacenter operators. The 25 GbE standard doesn't require a new cable infrastructure; it runs over the same 10 GbE cable installation. Compared to 40 GbE, 25 GbE is both more power efficient and cost effective. Since it s designed backward from 100 GbE technology using 4x25 GbE SERDES, it requires only 1x25 GbE SERDES just like the 10 GbE. Juniper Networks QFX5200 switches offer 10 GbE, 25 GbE, 40 GbE, 50 GbE, and 100 GbE interfaces.

3 Users can have 100 GbE by default or can also configure the interfaces as 40 GbE. 100 GbE can be channelized to two independent 50 GbE or to four independent 25 GbE (the channels are often called lanes ). Similarly, 40 GbE can be channelized to four independent 10 GbE using breakout cables. One of the advantages of 25 GbE is that 4-lane 40G NIC has the same price as 2-lane 25 GbE NIC, which means customers get higher throughput for the same Fiber count. Importance of Interoperability Testing Why is Interoperability Testing so important? Almost everything implemented today in networking is covered by some type of standard, either a commonly seen de facto standard or legally established de jure standard. For example, in the United States, locating the car steering wheel on the left or right is a de facto standard, and many postal vehicles locate the steering wheel on the right.

4 However, in the United States, driving on the left side of the road is a de jure standard governed by law and a violation (and dangerous). If one vendor says my product complies with all relevant standards and another vendor says my product also complies with the same standards, then there should be no need to test for Interoperability , right? Not necessarily. There are at least five reasons why a simple declaration of standards compliance is not a guarantee of Interoperability . 1. Standards are interpreted This is always the number one reason for Interoperability issues. Engineers ponder the text and often puzzle over the precise meaning of a term or condition. Sometimes different groups read and interpret things the same way, but often they do not. 2. Standards have options Options allow standards to apply in different situations. However, if a vendor applies Option Set A and another applies Option Set B, it s not always clear what they QFX5200 25 GbE Fiber Optic Interface Interoperability Testing 4 2017 Juniper Networks, Inc.

5 Have done (they often list their product as complying with the overall standard, and in general it does), or what might happen when they try to talk to each other. 3. Standards are incomplete Standards do not always cover every circumstance and application. Technology moves quickly, and many RFCs and ITU specifications include extensions or addenda to cover things that have come up once implementation begins. 4. Standards are wrong It s better to say standards have typos, or that long standards sometimes seem to have later sections that appear to contradict earlier ones (see Standards are interpreted above). Many RFCs and ITU specifications have errata attached the correct unintentional errors. 5. On purpose This does not imply that a vendor is trying to be difficult. Sometimes a vendor will purposely restrict operation to its own product groups until the standardized product is more mature.

6 The vendor might also be trying out an extension to the standard that might be added to the standard at a later time, but which for now is not supported by other vendors. In a perfect world, all of the cases outlined above would have documentation and would be well known. But often they are not completely known to everyone. In fact, it is often Interoperability Testing that detects many of the conditions listed above in the first place. Put these reasons all together, and you see why Interoperability Testing is essential, especially for newer technologies. Forward Error Correction (FEC) Technical Overview As systems get faster signal integrity becomes critical, both internally and over transmission links. New 25 GbE and 100 GbE systems are a good example of this. Such systems are expected to have lower tolerance for latency and errors. There are many techniques that can be used to enhance data reliability.

7 To correct for errors and protect the data integrity, FEC is used in 25 Gbps and higher speeds to improve data transfer throughput without the need for re-transmitting the data. FEC is a digital pre- and post-processing technique that greatly reduces the number of uncorrected errors and helps to extend the reach capability of both Copper and Optical PMDs. There are a number of FEC modes, offering differing degrees of error detection and recovery. The IEEE standard has defined Clause 91 with Reed-Solomon FEC (RS-FEC) for 25Gb Ethernet to support most of the copper and optical physical media dependent devices (PMDs). Clause 74 of the 100 GbE standard specifies BASE-R FEC also known as fire-code FEC (Fire code has nothing to do with flames, but is named for Philip Fire, who described this method in 1959), and is available in most 100 GbE capable switch silicon.

8 FC-FEC offers a weaker error correction than RS-FECs, especially with random errors instead of bursts, but with lower latency compared to RS-FEC. QFX5200 25 GbE Fiber Optic Interface Interoperability Testing 2017 Juniper Networks, Inc. 5 To enable FEC on a communication link, the same type of FEC has to be configured and enabled on both ends of the link. Figure 1: FEC Pre- and Post-Processing. FEC was developed originally for situations where re-transmissions are expensive or impossible, such as broadcast and multicast communications or for deep-space probes. As the error detecting and correcting capabilities of FEC increase with faster and more powerful pre- and post-processing, data transmission over noisy channels became more reliable and all but error-free. This use of FEC leads to improved bandwidth efficiency, lower costs, and higher achievable data rates.

9 In fact, without FEC, many higher-rate line speeds would be impossible to sustain. Implementation Starting with Junos OS Release , forwarding error correction (FEC) is supported on QFX5200 switches; users can disable or re-enable FEC clause 74 (CL74) as well as FEC clause 91 (CL91) using CLI commands. FEC CL74 is supported for 25 Gbps port configuration, when the ports are channelized either in 4x25 GbE or 2x50 GbE, FEC CL74 is enabled. FEC CL91 is supported for the 100 GbE configuration and enabled by default. QFX5200 25 GbE Fiber Optic Interface Interoperability Testing 6 2017 Juniper Networks, Inc. Configuration To disable FEC: set interfaces < Interface -name> gigether-options fec none To enable FEC: set interfaces < Interface -name> gigether-options fec <fec91/fec74> or delete interfaces < Interface -name> gigether-options fec none Verification To check FEC status: user@ QFX5200 > show interfaces et-0/0/0 Physical Interface : et-0/0/0, Enabled, Physical link is Up Interface index: 649, SNMP ifIndex: 505 Link-level type: Ethernet, MTU: 1514, LAN-PHY mode, Speed: 40 Gbps, BPDU Error: None, Loop Detect PDU Error: None, Ethernet-Switching Error: None, Source filtering: Disabled Ethernet-Switching Error: None, MAC-REWRITE Error: None, Loopback: Disabled, Flow control: Disabled, Media type: Fiber Device flags : Present Running Interface flags: SNMP-Traps Internal: 0x4000 Link flags : None CoS queues.

10 10 supported, 10 maximum usable queues Current address: 5c:45:27:37:55:c3, Hardware address: 5c:45:27:37:55:c3 Last flapped : 2014-04- 27 08:14:22 PDT (00:00:57 ago) Input rate : 0 bps (0 pps) Output rate : 0 bps (0 pps) Active alarms : None Active defects : None PCS statistics Seconds Bit errors 0 Errored blocks 0 Ethernet FEC statistics Errors FEC Corrected Errors 0 FEC Uncorrected Errors 0 FEC Corrected Errors Rate 0 FEC Uncorrected Errors Rate 0 PRBS Statistics : Disabled Interface transmit statistics: Disabled Logical Interface et-0/0 (Index 554) (SNMP ifIndex 506) Flags: Up 0x4004000 Encapsulation: ENET2 Input packets : 0 Output packets: 0 QFX5200 25 GbE Fiber Optic Interface Interoperability Testing 2017 Juniper Networks, Inc.