When the conversation at OFC turned to short reach client optics, faster speeds were the central focus. Several companies announced products and gave demonstrations of 400G capable technology. The 100G speeds and production ramp that had been such a hot topic at OFC 2017 had little follow-on during 2018.
Key Takeaways
- Despite enthusiasm at OFC 2017 for an extended run of positive returns, challenging profit margins arrived quickly in the 100GbE optics market. Companies are now trying to gain better leverage on existing investments, including exiting some product types and consolidating businesses.
- Eight lambda LR8 and ER8 400GbE solutions will reach the market this year, but a cost reduction is needed to trigger a mass migration from 100GbE in hyperscale applications. The four lambda LR4 and ER4 formats are the likely formats.
- There is an abundance of investment in PAM-4 DSPs with at least six companies developing silicon for the high volume LR4 and FR4 400GbE formats. This abundance will be positive for accelerating availability though it is clear not all these companies can get a return on their investment.
- Intel and others are trying to find other ways to solve the 400GbE client optics problem. Intel’s approach leans heavily on its 100G CWDM solution that is now in production. There is a lot of room for further innovation in this market.
- The OSFP vs. QSFP-DD format war is over, QSFP-DD won, but future 800GbE speeds will require a new approach.
100G QSFP Market Matures
A common belief at last year’s OFC was that there would be a shortage of 100G QSFP28 client optics for the remainder of 2017. Hyperscale operators were all migrating simultaneously to 100G inside the data center, and in March 2017 there was not enough supply to meet demand.
And yet, the shortage was not to be. Contrary to all presentations, discussions, and write-ups from last year, the 100G QSFP28 shortfall reversed itself by the end of 2017. At that point, the supply of the modules outstripped demand and pricing collapsed as component suppliers strove to move excess inventory.
At the same time, customer preferences shifted away from LR4 and towards lower cost CWDM optics. Some of the shift was due to the more widespread availability of silicon with IEEE 802.3 KR forward error correction (FEC). CWDM optics coupled with KR FEC enabled at the physical layer could reach as far as 10km, thus eliminating the need for LR capable optics. Since CWDM optics cost about a third of the price of LR optics, large cloud operators had a compelling incentive to abandon LR.
At OFC 2018, 100G QSFP28 discussions focused on how individual companies could use their existing technology or market edge to be profitable in today’s fiercely competitive market. Oclaro stated that it would focus on selling sub-components to other module assemblers, while Intel turned up the heat by shipping 100G CWDM4 modules in volume. Overall, OFC 2018 attendees acknowledge that the technology for 100G QSFP is mature and harder to differentiate than it is for emerging 400G applications.
400G Client Technology Trends
All of the OFC announcements and discussions on short reach interfaces revolved around the various ways to deliver 400G optics. Production is targeted to commence by the end of 2018, with new products designed to reduce cost following 12 months later. Two enabling components are required for 400G speeds to reach high volumes in the data center: Ethernet switch silicon and 2km client optics in a compact form factor.
Ethernet switch silicon will be ready this year and samples have already reached the market. Broadcom arrived first with its Tomahawk 3 (BCM56980) in December 2017, a 12.8Tbs Ethernet switch with 256 integrated 56G-PAM4 I/O. There are also other switch suppliers such as Innovium and Barefoot Networks. Given the relatively mature supply situation of 400G capable Ethernet switches, OFC attendees felt that the determining factors of when 400GbE volumes would ramp would be the availability of optics and the underlying PAM-4 silicon.
PAM-4 Silicon
For decades, all optical signaling was performed using non-return to zero (NRZ) modulation; the direct translation of binary information into the presence or absence of voltage or light. Electrical and optical signals were modulated on and off more and more quickly as speeds increased. This approach eventually broke down in the optical domain at 40Gbs, and the entirely new technique of coherent optical modulation (borrowed from the world of radio) was first introduced by Ciena commercially.
A similar transition is now underway in the electrical domain. PAM-4 uses multilevel electrical signaling to transmit two bits per period with the electrical voltage signaling four levels (0,1,2,3 volts for example) rather than two (on/off). This modulation increases the information rate of an electrical signal operating at 25GBaud from 25 to 50 billion bits a second. The PAM-4 modulation takes place in the I/O of the Ethernet switch chip or in a standalone PHY chip inside, or adjacent to, the optical module.
Inphi was an early technology leader for PAM-4 chips and is in production with 28Gbaud (56G-PAM4) PHY silicon. Broadcom, as well as FPGA companies Xilinx and Altera, also support these speeds. These companies will provide the first generation PAM-4 silicon designed to support 400GbE QSFP56 DD modules with eight optical channels – 400BASE-FR8 (2km) and LR8 (10km). These PAM-4 chips also enable an upgrade of the QSFP28 to 200GbE, a format of interest solely to Google.
Since the costs of optical modules tend to scale with the number of optical channels, one can make the approximation that an eight lambda module costs about twice what a four lambda module costs. If 400GbE is to be widely adopted, it needs to match the cost of existing 100G QSFP28-based solutions that are four lambdas. The current generation of PAM-4 doesn’t allow this, and the consensus is that FR8 and LR8 optical formats are not destined to be high volume solutions as a result.
There was a lot of buzz at OFC 2018 over announcements and technology demonstrations for the next speed iteration of PAM-4, which enables 400GbE over four lambdas. This iterative step requires doubling the baud rate on the optical side from 28Gbaud to 53Gbaud, thereby allowing a single lambda to carry 100Gbs of information. Bringing production grade silicon and optics to market over the next year is imperative for companies who want to be part of the transition to high volume 400GbE speeds at the end of 2019.
At least six companies are actively developing 53Gbaud PAM-4 silicon: Broadcom, IDT, Inphi, MACOM, Maxlinear, and Semtech. Maxlinear announced sampling of this silicon in January and Inphi announced the same at OFC this March. MACOM is also currently sampling, and it’s a safe bet that at least several companies will have working production silicon by the end of 2018. PAM-4 silicon availability should make 400GbE solutions based on single lambda 100G hit the market in 2019, thereby kicking off the next high volume transition of Ethernet speeds to 400G.
400G Optical Modules
A host of companies demonstrated first-generation 400G optical modules in FR8 and LR8 flavors at OFC 2018. These optics rely on readily available 28Gbaud PAM-4 silicon that transmits eight wavelengths using directly modulated lasers (DMLs). Finisar states that DML technology is a low-power, low-risk solution designed to get 400GbE to market this year, and other companies like Applied Optoelectronics would readily agree. Examples of this format were rampant on the show floor in both QSFP-DD and the larger CFP8 format. Several companies such as Kaiam and Finisar demonstrated active optical cables as well. All of these modules will be in production later this year and will allow data center operators to begin deploying 400GbE.
Remember, the four lambda client is now the low-cost, high volume endgame for 400GbE optics. Two target standards are the FR4 (2km) and LR4 (10km). Aside from the challenge of available low-power PAM-4 silicon to support the necessary 53 GBaud rate, there are significant hurdles with the optics. The requisite signal bandwidth of 53Gbaud PAM-4 eliminates the possibility of using DML lasers and requires the use of externally modulated lasers (EMLs). EML’s higher performance comes with greater power and complexity and is difficult to implement within the small confines of a QSFP-DD optical module.
Companies made use of OFC to demonstrate the ways in which they would overcome these challenges. Applied Optoelectronics, long a purveyor of DML solutions, presented its new EML laser technology designed to bridge its 100G-focused product line into the 400G era. Neophotonics announced a new module that integrates EML lasers, laser drivers, and receiver circuits as an all-in-one solution for companies seeking to assemble QSFP-DD LR4 modules. Finisar, Lumentum, Oclaro, Mitsubishi, and many more also had working prototypes of FR4 and LR4 QSFP28 modules.
Intel presented an interesting twist that leverages its silicon photonics heritage. It is championing an eight lambda CWDM solution for 400GbE called CWDM8 and has formed a consortium including companies like Applied Optoelectronics, Rockley Photonics, and Hisense. Intel feels its silicon photonics expertise lends itself to a low cost eight lambda solution with the performance to potentially reach as far as 10km. The design leverages its 100G CWDM4 product but uses a higher bandwidth design, adding in PAM-4 modulation and doubling the number of wavelengths.
Intel’s approach illustrates the rising competition between silicon photonics and InP based designs, and it will be exciting to see which technology ultimately succeeds. Four lambda 400GbE optical technology is transitioning out of the lab and into customers hands for evaluation, and by OFC 2019 it will be more apparent which companies and technologies have an edge.
QSFP-DD vs OSFP: Settled
The question of which module format – QSFP-DD or OSFP – would win out for 400GbE sparked debate at OFC 2017. Today, considering the investments already made and the work underway, it appears that QSFP-DD has won the debate. As long as QSFP-DD can deliver 400GbE operation across a variety of reaches and formats (which it now appears it can), companies and customers are hesitant to switch formats to OSFP.
Every module company is working towards QSFP-DD for its 400GbE designs, including the toughest specification of them all; 400G ZR. Companies commented that OSFP had more space and thermal margin and converting to that format would be easier than initially designing for OSFP and then having to then squeeze into a more constrained QSFP-DD module. Additionally, the QSFP-DD MSA group chose OFC to release a new thermal study indicating that QSFP-DD could operate as high as 15W. Even Arista CTO Andy Bechtolsheim, a long time supporter of OSFP, conceded the performance of QSFP-DD would suffice for 400GbE, with the exception of applications using AWG26 copper cable interconnect.
Yamaichi, a prominent connector manufacturer, said that the next speed transition – 800GbE – will obsolete QSFP-DD because the electrical specifications won’t be able to handle another speedup to 100 GBaud. 800GbE is the opportunity for OSFP to enter the market, though such a transition is probably at least three years away.