Part 1: Benefits and Challenges of Pluggables in Routers
Over the past several years, 400ZR and IP-over-DWDM have been prominent topics at OFC. 2021’s show featured concepts and prototypes, while 2022’s OFC focused on shipment announcements and interop testing. The question for OFC 2023 is – will 400ZR and its variants (collectively referred to as 400ZRx) finally usher in the long-awaited era of IP-over-DWDM?
In his 2014 bestselling book, Crossing the Chasm, Geoffrey Moore describes the adoption lifecycle of new technologies through the preferences and characteristics of different types of consumers. Many technologies often find adoption among an enthusiastic set of technology visionaries (early adopters) but ultimately fail to gain widespread market success. Moore called this yawning gap between early and mainstream adoption “The Chasm,” and its depths are littered with new technologies that, for one reason or another, were never able to make the leap.
IP-over-DWDM – integrating WDM optics directly into routers – is a concept as old as optical networking. Pioneered by Cisco in the early 2000s, it threatened to upend the way optical networks were built and operated but failed to gain traction outside of enterprise customers and innovative service providers. In 20 years of promotion, it never crossed the chasm.
But 400ZRx optics promise to change that. Using the router-friendly QSFP-DD form factor, IP-over-DWDM no longer imposes a port density penalty like past generations of router-hosted optics. 400ZRx timing coincides with the adoption of 400GE routers and interfaces instead of lagging by several years. And it’s interoperable, with specifications from OIF and OpenZR+ now proven to function as designed. Plus, a competitive landscape of vendors now exists to ensure supply and drive down costs.
400ZRx is already very successful. With more than 200,000 units shipped in 2022, its uptake has far outpaced the rate of previous coherent generations. But the vast majority of shipments have gone to a single customer type and application – webscale metro DCI. Far from disrupting telecom, this application is all about cloud scale and cloud economics and won’t necessarily translate into success among service providers.
However, according to “Crossing the Chasm,” initial success within a niche is a key prerequisite to widespread adoption among a broader user base. It just so happens that webscale metro DCI is a niche in which a very small set of customers drive enormous volume. The chart above compares yearly shipments of all coherent generations based on years from their respective introduction. In its third year of shipment, 400ZRx optics exceeded the year-three shipments of all other generations combined. Of course, total transport capacity is much higher now than when earlier generations were introduced. However, 400ZRx is also forecasted to represent more than 30% of total ports shipped in 2023, the largest share of any generation. Perhaps just as important as the proof point of mass adoption is the influence webscale operators have on the telecom market. The early majority looks to market leaders for guidance, and webscale operators’ technical prowess and leadership are well-established.
As we approach OFC 2023, what advancements and innovations will we see on the show floor that could propel IP-over-DWDM across the chasm?
ROADM Compatibility with High-Output Power
One of the shortcomings of 400ZRx optics hindering adoption among service providers is low launch power. Most coherent interfaces transmit at 0dBm or higher, whereas initial 400ZRx optics provide a much lower -10dBm. This isn’t a problem in the simple, point-to-point networks predominating in webscale DCI applications. Still, it presents an obstacle to adoption in the more complex, ROADM-based networks common among service providers, particularly in brownfield environments. At last year’s OFC, Coherent (then II-VI) and Acacia demonstrated high-output power versions of 400ZR+ optics in a QSFP-DD form factor (CFP2 versions already exist but are primarily intended for transponder applications). Since then, both module vendors have announced successful trials, and Acacia noted first customer shipments. Ciena and Infinera have indicated similar products are in the works. The additional complexity of high output power 400ZR+ optics likely means fewer suppliers and higher prices compared to -10dBm suppliers, where at least ten vendors are supplying 400ZRx DSPs, and the number offering complete modules is expected to grow significantly. Is the supplier base and cost of high output power 400ZR+ optics sufficient to drive adoption into service provider networks? Look for announcements from additional vendors and further indications of progress from Coherent, Ciena, and Infinera.
Long Haul Capability via Higher Baud Rate
High output power 400ZR+ optics solve the problem of brownfield compatibility, but they don’t address limited reach. Gen60 optics top out at roughly 1000km when running at 400G – more than enough for most metro/regional applications but insufficient for long-haul backbones. Embedded Gen90P (90 Gigabaud, high-performance) optics from Ciena, Infinera, and Nokia currently address 400G+ over long-haul distances, but the industry appears to be coalescing around 120+ Gigabaud optics as the next step for both performance and compact optics. OFC 2023 is sure to provide announcements and demonstrations of Gen120P interfaces, but the status of Gen120C (compact, pluggable) optics for IP-over-DWDM is uncertain, as products have yet to be announced.
Gen120C optics will likely adopt the 800ZR/ZR+ naming convention and will indeed transport 800-gigabit ethernet in metro DCI applications, but they will also be used in carrier networks to transport 400G over long-haul distances. Many operators will value the consistency of using IP-over-DWDM for all their network links, boosting adoption. And just as the OIF 400ZR specification served as the foundation for 400ZR+, the 800ZR specification will likely be the precursor to 800ZR+ variants built to address specific applications. The 800ZR implementation agreement isn’t yet complete, but it would be surprising if there were no announcements regarding Gen120C optics at OFC. Just please don’t call them 400ZR++.
Transport-friendly management and operations
The historical failure of IP-over-DWDM to cross the chasm can be attributed as much to its violation of network operational norms as to its technical shortcomings. Transponders provide a clear point of client demarcation and management responsibility. They also make available a vast set of layer-1 performance monitoring metrics and provisioning parameters which have traditionally been lacking in pluggable optics and the routers that host them. For service providers, provisioning and assurance are the lifeblood of customer satisfaction, and the potential cost savings of IP-over-DWDM aren’t necessarily enough to overcome the operational pain of its implementation. The industry is tackling these challenges through a combination of module-level standards and proprietary management applications; expect this to be an active topic of conversation at this year’s OFC.
Along with its implementation agreement for 400ZR, the OIF specified the Coherent Common Management Interface Specification (C-CMIS). This quasi-standardization facilitates the integration of different manufacturers’ 400ZR modules into host platforms and provides some physical layer intelligence, but it falls short of providing the degree of management visibility and control that optical network operators are accustomed to. It also does nothing to address the issue of separate and distinct IP and optical operations that are often found within service providers.
A complementary and more advanced method is being proposed by Infinera via the Open XR forum, in which pluggable optics are enhanced with a broad set of transport functions typically found in a dedicated transponder. The company proposes an architecture in which a lightweight agent runs on the host platform, passing messages to/from a centralized optics controller directly to the module. This has the advantage of decoupling the capabilities of the optics from specific support by the host router, as well as providing a method of management separation.
Vendors such as Cisco, Nokia, and Ciena, aided by their portfolio of both routers and optical transport platforms, tend to promote unified management of IP/optical networks. This multi-layer functionality is integrated into their SDN controllers and/or NMSs and enables the supposed panacea of “single pane of glass” management of a converged architecture. However, acknowledging the likely persistence of separate IP/optical operations, such solutions also generally support the concept of a virtualized transponder, in which router-hosted pluggables can be managed by transport operations independent of and isolated from routing functions. This approach relies upon standardized router APIs, as opposed to the potentially nimbler and more feature-rich direct module communication proposed by Open XR.
Can IP-over-DWDM cross the chasm?
400ZRx optics are here to stay. They are already finding widespread use among the largest cloud operators, enterprises, and early adopter service providers. But for IP-over-DWDM to become a permanent part of the optical transport landscape, it must win over the slow-moving and notoriously conservative Telcos. ROADM compatibility, long-haul capability, and transport-friendly management are just a few of the advancements we’re likely to see at OFC 2023 as IP-over-DWDM attempts to make the giant leap across the chasm into mainstream adoption.