Each year at OFC, we have the valuable opportunity to hear from the network operators themselves to better understand their vision for the future network. This year, there were some excellent presentations and discussions involving operators from several network verticals – Incumbents, Cable MSOs, and of course the ever-expanding cloud and colocation operators.
- Cloud and colo companies like Google, Microsoft, and Facebook are all moving to disaggregate the hardware components of their transport networks. Compact modular equipment should meet these needs – it is not evident that they will build WDM equipment in-house, yet.
- Microsoft has deployed at least 15k ColorZ units from Inphi and is pleased with the network model it enables. The company wants to build on this success and integrate longer reach WDM optics and its layer 2/3 switches.
- Google outlined a network model to build open line systems with three discrete building blocks and plans to deploy CDC ROADMs exclusively.
- Tencent, a major Chinese cloud company, has networks and requirements that mirror the west. Like Microsoft, the company favors tighter IP and optical integration.
- Verizon expects a slight ramp in the deployment of its Unified Transport metro-WDM network in 2018. It also will reduce the number of routers in its metro network by a staggering 90%.
- Quantitative details on cable MSO’s fiber deep deployments are still sparse, but the companies are embarking on a path that will eliminate specialty hardware wherever possible and rely on generic Ethernet and optical transport components.
Cloud & Colocation Operators
Facebook, Google, Microsoft, and Tencent all made presentations during Market Watch sessions or Operator Summits. Unlike the high-level presentations of years’ past, these discussions included detailed specifics on the companies’ network infrastructures.
Microsoft made several presentations during OFC; in fact, more than any other cloud operator. Mark Filer, part of the Azure optical network architecture group, outlined the current connectivity topology of Microsoft’s data centers (link to slides) and discussed its plans for transitioning to 400GbE. The company consistently messaged that it designs primarily for power, not cost. Often the indirect savings from lower power consumption will outweigh the direct costs of purchasing more expensive and exotic technology.
Inside the data center, the Microsoft has already invested in parallel fiber and plans to leverage this with DR4 optics. The company currently uses 100Gbs direct attach cabling (copper twinax) for intra-rack communication but anticipates a move to active optical cables (AOCs) for 400GbE.
Outside of the data center, Microsoft indicated it had deployed at least 15,000 ColorZ direct detect modules from Inphi (although we understand that Inphi has shipped more than twice this amount of modules to Microsoft). Microsoft is pleased with this solution and the savings it entails by integrating WDM optics into its switching equipment. The ColorZ plugs directly into the layer 2/3 equipment and eliminated standalone optical transport equipment from Infinera, Ciena, and others. Microsoft uses this solution for connections between data centers in a metro area (less than 80km) to create a relatively cheap mesh computing solution.
Microsoft holds that the ColorZ won’t scale to 400G and anticipates moving to the upcoming 400G ZR standard, which should plug into its Ethernet switches just like ColorZ does. For longer distances, Microsoft is looking at conventional WDM equipment but is also considering the possibility of embedded coherent optics directly in its equipment.
Google continues its network design based on simple building blocks that allow it to scale seamlessly. These “blocks” can be purchased from multiple vendors, which allows Google to make choices based on who is currently offering the best performance. Compact modular equipment is designed to serve this model, and Google said it prefers the fast innovation cycles it has observed from vendors of this equipment.
Google presented plans for implementing optical open line systems (OLS) during OFC’s Network Operator Summit. Open line systems are already in its long-haul transport networks, but Google wants to improve the operational model and implement the preferred modular approach. The company wants “open, modular line systems to unlock the best in breed building blocks.”
It envisions three discrete blocks: an in-line amplifier, a colorless, directionless, contentionless (CDC) ROADM for signal routing, and a drop-side ROADM for connecting transponders. CDC is fundamental, as the company wants to add and remove transponders and optical links without having to touch the cabling infrastructure.
Tencent, a Chinese cloud provider with market capitalization comparable to Facebook, gave one of the best presentations of the Data Center Summit. Tencent operates 80 data centers worldwide with over 1M servers and 70k network devices and provides numerous consumer services in China including the omnipresent WeChat.
Like its western counterparts, Tencent sees the fastest growth in its internal machine-to-machine networks, particularly in metro networks that mesh datacenters in a Chinese province capital region. Inside the data center, the company uses a combination of 100G SR4 and CWDM4 for intra-building and intra-campus connectivity while relying on AOCs for intra-rack connections. Tencent is just beginning a transition to 100G optics inside the data center and doesn’t anticipate a move to 400G until after 2019.
Microsoft and Tencent share a similar vision for IP and Optical convergence for WDM connections. Tencent highlighted the popularity of standalone OTN switching and transport equipment in China among the incumbent Chinese carriers. The company went on to say that this approach would not work for it and therefore it would build its own IP networks with coherent optics inside the routers.
For many years, Verizon operated separate, balkanized WDM transport and access networks; one for Fios broadband, another for Verizon wireless, another for enterprise connectivity, and even more as a result of mergers. The company recognized the enormous cost and complexity of this approach and sought to unify these networks – a project Verizon calls Unified Transport (UT).
Cisco and Ciena are the suppliers of the Packet-OTN switching equipment that Verizon is using to build out this network. Verizon still expects more deployment activity for UT in 2018 than 2017 but commented that a rapid ramp in deployment is not possible. In the words of one Verizon engineer: “we only have so many people and so many trucks to do the work.”
Verizon will also eliminate 90% of the edge routers from the network – a truly staggering number. Just as there were separate WDM networks for each Verizon division, there were also specialty edge routers designed to handle the traffic for that network. New router design with network function virtualization allows one piece of equipment to support a multitude of protocols, unifying this disparate traffic.
AT&T provided a brief update on its plans to disaggregate the design of its optical network. Like Google, the company wants to adopt a network equipment deployment model that provides more flexibility.
To do so requires solving several longstanding problems that most incumbent operators have: a lack of optical vendor interoperability, vendor lock-in, an inability to rapidly switch vendors to take advantage of innovation, and a requirement to build overlay networks when vendors are mixed.
AT&T outlined a 4-phase plan that will use white box hardware, first with client transceivers, then WDM transponders, followed by solving the WDM interoperability problem. The last phase is the deconstruction of the common equipment and implementation of an open line system. The company announced it would begin deploying hardware conforming to its Open ROADM initiative by the end of 2018, although this is later than planned.
Ultimately, AT&T believes these changes will not just give it better leverage over suppliers, but that they will allow it to build networks that are viable for longer periods of time and can adapt to the rapid pace of network technology.
Comcast also participated in the Network Operator Summit and reflected on how its network has changed over the past 20 years as optical fiber penetrated deeper into its access networks. Comcast is on the cusp of another change; the implementation of distributed access architecture (DAA) commonly referred to as “fiber deep.”
The company is active within the ONF to remake the hardware and software that comprises its network; moving from hardware with dedicated functions to virtualized hardware running on servers. Comcast is a big supporter of the ONF’s Open and Disaggregated Transport Network (ODTN) and plans to build its new transport network in this image, with transport and layer-2 switching components being essentially interchangeable from multiple vendors.
Absent from the Comcast discussion was a discussion of the enormous increase in the amount of Ethernet optics it will need as part of driving Ethernet-based protocols deep into its network. The exact size of this opportunity is not yet clear, but an initiative spearheaded by CableLabs (the CableMSO technology working group) is underway to standardize 100G coherent optics, essentially a slower speed variant of 400G ZR.