[ad_1]
Energy claims for networking tools are sometimes obscure. It’s not at all times clear whether or not chip, card, or system energy is being acknowledged and what circumstances have been used for the analysis. If we’re going to pursue extra energy-efficient routers and switches, we have to perceive what that actually means.
Let’s take a step again and take into account a greater approach of preserving rating. In my view – and hopefully quickly yours too – these are the day-to-day system energy consumption and the provisioned energy (quantity and capability of PSUs to help redundancy and worst-case circumstances). This publish will give attention to exploring the parts of system-level energy consumption. Future posts will examine energy provisioning and organising a check atmosphere to standardize energy analysis.
Let’s begin by defining a unit for the metric. I suggest “watts per 100G with out optics” primarily based on system energy consumption. It represents a typical unit of bandwidth and (for high-end routers) at the moment yields numbers within the 1-50W vary that almost all of us are comfy with. Optics ought to be subtracted as a result of they’re extremely variable primarily based on module kind (e.g., 400G ZR+ is twice the ability of FR4). As optics could impression cooling necessities, a extra particular definition of “Watts per 100G at typical temperature with 2km optics minus optics module energy” is an efficient benchmark. Variations on this, corresponding to ZR+ optics or fewer playing cards or ports, may be outlined for operator-specific comparisons.
System energy
System energy consumption is the important thing metric to give attention to. Chip energy is an attention-grabbing strategy to evaluate gadgets, nevertheless it’s not related relating to paying the ability invoice or assessing environmental impression. Let’s have a look at what makes up system energy by reviewing the next key parts:
- Electrical information path
- Optics
- Cooling
- Mechanical design
- Management aircraft
- Losses in voltage conversion and signaling
Electrical information path
The important thing parts of {the electrical} information path energy comprise PHYs, retimers, and forwarding ASICs. PHYs are small gadgets used for features corresponding to MACsec or as gearboxes which mix Serializer/Deserializers (SerDes, used for quick I/O by way of PCBs) or cut up them into completely different speeds. A typical gearbox operation is to transform 2x 56G SerDes from the ASIC to 4x 28G SerDes related to a QSFP28 100 GbE port. Retimers are used to increase electrical alerts. Retimers are generally used for lengthy traces corresponding to connecting optics to ASICs within the “again row” of a line card.
Energy-wise, ASICs are the celebs of the present, so it’s comprehensible that they get many of the consideration. They’re essentially the most complicated parts and are the important thing driver of system energy for big, modular programs just like the Cisco 8800. ASIC energy will be damaged into three important classes: SerDes, idle core energy, and dynamic core energy. In high-bandwidth chips just like the Cisco Silicon One Q200, these are roughly equal. I’ll be specializing in this class of chips on this weblog. It consists of gadgets corresponding to Broadcom’s Jericho 2 (in NCS 5500) and Trident/Tomahawk (NCS 5000). Edge-class chips corresponding to Cisco’s Lightspeed Plus have comparable traits however eat extra energy in complicated per-packet operations (e.g., extra counters) and fewer in SerDes attributable to decrease bandwidth.
Every of those classes has completely different traits of energy consumption. SerDes require fixed energy impartial of visitors load, however they are often powered down to save lots of energy. On the 8000 Collection, this happens when ports are shut down or when a cloth or line card slot is empty. Within the latter case, the corresponding SerDes on the connecting card are shut down (e.g., some Cloth Card (FC) SerDes are disabled when Line Card (LC) slots are empty).
ASIC idle energy is strictly what it seems like. It’s the ability required to maintain the chip’s inside blocks up and operating with no visitors load. Dynamic energy happens when packets are forwarded. Some dynamic energy is consumed for per-packet operations (pps load) and a few is required for transferring the bandwidth alongside the forwarding path (bps load).
Core (non-SerDes) ASIC energy will differ drastically amongst gadgets as networking ASICs serve many roles with quite a lot of necessities. Core ASIC energy could differ primarily based on a number of elements:
- Operate – DC vs. SP Edge vs. SP Core carry out completely different ranges of per-packet operations
- Community vs. cloth capability – ASIC interfaces that solely can hook up with a swap cloth are a lot easier and decrease energy
- Course of node (7nm is the present technology, 16nm and 28nm have been the earlier TSMC nodes)
- Design – Newer microarchitectures are likely to have higher effectivity than chips carrying legacy constraints. The ability of the design engineers can also be vital and reveals up within the energy backside line.
- Machine temperature – A sizzling ASIC has marginally greater energy.
- Load – The variety of transistors flipping at a given time (primarily a perform of pps and bandwidth, but in addition of the design’s capacity to gate unused logic)
Most legacy SP ASIC designs have been optimized for big modular programs with swap materials. This method resulted in among the ASIC I/O (cloth requires speedup so often a bit greater than 50% to account for overhead) being arduous coded for cloth operations. This drastically simplifies the design relative to chips that may dedicate their full bandwidth to the community. The logic for cloth lookups is dramatically easier than community operations and, thus, requires much less silicon actual property and energy.
The draw back of fabric-bound I/O reveals up when utilized in single-chip programs. The material bandwidth is misplaced and can’t be redirected to the community. This may be offset by utilizing two chips back-to-back with a point-to-point “cloth”, however that’s nonetheless much less environment friendly than a single chip that may join all its bandwidth to the community. When two ASICs are required, the chip energy is doubled. If extra chips and a swap cloth are required for small (2-3 RU) programs, the ability goes even greater.
Optics
Optics are the divas of router {hardware}. They eat vital energy, present a small footprint for warmth sinks, and their max temperature is 50C cooler than ASICs. The thermal administration of optical modules was one of the crucial troublesome challenges for the preliminary mechanical design of 400G-class programs – after which ZR+ modules doubled the requirement from 11W (DR4/FR4) to 22W (ZR+).
Optical module energy is comparatively constant for every attain (e.g., LR8 or FR4) amongst distributors. The system energy to chill the modules varies. For instance, it could be tougher to chill a QSFP-DD module in a conventional chassis with vertical airflow (i.e., every module preheats air for the one above it) than a more recent orthogonal chassis (direct chilly air to all modules). The QSFP-DD and OSFP designs additionally impression energy required for cooling. Each can help excessive energy optics, however the QSFP-DD design permits for ongoing warmth sink innovation personalized to the chassis. This leads to much less airflow wanted to chill the modules and thus decrease system energy than OSFP which is locked in a standardized warmth sink design that restricted innovation.
Cooling
Routers are air-cooled; followers push or pull air by way of the chassis. The airflow path is essential as every part heats the air for downstream gadgets throughout the chassis. Fan energy is extremely variable amongst programs and completely different working circumstances. On the 8800 Collection’ largest chassis (the 8818) the 40 followers can eat from 600 to 6400W. This nice vary is because of the nature of fan energy consumption. It’s roughly cubic with the air moved. When fan speeds get into the 80-100% vary, their energy goes up dramatically so care ought to be taken to attenuate the necessity for high-speed fan operation by solely setting the followers to the utmost velocity when completely required.
Mechanical design
Cooling isn’t the one side of mechanical design. Different choices embody the cardboard top, quantity and dimension of followers, board layouts, warmth sinks, and extra. For instance, small variations in line card top can impression the dimensions of warmth sinks and the way constricted the airflow is. These each have an effect on chip temperatures and fan energy.
Management aircraft
The management aircraft consists of CPUs, DRAMs, SSDs, inside ethernet switches, and small parts corresponding to consoles and administration ethernet. CPU energy is comparatively low in comparison with ASICs. The opposite management aircraft parts don’t eat vital energy.
Losses in voltage conversion and signaling
The seen parts aren’t the one customers of energy. 5 to 10% of complete energy is misplaced when the ability provides convert the ability voltage (e.g., 48V DC or 220V AC) to the system voltage (e.g., 12V or 54V DC). Additional energy is misplaced because the system voltage is transformed to the degrees utilized by the electronics (e.g., 3.3V for optics). Energy can also be misplaced to the resistance of the PCBs as alerts transfer from gadget to gadget.
Up to now, these elements could possibly be ignored by customers, however as energy consumption turns into extra crucial, they shouldn’t be any longer. The 8000 Collection consists of many “behind the scenes” optimizations in these areas corresponding to new energy provides and extra environment friendly inside energy distribution.
Metrics for the Cisco 8000 Collection
Initially of this paper, the metric of watts per 100G with out optics was proposed. Let’s conclude by taking a look at these values for some particular configurations. The info under is examined at 22C with 768B packets, 15% bandwidth, and with out optics.
Conclusion
A lot is manufactured from chip-level energy effectivity, however that’s not the final word purpose – merely a method to an finish. Operators shouldn’t care if energy is used to flip transistors or spin followers. They simply need the system to be environment friendly. Evaluating programs ought to embody benchmarks that may be standardized in addition to network-specific configurations which can embody a number of generations of {hardware}, {a partially} stuffed chassis, or completely different visitors hundreds.
Within the 8000 Collection information sheets, the everyday energy is outlined by a completely loaded system at 25C with 15% of max bandwidth and 768B packets with out optics. For the 8800 Collection, the Q200-based 8800-LC0-36FH is used as the road card together with the latest swap cloth obtainable for the chassis.
Different Cisco IOS XR platforms use greater visitors ranges for his or her “typical” case in Cisco Energy Calculator and bundle LR4 optics on all ports. We made the adjustments for the 8000 Collection to raised replicate on a regular basis visitors patterns reported by our clients and from information we collected from manufacturing networks. If we elevated the visitors load to 512B and 50% max bandwidth, energy could be about 10% greater.
This will likely not signify your community or RFP benchmark, so for the 8800 Collection we’ve developed fashions primarily based on in depth lab testing that may present correct estimates for any mixture of line playing cards, kind and variety of cloth playing cards, packet dimension, bandwidth, optics, and temperature.
In abstract, the important thing metric for energy consumption is system energy in your typical circumstances. When router procuring, ask distributors for system-level information that meets your actual necessities and for particulars on how they have been examined or calculated. Be certain that to additionally look intently at timelines for qualification and ramp to deployment so that you just consider programs obtainable in the identical technology and don’t evaluate deployable {hardware} with press releases alone.
Be taught extra
At Cisco, we consider an inclusive future for all should take everybody under consideration, together with future generations. That’s why we’re setting long-term, actionable targets to handle the environmental impacts of our merchandise and operations. As a part of this effort, we’re targeted on rising power effectivity and decreasing emissions. Our sustainability initiatives embody greater than 440 power effectivity initiatives prior to now 5 years, and there are loads extra to return. We encourage you to spend a couple of minutes reviewing our journey towards a sustainable, inclusive future to grasp what this implies for you and your group.
Share:
[ad_2]
