Cisco MDS 64G SAN Analytics: Structure evolution

February 16, 2022

287

[ad_1]

Cisco lately introduced software program availability of NX-OS 9.2(2) with help for SAN Analytics on the Cisco MDS 9700 Collection switches with 64G Modules. This software program launch begins the following section within the structure evolution of SAN Analytics.

On this weblog we are going to do a high-level comparability of SAN Analytics Structure between the Cisco MDS 32G and 64G platforms and have a look at a few of the new improvements of Cisco MDS 64G SAN Analytics.

However first, let’s cowl methodologies used for efficiency monitoring. Utilization, Saturation and Errors (USE) is a generic methodology for efficient efficiency monitoring of any system. The USE metrics establish efficiency bottlenecks of a system. Within the context of a storage system, we will add Latency as an extra ingredient into the USE methodology to create LUSE. A full visibility into LUSE metrics of a storage infrastructure is essential for efficiency monitoring and troubleshooting.

SAN Analytics and SAN Insights are advance options of the Cisco MDS 32G switches since NX-OS 8.3(2):

SAN Analytics is an advance function of Cisco MDS switches that collects storage I/O metrics from switches unbiased of host and storage techniques. Over 70 metrics are collected per-port, per-flow (ITL/ITN) and streamed out. These metrics could be categorized into one of many ‘LUSE’ classes.

SAN Insights is a functionality of Cisco Nexus Dashboard Material Controller (Previously DCNM) SAN that receives the metrics stream from SAN Analytics. It gives the visualization and evaluation of cloth extensive I/O metrics utilizing the ‘LUSE’ framework.

Cisco MDS 32G SAN Analytics

Entry Management Lists (ACL) implement entry management on each body switched by the ASIC. The ACLs are matched extracting sure fields from the body header and on a match the motion equivalent to the entry is taken. On an F-port, FC Exhausting Zoning entries are programmed as ACLs within the ingress path primarily based on Zoning configuration to match on the body SID and DID with an motion to “ahead” the body to the vacation spot.

On Cisco MDS 32G switches, the I/O metrics are computed by capturing FC body headers within the information path utilizing an ACL primarily based ‘Faucet’ programmed within the ASIC on ingress and egress path of the analytics enabled ports. These Faucet ACLs match on frames of curiosity for Analytics viz. CMD_IU, 1^st DATA_IU, XRDY_IU, RSP_IU and ABTS. A duplicate of the body matching the Faucet ACL is forwarded to an on-board NPU related to the 32G ASIC.

When SAN analytics is enabled on a port, the ACLs are programmed relying on the port kind and path as proven in Determine 1 beneath:

F_Port Ingress: Analytics Faucet ACLs + Zoning ACLs

F_Port Egress, E_Port Ingress, E_Port Egress: Analytics Faucet ACLs solely

Determine 1: Port Analytics Faucet and Zoning

The Cisco MDS 32G NPU software program Analytics Engine could be modified to accommodate customized metrics (Eg: NVMe Flush command metrics) or futuristic storage command units (Eg: NVMe-KV) with the required ACL Faucets in place.

Cisco MDS 64G SAN Analytics

The Analytics Engine strikes into the ASIC on Cisco MDS 64G switches, giving it a {hardware} acceleration. The Cisco MDS 64G Module has two 64G ASICs and every ASIC has six {hardware} Analytics Engines (one for each 4 ports). These Analytic Engines can compute I/O metrics at line fee on all ports concurrently with capability to research upwards of 1 billion IOPS per Module. The {hardware} Analytics Engines have built-in Faucets and don’t want the ACL primarily based Faucets to be programmed.

The metrics computed by {hardware} Analytics Engines are saved in a database contained in the ASIC and periodically flushed to the NPU. The NPU runs a light-weight software program course of on high of DPDK (an open supply extremely environment friendly and quick packet processing framework) that collects and accumulates the metrics pushed periodically from the {hardware} Analytics Engine. Although the NPU doesn’t run an Analytics Engine, it maintains the persistent metrics database per-flow and stays the essential ingredient of the answer. The transport of metrics from the NPU database to the Supervisor is an identical to the Cisco MDS 32G Structure. The Cisco MDS 64G {hardware} Analytics Engine doesn’t preclude a NPU software program Analytics Engine to be enabled in a future software program launch for flexibility and programmability advantages.

A comparability of the Cisco MDS 32G and MDS 64G architectures are proven in Determine 2 beneath:

Determine 2: Cisco MDS 32G and MDS 64G SAN Architectures

The Cisco MDS 64G {hardware} Analytics Engine computes some further metrics for deeper I/O visibility:

Multi-sequence write I/Os are giant writes involving a number of XRDY sequences. The write alternate completion time for these writes embody delays launched by the Host (Rx XRDY_n to Tx first DATA_n+1) and the Storage (Rx Final DATA_n-1 to Tx XRDY_n). These metrics present higher evaluation and correct pinpointing of enormous write efficiency points. The Analytics Engine individually tracks:
- Avg/Min/Max host write delay
- Avg/Min/Max storage write delay

The overall busy time metric tracks the whole time there was at the least one excellent I/O per-flow. This metric helps to characterize the ‘busyness’ of a stream relative to different flows.

The {hardware} Analytics Engine by default tracks SCSI and NVMe I/O metrics at ITL/ITN granularity. Nonetheless, it will also be programmed to trace metrics for varied stream granularity of IT, ITL-VMID, ITN-NVMeConnectionID or ITN-NVMeConnectionID-VMID. This provides flexibility in selecting the granularity of metrics and I/O visibility.

The 1GbE analytics port on the Cisco MDS 64G Module can stream the per-flow metrics immediately (with out involvement of Supervisor) in an ASIC native or customary gPB/gRPC format. This may serve future use-cases that require visibility into micro telemetry occasions, which might require excessive frequency telemetry streaming.

Abstract

The Cisco MDS SAN Analytics and SAN Insights is a key answer to observe and troubleshoot efficiency issues within the MDS FC SAN utilizing a ‘LUSE’ or any equal methodology. The Cisco MDS 64G platforms (working at any pace) now comes with a {hardware} Analytics Engine that may compute I/O metrics at line fee on all ports. The Cisco MDS structure is the business’s most versatile, programmable, scalable, and future proof SAN options with no forklift improve of chassis or rip and substitute to undertake the newest SAN improvements.

To study extra, go to Cisco SAN Analytics and SAN Telemetry Streaming Answer Overview

Cisco MDS 9000 is constructed to satisfy right this moment’s calls for whereas accommodating future innovation. The Cisco MDS structure is the business’s most versatile, programmable, scalable, and future proof SAN options that help Multi-Era and Multi-Velocity interoperability of current 16G, 32G, and new 64G line playing cards in current chassis for sleek migration and adoption of the newest SAN improvements.