Mellanox Tailor Made Network Operating System

October 16, 2018

Running Docker Containers on Top of Mellanox’s SN2000 Spectrum™-based Ethernet Switches 

Modern data-centers have evolved from monolithic architectures running applications on a single host or virtual machine into lightweight Linux-based containers.

Containers and Docker

A container image is a lightweight, stand-alone, executable package of a piece of software that includes everything needed to run it: code, runtime, system tools, system libraries, and settings. Available for both Linux and Windows based applications, containerized software will always run the same, regardless of the environment. Containers isolate software from its surroundings, for example differences between development and staging environments, and help reduce conflicts between teams running different software on the same infrastructure.

Docker is the world’s leading open source software container platform. Docker provides an additional layer of abstraction and automation of operating-system-level virtualization on Windows and Linux. Docker allows independent “containers” to run within a single Linux instance, avoiding the overhead of starting and maintaining virtual machines.

Mellanox’s Spectrum™-based 25GbE and 100GbE switches offer end-users the ability to create their own unique differentiation since they are only solution in the market to enable
tailored applications by direct access to Spectrum SDK on top of Docker containers.

Spectrum Added Value

Container support in Mellanox Onyx™ (Mellanox operating system) or Cumulus Linux offers customers the flexibility to load container-based applications directly onto the Mellanox Spectrum platforms. Running Docker containers over Spectrum simplifies the development and deployment of applications on switches. It also allows deployment of any of the thousands applications available from Docker Hub.

SN2000 platforms are the only solution in the market to enable direct access to system resources via the Mellanox open SDK API:

• Tailor your own application in an isolated container on top of Spectrum SDK
• Fast integration and validations cycles for new features on top of Spectrum SDK
• Enjoy all of the Spectrum switch capabilities and its features set even if not yet supported by a NOS
• Persistent – up after reload with the same parameters and configuration
• Share selected storage spaces between various containers and the Onyx operating system itself

Docker Use Cases Using Spectrum

Use Case 1 – Telemetry

Network change cycles are growing increasingly shorter and more frequent. As a result, operators need as much visibility into the network as possible to help them identify, isolate and resolve potential issues as quickly as possible to keep their networks running smoothly. A network with increased and real-time visibility enables better reliability and real-time control.

Spectrum provides real-time hardware buffer monitoring. The architecture enables the switch to detect delays that occur in traffic flows when packets are buffered. Detecting the presence of any congestion at the hardware level provides useful visibility into the switch. Awareness of the location, depth and duration of the congestion enables for additional optimizations. Indications like buffer level, latency and bandwidth supporting watermarks, histograms and thresholds while streaming data to a local CPU. Docker on top of Spectrum switches gives the user a unique ability to collect, alert and analyze in real time, any event or alert supported by Spectrum and streamed out to visualization tools, such as Grafana, to gain better insight into metrics and enable fabric optimizations (and thus increasing revenue).

Use Case 2 – Automotive

Modern cars and soon to come autonomous vehicles are small data centers with dozens of end customers (i.e. sensors, cameras) with need for fast and synced interconnect in order to ensure a safe ride.

The automotive industry mandates the unique but rarely implemented 802.1AS networking protocol which ensures not only precise timing and synchronization between network components, but also high bandwidth performance. Spectrum’s extensive accessibility allows end-users to achieve a competitive edge even faster than the switch vendors themselves by implementing 802.1AS on top of Spectrum.

Running tailored applications allows users to achieve the needed clock accuracy which is cardinal for the application to work properly and accurately. The container can configure the SDK to get the unique PTP packets from the network to enable a fast and safe integration of new code.

Use Case 3 – Software Defined Storage (SDS)

Software-defined storage is a marketing term for computer data storage software for policy-based provisioning and management of data storage independent of the underlying hardware. Software-defined storage typically includes a form of storage virtualization to separate the storage hardware from the software that manages it. The software enabling a software-defined storage environment may also provide policy management for features such as data deduplication, replication, thin provisioning, snapshots and backup.

Here, a customer is building his own tailored SDS application using containers running on top of each of the switches in the fabric using a clientserver/master-client architecture. Each of the clients monitor the switch status by polling all of the Spectrum counters exposed through the SDK open APIs. Moreover, the master can manage and configure the fabric according to its pre-defined policies.

How To Run Docker Over Spectrum

Docker can be installed directly on a Spectrum switch running either Onyx or Cumulus Linux, and Docker containers can be run natively on that switch.
To run Docker in Onyx:

To run Docker on top of Cumulus Linux, please refer to: https://docs.cumulusnetworks.com/display/DOCS/Docker+on+Cumulus+Linux.

Note: MLNX-OS 3.6.4006 and Cumulus Linux 3.3 or higher support the Docker engine. Mellanox Onyx was previously named MLNX-OS for Ethernet.

Pave Your Own Path

Mellanox Spectrum switches are the only solution in the market to offer a full package to support speeds from 10GbE up to 200GbE with a full set of Layer 2 and Layer 3 features and unique capabilities, with no compromise on performance. With the latest addition of Docker containers access to Spectrum SDK API, Spectrum is the only solution to enable direct access to switches and provide DevOps-users with flexibility, faster time-to-market, and a unique edge when developing tailor made next-generation applications.