Would you like CPUs with that sir?

These days you can stick a CPU on just about anything to make it ‘smart’. You’ve probably heard about Smart NICs, Infrastructure Processing Units (IPUs) or Computational Storage as concepts, but these technologies are being rapidly developed and will be let loose on the mainstream market soon. So, what’s it all about?

Let’s start with Smart NICs (aka IPUs) as the idea here was quite simple. Placing a CPU and memory on a NIC means that you can offload network processing functions to where the networking happens, not contend with the apps and services running on the central motherboard for a slice of processing resources. It’s a win-win, network latency and app performance are improved.

Smart NICs were rapidly adopted by hyper-scalers and telcos to enable them to offload virtual switching and network security functions to the Smart NIC. The more general adoption of Smart NIC technology may not necessarily come from this most simple of use cases though.

Many of you will have heard about Project Monterey at VMware. The idea here is to install the VMware hypervisor onto the Smart NIC and offload the VMware Cloud Foundation (VCF) functions to it, namely NSX virtual networking and security and vSAN virtual storage, as per the illustration below from VMware’s Project Monterey blog.

This provides a composable option for the central server resources, it could be a virtualised host or it could be a bare metal Windows or Linux server – either way it will consume networking and storage services from the Smart NIC, which will also manage the central server resources. With VMware’s ESXi hypervisor now supporting Kubernetes workloads as well as traditional VMs (VMware vSphere with Tanzu) the Smart NIC-enabled technology enables you to build a completely new kind of software-defined data centre of uniform hardware blocks that can be composed to host virtual machines, Kubernetes clusters and physical machines, all on mainstream server hardware. The intended outcome is accelerated infrastructure performance with 100% of the motherboard resources available for processing of applications and data.

In August 2021 an Early Access controlled release of Project Monterey became available, comprising of Dell PowerEdge server hardware, NVIDIA smart NICs and VMware ESXi software. It will be interesting to see what customer benefits are realised at the end of this program and what demand that generates for its general release – will this be a niche or mainstream technology?

As well as sticking CPUs on network cards, CPUs have also been put onto storage drives, giving us Computational Storage Devices (CSx). Like Smart NICs the idea is to offload processing from the host to the storage devices, freeing resources on the host while reducing the latency of data processing. Unlike NICs servers have many storage drives and there is a linear increase in computational power with each additional CSx added to a host. The following diagram taken from a VMware blog illustrates the concept.

There are many different use cases cited for the in-situ processing capability of CSx, from creating intelligent edge devices that directly process incoming raw data to resilient accelerated database servers with resilient clusters within a single host. More likely initial applications will just perform simple offloaded tasks, like real time encryption.

VMware is working with NGD Systems in this space, exploring what options putting the VMware hypervisor on storage drives brings – from low-latency database and ML query handling directly on the data to hosting an entire vSAN cluster within a single server. So, forget hyper-converged infrastructure, this could be the start of the intra-converged infrastructure, hosted within standard server hardware.

However, all of these technologies pan out, it’s clear that the data centre architecture is going to get disrupted again with composable infrastructure sitting inside standard server hardware, data processed in real time and data clusters within a single server host.

To continue this discussion and envision the positive impacts these emerging technologies could have on your business, contact us at Logicalis.