Cloud Edge Use Cases Part 3: Cloud Edge Machine Learning

In part 1 and part 2 of this blog series, we mentioned the three use cases for the cloud edge (or the new middle mile) – edge multicloud networking, storage repatriation, and cloud edge machine learning. We discussed both the edge multicloud networking and storage repatriation use cases in more detail as well.

In this blog, we will cover the remaining and perhaps the most exciting use case – cloud edge machine learning.

Today, by and large, users run machine learning inferencing or light training such as for large language models i.e. LLM on-prem or in the public cloud. Both approaches have pros & cons.

What if there was a third-approach? Here the machine learning processing would occur at the cloud edge. See blog titled “AI and 5G Are Better at the Edge” by Oleg Berzin and Kaladhar Voruganti that addresses this scenario. While the blog links machine learning at the cloud edge to 5G, most of the blog applies to other access technologies as well. The authors cover three use cases – smart parking, AR/VR for predictive maintenance, and V2X and make a compelling case for the cloud edge.

In my view, the benefits of performing machine learning processing at the cloud edge include:

• The ability to process data close to where it gets produced; this location is not as ideal as on-prem from a data proximity point of view, but it is a lot better than the public cloud. For the use cases listed above and for computer vision applications, the cloud edge can be a boon.
• Ease of use features at part with the public cloud.
• OPEX model.
• On-demand, pay for what you use.

The benefits are further magnified when we talk about generative AI:

• By using open source models such as Llama or Dolly, the user can have full control over the LLM model.
• 0% (ZERO) probability of data leakage – one of the biggest fears of commercial LLMs is that a company’s intellectual property might leak into public models. By using a private model, there is 0% probability that sensitive IP leaks into the public domain.
• Given that the cloud edge can be easily connected to a company’s private data by using a private link to their datacenter cage or through SD-WAN breakout (see figure below), a cloud edge LLM might have much easier access to sensitive data for training purposes than a private or public LLM running in a public cloud.

The above figure shows a Cloud Edge ML implementation with connectivity to the company’s on-prem locations over SD-WAN. The ML workloads could be LLMs like Llama or Dolly or computer vision ones such as NVidia Metropolis.

Aarna Edge Services (AES)

The AES SaaS offering provides machine learning at the cloud edge. It features an easy-to-use GUI that can slash weeks of orchestration work into less than an hour. In case of a failure, AES includes fault isolation and roll-back capabilities. A private beta is coming soon with support for:

‍

• Equinix Metal Servers with GPUs
• Equinix Fabric & Network Edge with Azure Express Route/AWS Direct Connect
• Pure Storage
• ML workloads:

• NVidia Fleet Command + Metropolis, OR
• Open source Llama LLM, OR
• Open source Dolly LLM

Our partner Predera provides support and professional services for the ML workloads. Reserve your spot today to get on the private beta list as we will initially be enabling just a few users!