Artificial Intelligence has taken the world by storm in recent years. Its applications have touched diverse fields such as, predicting how proteins fold from a chain of amino acids, to beating the World Go champion in a series of games, to the ever helpful LLMs like ChatGPT.

All these complex AI models need the most skilled and talented people to build applications and huge computing resources to train, and to infer.. These applications are used by millions of people daily. One estimate puts ChatGPTs MAU(Monthly Active Users) at about 180 million. And of course, behind the scenes, there are powerful datacenters/cloud providers that keep the infrastructure running. And it’s the power consumption of these datacenters/cloud providers that people have started to take notice of.

Power hungry GPTs

Let’s take ChatGPT for example. It typically requires specialised hardware accelerators such as GPUs or TPUs for efficient computation. These are optimised for parallel processing, which helps reduce power consumption compared to traditional CPUs. However, even with this breakthrough, the sheer complexity of training these models, with billions of parameters, requires substantial energy.

On average, running a deep learning model might consume anywhere from a fraction of a watt to a few watts per query. This estimate can vary significantly based on factors such as model size, complexity, and the efficiency of the hardware infrastructure. But given the scale of usage of AI applications, even a low estimate of a fraction of a watt, can result in significant power consumption.

For example, let us look at the power consumption of ChatGPT. Consider the energy required to charge a smartphone to put this into a more relatable context. An average smartphone charge might take about 5Wh. As per one research, the energy used for a single request to ChatGPT is equivalent to charging a smartphone 60 times!

It’s quite clear that traditional AI learning models often require substantial computational power and energy, making them unsuitable for resource-constrained environments such as IoT devices and wearables. 

But there is a solution, Distributed AI.

What is Distributed AI?

Distributed Artificial Intelligence (DAI) also called Decentralised Artificial Intelligence is a subfield of artificial intelligence research dedicated to the development of distributed solutions for problems. DAI is closely related to and a predecessor of the field of multi-agent systems.

How does distributed AI conserve power?

Instead of running a big fat model that can understand and respond to a multitude of use cases,  we can develop optimised lean models that are use-case specific. These lean/localised models can be smaller in size and needs less computer power and consume less energy. The AI engine can be trained for big fat model in the cloud and derive multiple use-case specific model from them. At the location where we need models for specific use-case we can use localised models. 

The large models require a huge amount of computer power and energy to develop, train and validate. The lean localised models can be run on low computing devices and require very less energy. However, these models can work only for the specific use-case. 

TinyML for distributed AI

Tiny machine learning is broadly defined as a fast-growing field of machine learning technologies and applications including hardware, algorithms, and software capable of performing on-device sensor data analytics at extremely low power, typically in the mW range and below, and hence enabling a variety of always-on use-cases and targeting battery operated devices.

TinyML leverages advanced algorithms and optimisations to run efficiently on SOCs with minimal energy consumption. By utilizing techniques like quantization, model pruning, and hardware acceleration, TinyML significantly reduces the computational and energy requirements of machine learning algorithms. This enables devices to perform inference tasks locally, without relying on cloud servers, thereby enhancing privacy, reducing latency, conserving bandwidth and at the same time reducing the power consumed. This could potentially make our entire transition to AI/GenAI “green”.

Moreover, TinyML opens doors to a myriad of innovative applications across various industries, including healthcare, agriculture, and smart infrastructure. From real-time health monitoring to predictive maintenance in industrial equipment, the possibilities are limitless.

In conclusion, while technologies like TinyML are becoming popular, we will be seeing more such solutions to make future AI deployments sustainable and green.