Generative Artificial Intelligence (generative AI) is already benefiting Edge AI projects in exciting ways today —helping Machine learning (ML) engineers streamline the development process, generate training data for new scenarios, and more. However, we haven’t seen the full potential of this tool yet. This article explores what is possible in the future and what needs to happen first, with insights from Imagimob CTO Alex Samuelsson.
“Looking back 10 or 20 years, deploying deep learning models on edge devices was not feasible, but today it is,” says Alex. “Generative AI on the Edge shares a similar trajectory. Technological advances are making models smaller and more efficient, while edge processing power is increasing due to better neural network accelerators. We're also seeing improved tools for creating and deploying these models on edge devices.”
Operating generative AI on the edge is expected to offer exciting new possibilities and experiences for ML engineers and edge device users, with applications ranging from personal to industrial. Here are some of the highlights.
Dynamic Model Interaction
In contrast to today’s cloud-based generative AI models, generative AI that is embedded in edge devices will allow you to respond much faster to local conditions and make real-time model updates to enhance safety and efficiency.
“One of the interesting aspects of generative AI is its dynamic nature,” says Alex. “In the future, when generative AI models are deployed on embedded devices, we will be able to better adapt to the specific realities of those devices. For instance, consider devices on a factory floor that report any dangerous incidents. If a new hazardous situation arises, you can simply update the guidelines for that model, instructing it to monitor for the new scenario, then implement this update across all devices immediately.”
Responsive User Interactions
In everyday user scenarios, running generative AI on edge devices can create a more personalized experience where the interactive model responds to user requests or data in real time, analyzes their preferences and patterns, and adapts or optimizes accordingly.
“In the future, devices will become more personalized and interactive,” says Alex. “For example, imagine you want to cook dinner at home but aren’t sure what to make. Instead of searching for recipes on your laptop, you can interact with your fridge's generative AI model. The fridge can analyze the available ingredients and offer recipe recommendations while also learning about your user preferences over time.”
Proactive Maintenance
Running generative AI on the edge can make it possible to constantly monitor machinery, analyze error logs, and predict maintenance needs, thereby improving operational efficiency and preventing breakdowns.
“Today, it's really difficult to understand what's happening inside a machine,” says Alex. “It requires experts to sift through error logs, classify error messages, and determine the underlying issues. However, in the future, when we deploy generative AI on edge devices, these models will operate continuously, analyzing the situation and making proactive decisions in real time. This approach allows for local decision-making, eliminating the need to send all data to the cloud for processing. Furthermore, these models can become specialized; they adapt to their specific unit and, based on historical data, can recommend an optimal maintenance routine.”
Hybrid Solutions
Combining Edge and cloud AI can be a cost-effective and beneficial solution, allowing you to benefit from both worlds: you save on expenses while still utilizing powerful generative AI for informed decision-making.
“A hybrid solution would allow you to train a model on the edge to detect specific events and program it to transmit data to the cloud if it identifies anything potentially interesting or dangerous,” explains Alex. “A cloud-based generative AI model could then analyze incoming data more thoroughly and make better-informed decisions. In this way, the Edge AI model acts as a filter, preventing unnecessary data from reaching the more costly cloud-based models.”
How do we close the gap between what is possible today and what might be possible in the future? Running generative AI on the edge hinges on some key technological advancements, including in the areas of memory footprint, processing power, and development tools.
Oversized memory footprints
To run generative AI on edge devices in the future, models need to be smaller and feature more efficient layers and smarter architectures that require less memory.
Inefficient processing power
There is a significant difference between a phone and the type of high-performance, power-efficient embedded processors or systems needed to run generative AI on edge devices. Innovations like the PSOC™ Edge high-performance ML microcontrollers show promise of helping break through this barrier in the future.
Lack of deployment tools
Deploying generative AI models on the edge will require even more effective tools and frameworks in the future, which can optimize layers and guide deployment. Advancements in DEEPCRAFT™ Studio, for example, will continue to pave the way in this area, helping make smaller more complex models more optimized and innovative.
“A combination of these three ongoing advancements should make it possible to deploy generative AI on the edge without worrying about logistics,” says Alex. “However, it will require some time as we evaluate specific user cases and other important factors."
Generative AI has already begun to transform the way machine learning engineers develop and train Edge AI models (as we covered in first blog in this series) —and the future holds even greater possibilities. Although it will take some time for generative AI to truly exist on the edge, the advancements we need to get there are ongoing as models and processing power become increasingly more efficient. We also anticipate that developers will come up with new and creative ways to use generative AI, pushing the boundaries of its applications.
“I envision generative AI on the edge leading to a huge explosion of creativity as ML engineers use it to write code, check for bugs, create unit tests, and more,” says Alex. “At Imagimob, we are carefully watching the developments to see how we can leverage these benefits for our customers.”
Don't miss out on the latest blogs, news and more from Imagimob!
Subscribe to our monthly newsletter to stay up-to-date on all the latest blogs, news, events, webinars and more from Imagimob.