Embedded AI Demystified: A Deep Dive into Applications and Benefits

In recent years, the field of AI (artificial intelligence) has experienced remarkable advancements and results across various industries by transforming the way we interact with technology. Over the past few months, the world has witnessed the rise of ChatGPT, Photoshop AI (its generative fill function), and other AI tools that push the boundaries of what is achievable through their help each day. While AI programs have the possibility to assist in various fields, they lack the ability to interact with the physical world.

Enter – embedded systems. Embedded systems can serve as a bridge that connects AI to the physical world and enables it to interact with the environment. Unlike traditional AI systems that rely on cloud computing and external servers, embedded AI brings the power of machine learning and decision-making algorithms directly to the edge devices themselves.

By embedding AI into devices such as smartphones, smart home appliances, and industrial machinery, these technologies gain the ability to analyze data in real-time, make intelligent decisions and perform complex tasks without relying on constant connectivity or latency issues associated with cloud-based solutions.

What is AI/ML/DL?

You have probably heard some buzzwords like AI, machine learning and deep learning. But what exactly are they and how are they connected?

In classical computer programming, problems are solved by writing code that contains specific algorithms and instructions on how the code should behave. Those algorithms are encoded into the code, guiding computers on how to solve those problems or produce wanted outputs.

Artificial intelligence, on the other hand, is a technique that enables machines to imitate human intelligence and behavior. It involves developing systems capable of performing tasks that would typically require human intelligence, such as visual perception, speech recognition, problem-solving, and decision-making.

Machine learning (ML), a subset of AI, focuses on algorithms and statistical models that allow machines to learn and improve from data (experience) without being explicitly programmed. A machine learning model is trained on a dataset and based on that training, it can make observations or predictions about new, unseen data. Machine learning is an essential component in developing AI systems.

There are three types of ML algorithms:

• supervised learning,
• unsupervised learning,
• and reinforcement learning.

Supervised learning involves training a model on labeled data, where the correct output is already known. Unsupervised learning involves training a model on unlabeled data, where the correct output is not known. Reinforcement learning involves training a model through trial and error, where the model receives feedback in the form of rewards or punishments.

Deep learning (DL), which is now a subset of machine learning, utilizes artificial neural networks with multiple layers to process and learn from vast amounts of data. It aims to mimic the structure and functionality of the human brain.

What is Embedded AI?

Embedded AI is the application and integration of AI capabilities (by ML and DL) into small-scale devices such as smart appliances, wearables, sensors and smartphones. It enables them to learn from their environment, adapt to user preferences and autonomously respond to changes.

AI models for embedded systems must be relatively small in size considering they are to be run on microcontrollers, DSPs, and other low-power processors. These AI models are typically a few kilobytes in size and perform a particular function like identifying people/objects, recognizing a wake word and interpreting user commands (some while still having a low power consumption). The training of these models is done by capturing the data from sensors and training it over a cloud platform before finally transferring it to the embedded device (often by tools such as TensorFlow Lite and X-CUBE-AI). The device then consumes real-time data from the sensors and applies the model to that data.

Some of the features characterizing embedded AI applications:

• Runs on a microcontroller, DSP, or a low-power microcomputer with limited RAM, flash memory and sometimes even battery power.
• The AI model should be executed at the device level (locally), meaning the application on the edge device can execute without any need for data communication over the network. The network should only be used to transfer results and other data not used by the AI model.
• AI learning model should have a minimal footprint (a few tens of kilobytes), so it can be run on microcontrollers that have limited memory and computing resources.

What are the Embedded AI Benefits?

The growing popularity of embedded AI is driven by multiple factors and benefits. By deploying AI algorithms directly on the edge, the latency (response time) is significantly reduced resulting in improved real-time performance, achieved by removing the delays imposed by transmitting data to and from the cloud. All the sensor data is captured locally, and an already-trained AI model is used to derive inferences. The result may then be transferred to a server or cloud, but the data exchange is not required for the functionality of the device. Instead of sending massive amounts of raw sensor data to the cloud for processing and analysis, only relevant data is sent.

This not only leads to more efficient network utilization, reduced data traffic to the cloud and, with that, reduced data transmission costs. It also minimizes dependencies on the cloud infrastructure, allowing applications to run offline and operate seamlessly during limited or non-existent network connectivity.

Furthermore, privacy, which is a significant issue in IoT (Internet of Things), is increased because everything is done locally without transferring and storing the data on the cloud.

What are the Embedded AI Applications?

Embedded AI is revolutionizing various industries by enabling intelligent decision-making right at the edge.

Some of the applications where embedded AI found its purpose are:

• Real-time predictive maintenance – preemptively makes critical maintenance decisions before a disruptive event can occur, schedule maintenance, repairs, and avoid unexpected failures. It contributes to lower maintenance costs, downtime reduction, increased lifespan and maximized ROI. Some applications of predictive maintenance include water utility leak detection, appliance failure detection and equipment predictive maintenance.

• Smart homes – by incorporating AI-powered sensors and devices, homes can intelligently adjust the temperature settings, lighting conditions and even anticipate occupants’ needs. This leads to improved energy efficiency and a more comfortable environment. It also found its use in voice assistants, understanding user commands and detecting “wake” words such as – “OK, Google” or “Alexa”.
• Camera-related applications – embedded AI systems can be used to perform object detection, facial recognition and image classification in real-time, for example, granting people/objects access or calling security for blacklisted people/objects. A major concern for camera usage in such applications, where personal data is collected through video recordings and then processed, is GDPR which aims to ensure that individuals have control over their personal information. That means companies should be careful and ensure the responsible and ethical use of the personal data captured.
• Autonomous vehicles – assists in real-time object detection, lane assistance and pre-sense collision avoidance
• Smartphones – maybe you heard something about the Samsung moon controversy where it was suggested that Samsung fakes moon photos when using 100x zoom. Spoiler ahead – it turned out to be the “Scene optimizer”, an image enhancement software powered by AI. When images are captured in 100x zoom, the AI uses all its wits to enhance the image and improve the quality drastically.

Source: https://www.gsmarena.com/samsungs_moon_shot_is_scene_optimizer_super_resolution_and_ai_magic-news-57932.php

• Healthcare – early disease detection (mosquito-borne diseases – Zika virus, West Nile virus, malaria, etc.), personalized treatment plans and fitness equipment.
• Agriculture – detect potential natural disasters, detect diseases and pests in plants
• Retail – automation of inventory management. Used for tracking items on store shelves and sending an alert before they get out of stock.
• Transportation – embedded AI can be used to monitor traffic and detect traffic jams. One of the possibilities is also interconnecting the system with traffic light management to optimize traffic in real-time or even accident detection so it can send alerts to the nearest emergency ambulance service, etc.

You get it, the possibilities are endless.

As technology continues to advance, we can expect to see even more innovative applications of embedded AI technology in the future.

What are the Embedded AI Challenges?

As this technology continues to evolve and become more popular in various industries, several key considerations, challenges and limitations have to be taken into account.

One of the major challenges is the limited computational power, energy resources and memory in those resource-constrained devices, making it difficult to implement complex models on those devices, often leading to a trade-off between accuracy and resource usage. This trade-off can impact their performance in complex tasks, especially compared to cloud-based AI applications. The hardware for embedded AI should be carefully selected and designed to handle its tasks successfully. Balancing power consumption, memory usage and processing speed is essential for ensuring optimal performance without compromising other functionalities.

Another challenge is the need for extensive data collection and preprocessing to train AI models effectively, considering that once ported to an embedded system, the model has no more training (or retrained after years). It only processes the real-time data from the sensors or input devices and applies the model to them. Therefore, an AI model needs to be highly robust and pass through a rigorous testing procedure before conversion so that the safety and reliability of embedded AI systems are ensured, and errors or biases are eliminated, especially in critical applications such as healthcare and autonomous driving. These systems should be capable of handling unexpected scenarios or variations in input data without compromising accuracy or safety. Ethical considerations also play a fundamental role in shaping the development of such systems. Algorithms designed must not exhibit biases or discrimination based on race, gender and similar characteristics so that transparency and fairness are ensured.

Lastly, the lack of standardized frameworks and tools for developing and deploying embedded AI systems poses a challenge to developers, whether those developers would be embedded engineers or exclusively ML engineers who deliver developed models/systems.

What holds the future of the Embedded AI?

As technology continues to advance at an exponential rate, the future of embedded artificial intelligence holds immense potential and will shape the landscape of technology over the next decade. The demand for embedded AI technologies has been driven by hardware advancements and the increasing availability of AI algorithms, resulting in significant market growth and a bright future. Advancements in hardware design have been made, such as Google’s TPU (Tensor Processing Unit) ASIC and neuromorphic chips. The most power-hungry mammalian brain, the human brain, only takes 25 watts to drive 86 billion processing units within it. Comparing it to Google’s TPU which needs 250 watts to drive the 65 thousand processing units, it becomes obvious that we are still orders of magnitude away from what is achievable.

Embedded systems and artificial intelligence complement each other well and can come together to create interconnecting solutions on edge devices or standalone embedded systems. They will make IoT applications more private and secure.

Although there are numerous and huge advancements to be made in improving hardware and AI, embedded AI devices can already be deployed with good performances and lightweight AI models, as we can see in a great number of its applications in the real-world. An edge AI platform, “Edge Impulse”, already enables the deployment of highly optimized ML on embedded devices for sensors, audio and computer vision by handling data collection from sensors, labeling the data, performing calculations, and training the models. Developers can then test it on live data and generate firmware that can run on any number of platforms.

To conclude, businesses that understand how to work with AI and develop intelligent embedded systems will be well positioned to take advantage of many opportunities these technologies will create.