Embedded Computing and AI at the Edge
Early IOT devices were typically based on quite simple architectures. Typically, they would incorporate a simple microprocessor, a sometimes separate radio chip (often Bluetooth Low Energy), and some sensors; their only function would be to gather some data and transmit it onwards to other systems. Since then, IOT devices have developed enormously in terms of performance, features and sophistication. In effect, they were simple cable replacement applications.
Device Computing Power and Security
First generations with devices with both integrated radios and microprocessors have been with us for quite some time. However, there is a night and day difference between the processors in early devices and those of the latest generation devices. Typically based on ARM cores, early devices would have the most basic M0 core running at perhaps 16MHz. The state of the art today has M33 core devices running at up to 320MHz – a 20x speed increase in little over a decade. This has been achieved without increasing power consumption, and indeed reducing it in some cases.
The most advanced devices today offer multicore processors, typically independent network and application processors, which allows real time response to inputs without interfering with over the air communications. Often this has allowed end device manufacturers to dispense with additional microprocessors even for more sophisticated applications. Clearly such processors can do much more than merely handle data communications.
Security in early IOT devices was typically weak, with encryption of over the air traffic being about the limit of what was offered. This is no longer acceptable, as IOT systems become ever more connected and integrated into mission critical applications.
Secure microprocessors, incorporating cores such as ARM Trustzone, and secure key storage for end-to-end authentication and encryption are becoming standard. The European Cyber Resilience Act is making much of this obligatory, but even in other regions, the reputational risk of insecure systems is driving an increased focus on cyber security.
The majority of IOT systems will incorporate over the air update capabilities to fix security flaws as well as provide feature updates, just as phones, tablets and PCs already do. However, these require strong authentication and encryption processes to be helpful, otherwise they could cause more harm than good. In addition to secure software, physical tamper resistance is also likely to become standard in the future.
Rich Peripheral Sets
A further development has been the range of peripheral connections supported by modern devices. Early devices were largely focused on reading sensors, via fairly simple connections such as SPI and I2C. The next generation offers a much broader range of options, including ethernet drivers, CAN buses, high speed USB, dedicated audio drivers and more. In addition, some devices offer separate programable peripheral cores to allow users to develop their own drivers without impacting the main application processor.
AI at the edge
The concept of AI functionality on a small battery powered device may seem strange, given the typical image of AI as something delivered by vast server farms. However, small AI inference engines are appearing on IOT devices. The idea is not to run a Large Language Model (LLM) general purpose AI, which would be unfeasible. Rather, the concept is to run an inference engine, targeted at a limited problem domain, using an AI model trained on some larger system. Potential applications could be simple speech recognition, face detection, or intelligent presence detection.
The aim of such systems is to achieve more efficient processing than conventional manually created logic, and thus reduce power, enable new applications, or perhaps reduce the complexity of sensors systems. AI enhanced edge computing can also reduce network traffic by locally analysing and summarising data, and provide a low latency response locally.
Such an application of AI may turn out to be more immediately useful and profitable than the grand vision of general intelligence via LLMs, which comes with major issues of cost and resource consumption. The viability of huge LLM based systems is still in the balance today, in terms of the potential revenues justifiing the capital outlay.