Each day, we are using devices equipped with Edge AI – from the face recognition function on our smart phones, to cars and satellites with on-board cameras, to our TV and home cameras. Artificial Intelligence (AI) is transforming the way we live and work as well as how we do research. AI on the edge allows us to process images and audio locally, close to the user, without the need to transmit data.
AI is becoming increasingly essential for particle physics. It helps researchers at CERN to analyse data in order to understand particle behaviours and their interactions in real time and run their experiments 24/7. Edge AI is a promising technology, which brings computing capabilities close to where data is actually collected. Data can be processed by operating an AI algorithm locally on the device itself without the need of the so-called cloud (a global network of remote servers). Being able to process the data at the source, not only improves performance, but also saves energy.
Particle detectors could be described as the ultimate Edge AI devices. Transferring data to the cloud isn’t feasible as decisions about the significance of a particular particle collision need to be made within microseconds, while there are about 40 million collisions per second occurring in some of the LHC’s detectors. To process this ever-increasing amount of data, CERN embarked on a search for a viable solution that would, at the same time, minimise power consumption. As of 2019, the market offered no such solution.
"Ceva came to CERN, as part of a delegation aiming to connect Israeli companies with CERN, just after Israel became a CERN Member State. The colleagues present at the meeting, from the CERN Knowledge Transfer group, called me and said the people from Ceva were interested in neural networks and I had 15 minutes before they left CERN. I ran there and it was clear very quickly that it was a match!" says Maurizio Pierini, Senior Physicist at CERN.
Ceva‘s expertise on innovative neural networks for data compression fitted perfectly with CERN’s needs and CERN provided the perfect environment for Ceva to implement and improve their knowledge: ideal conditions for a fruitful collaboration.
Despite very different use cases and types of data to process, both parties had common goals. They were interested in simplifying neural networks to reduce the computational cost, while maintaining the accuracy of the output. The collaboration involved CERN experts working on new compression algorithms to reduce the size of the data, which means simplifying the numerical representation by having fewer binary digits or ‘bits’ in the calculation process. The goal is reducing the size of the neural network models while preserving accuracy and performance. Meanwhile the Ceva team concentrated on the development of a Digital Signal Processor (DSP) and hardware accelerator, a specialised microprocessor chip able to work with these compressed models while maintaining accuracy, at the very high speeds needed to process the LHC data.
Adrian Alan Pol, employed to work for this project and placed at CERN, now working as an Applied Machine Learning Specialist at Thomson Reuters, describes the collaboration with Ceva as a thrilling experience.
It exposed me, as a scientist, to the industrial way of thinking on a full-time basis. While the focus remained on experimental physics, we needed to ensure our solutions were transferable to Ceva's needs. It is always good to know how your model design choices will affect performance on different computing architectures - a skill I learned during the collaboration, thanks to feedback from Ceva’s experts.
Through working together, the CERN and Ceva teams were able to create algorithms which can adapt in real-time and be trained to optimise (choose how many bits to use) regionally across a neural network. Thus, developing a high-performing and power-efficient Edge AI hardware solution tailored specifically for CERN’s extreme low latency need (delay to perform the inference and produce an output). A summary of their findings was published on the open arXivLabs platform which shares research to benefit the scientific community.
This is a perfect example of a symbiotic relationship with both partners benefiting from the collaboration. As a result, CERN can add the newfound algorithms to their software library for future experiments while Ceva plans to integrate the new technology into their consumer products later in 2024.
"Thanks to our collaboration with CERN we were able to develop an innovative approach that enables the networks to run up to 15 times faster compared to 16-bit baseline models. Its enhancing network speed and reducing energy consumption by up to 90% while maintaining comparable accuracy." shares Olya Sirkin, Senior Deep Learning Researcher at Ceva.
_______
This collaboration started in 2019, funded by the Israel Innovation Authority (IIA) to explore how cutting-edge Israeli companies and institutes could embrace CERN’s unique technological know-how to fuel their innovation and help to make a positive impact on society.