REBECCA Newsletter No1

Reconfigurable Heterogeneous Highly Parallel Processing Platform for safe and secure AI 01 Newsletter REBECCA receives funding from KDT JU and national agencies under Grant Agreement No.101097224. Funded by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the granting authority. Neither the European Union nor the granting authority can be held responsible for them. Website www.rebecca-chip.eu LinkedIn rebecca-kdt-ju July 2024 Short News 01 2 EDDL integration into Bonseyes platform. Solver Intelligent Analytics EXAPSYS A functional quad-core RISC-V processor capable of booting Linux is available on the emulation platform. A custom AXI4 Network on Chip (NOC) has been developed to integrate AI and security accelerators. The system supports a shared global address space. 16 Deliverables have already been prepared and submitted. Within the REBECCA ad-hoc Safety and Security working group, SYSGO has contributed to safety and security need analysis. As WP3 lead, it was a pleasure to work with FORTH (editor) and other WP3 partners to prepare D3.1 on the alpha version of the software stack. SYSGO Deliverables Technical University of Crete TUC has developed a pattern-matching algorithm for IDS (Intrusion Detection System) and introduced it in the emulation platform. The IMEC neuromorphic AI accelerator has been developed, integrating several IBEX RISC-V cores. Imec Lund University The on-chip memory AI processing module has been developed, and back-end implementation has started. FORTH Kubernetes/K3s has been successfully ported to RISC-V. University of Athens The University of Athens team has developed a novel safety/reliability assessment simulation-based framework for heterogeneous SoCs which employs fault-injection to quantify the vulnerability of the RISC-V CPU and the REBECCA accelerators to transient and permanent hardware faults when the platform is deployed at the edge. July 2024 Other News 3 Excellent performance results with a reduction in latency of 80% RISC-V, the open standard instruction set architecture (ISA) based on RISC principles, has revolutionised the tech landscape since its inception at the University of California, Berkeley in 2010. Free and open source, RISC-V's modular and simple design supports a wide range of applications, from embedded systems to supercomputers, making it a highly scalable and efficient solution. Managed by the RISC-V Foundation, its growing community and ecosystem provide a flexible and cost-effective alternative to proprietary ISAs like ARM and x86. RISC-V is increasingly adopted for AI on the edge, thanks to its open and customisable nature. It is well-suited for edge AI applications where efficiency, low power consumption, and tailored hardware are crucial. Its modular architecture allows for custom extensions optimised for AI workloads, such as vector processing and machine learning accelerators. This flexibility enables the design of specialised processors that handle AI inference and processing directly on edge devices, reducing the need for constant cloud connectivity and enabling faster, more efficient AI operations in applications like IoT devices, smart cameras, and autonomous vehicles. Klepsydra has benchmarked the execution of AI algorithms on the Microchip PolarFire ICICLE using ESA’s OBPMark-ML models and TensorFlowLite as a baseline. The results were impressive, with Klepsydra AI running up to 5x faster than the baseline! Press Release Was successfully shared by these media: DATA CENTER MARKET TECNOBITT DIRIGENTES DIGITAL LA VOZ DE TOMELLOSO OBJETIVO CASTILLA LA MANCHA LA COMARCA DE PUERTO LLANO LANZA DIGITAL NATION WORLD NEWS Performance comparison Klepsydra AI / TensorFlowLite (TFL) Key Highlights: - “PolarFire Icicle Kit”:Utilises RISC-V architecture with SiFive's U54-MC cores and a PolarFire FPGA. - “OBPMark-ML”: An ESA initiative benchmarking frame work for validating space-qualified onboard processors. Includes AI models for cloud detection, ship detection, CME detection, etc. EL NEGOCIO July 2024 Past Events TheEdgeAIAI Tech Conference 17-19 October, 2023 We are proud to announce thatREBECCA KDT JUpartners, includingEXAPSYS Exascale Performance Systems,Foundation for Research and Technology - Hellas (FORTH), Bonseyes Community Association (Non-Profit), and Lund University, recently showcased their groundbreaking innovations at theEdgeAIAI Tech Conference. REBECCA KDT JUwas a vital part of this remarkable event, driving innovation and progress in the field ofedgeAI. REBECCA KDT JU(Reconfigurable Heterogeneous Highly Parallel Processing Platform for safe and secureAI) is on amissionto democratize the development of edge AIsystems. This project is creating a comprehensive hardwareandsoftwarestack built around a RISC-V CPU. The result? Unparalleled performance, energy efficiency, safety, and security in comparison to existing systems. https://edge-ai-tech.eu/conference-homepage/ 4 July 2024 30 November - 1 December, 2023 TheChips Joint UndertakingLaunch event Our team members are thrilled to be part of theChips Joint UndertakingLaunch event! We are grateful for the opportunity to contribute to success and be part of this milestone. The spirit ofcollaborationand shared enthusiasm has fueled our commitment toexcellence, and we look forward to continuing this journey hand in hand withChips Joint Undertaking. www.smart-systems-integration.org/ eventchips-ju-launch-event 5 July 2024 14 December, 2023 The 1st Review meeting 17-19 January ,2024 HiPEAC 2024 We are thrilled to announce our participation as co-organizers of the “Driving Next-Gen Edge AI Technologies” workshop atHiPEAC2024, alongside industry leaders likeChips Joint Undertaking, EdgeAI, ANDANTE-AI,CLEVER-project, and theNeuroKit2E! Representatives fromSINTEF, our esteemed partner, played a pivotal role in organizing the workshop, which featured insightful presentations and engaging panel discussions. It was truly an exhilarating experience diving intoAIinnovation at theedges, uncovering the synergies betweenartificial intelligence,edge computing, and the Internet of Things (IoT). Stay tuned for more updates on our journey towards driving the future of edge AI! Many thanks to the project team for the constructive work during the first reporting period and for thecollective efforts to demonstrate our achievements during the meeting. Thanks a lot to the Project Officer and the project Reviewers for every comment and recommendation. This will help us continue the project in a way that we can all be proud to have been a part of! www.hipeac.net/2024/munich/#/ 6 July 2024 Underwater Robot for infrastructure inspection with edge AI REBECCA Chip One of the use cases of the REBECCA chip is an underwater robot, developed by AquaSmart Engineering and Almende. The goal of this underwater robot is to inspect underwater infrastructures, like quay walls, sheet pile walls and pillars. There can be damage present on those objects, and to aid in the inspection an AI recognition algorithm is under development. This will run ‘at the edge’ on the REBECCA chip, and will help detect damages, like cracks in concrete walls. The first iteration of this robot has been built in the previous months, and right now the second iteration of the robot is in the concept phase. The first version has been tested in simulation, on a workbench, in a controlled test environment (a test container), and in the river Schie in Rotterdam. Testing the robot was a lot of fun, and the results were very promising! Almende and AquaSmart Engineering are looking forward to developing and testing the second iteration, and testing the edge AI REBECCA Chip! www.almende.nl www.aquasmartxl.com Use case 7 Underwater Robot being tested in the controlled test environment Underwater robot being tested in the river Schie Example of inspection data captured in the controlled test environment July 2024 Real-time defect detection in PV panels on unmanned aerial vehicles (UAV) devices Use case The “Real-time Fault Detection in PV Panels on Unmanned Aerial Vehicles (UAVs)” is one of the REBECCA project’s Use Case that is being developed by Intecs Solutions. The goal of this Use Case is to develop an automated Deep Learning-based model for defect detection in PV panels based on aerial images captured by IR camera mounted on board of an Unmanned Aerial Vehicle (UAV) and with the processing executed directly on-board. The results of this processing will be displayed as a predicted on-ground defect positions in a 2D map of a PV plants directly visible on the remote controller of the UAV pilot. The current state of the art for AI-based solutions in this area typically involves initial data collection by O&M operators in the field, followed by offline processing in the O&M control room via a workstation. This results in a delayed intervention by the O&M operators, which leads to higher maintenance costs. Example of O&M OperationsIntroducing on-board processing bypasses the need for data transfer and enables instant analysis, reducing O&M costs while increasing maintenance effectiveness. Our goal in this case study is to streamline the model pipeline to reduce inference time, develop leaner CNNs, and employ structured/unstructured pruning techniques to reduce the required volume and floating-point operations (FLOPs) while ensuring a good trade-off with accuracy metrics. We're also exploring the integration of GNSS receivers to increase the resolution of defect localization to single cell granularity. In this phase we are currently working on our algorithm and in defining collaboration with the REBECCA’s subcomponents providers. Example of O&M Operations UC’s Work Pipeline www.intecs.it 8 July 2024 AI-powered fridges that recognize food with image recognition Use case Another use case of REBECCA involves an AI-powered fridge developed by Arcelik. In this scenario, the fridge is equipped with two cameras: one mounted on the door, capturing the interior, and the other inside the body, capturing the door. When the door is opened, these cameras are triggered to capture multiple image frames until the door is closed again. Among these frames, the best ones, which provide optimal visibility of the fridge interior, are selected using a TOF sensor that measures the distance between the door and the body. The selected pairs of images are then transferred to the cloud for image processing and analysis, after which they are sent to the related users' Arcelik-specific mobile application (HomeWhiz™) for remote monitoring of the fridge interior. In the current cloud-based architecture of this use case, all image processing algorithms and AI-based computer vision models, including object recognition, human detection, distortion removal, and cropping algorithms, run on the cloud platform. The object recognition model identifies and localizes 33 different objects, such as eggs, milk, water, apples, and more, creating an inventory list. The human detection model checks if there is a human in an image, which may occur unintentionally, and deletes such images if any are found due to privacy reasons. As part of the REBECCA project, edge versions of these AI models are being developed. These edge versions will initially be deployed, executed and evaluated on REBECCA emulators and eventually integrated into REBECCA chips. User interface of the HomeWhiz™ mobile application www.arcelik.com.tr 9 July 2024