Comparative Evaluation between Accelerated RISC- V and ARM AI Inference Machines

Comparative Evaluation between Accelerated RISC-V and ARM AI Inference Machines 1st Vasileios Christofas Department of Computer Science, International Hellenic University Kavala, Greece v[email protected] 2nd Petros Amanatidis Department of Computer Science, International Hellenic University Kavala, Greece [email protected] 3rd Dimitris Karampatzakis Department of Computer Science, International Hellenic University Kavala, Greece [email protected] 4th Thomas Lagkas Department of Computer Science, International Hellenic University Kavala, Greece [email protected] 5th Sotirios K. Goudos Department of Physics, Aristotle University of Thessaloniki Thessaloniki, Greece [email protected] 6th Konstantinos E. Psannis Department of Applied Informatics, University of Macedonia Thessaloniki, Greece [email protected] 7th Panagiotis Sarigiannidis Department of Electrical and Computer Engineering, University of Western Macedonia Kozani, Greece [email protected] Abstract—Embedded AI development has been rapidly improving for the past few years and has had a great impact on edge AI networks. However, as neural networks become deeper and deeper it becomes more difficult to execute complicated tasks without sacrificing a good amount of power and performance. In this paper, we make a comparative evaluation between two AI acceleration devices. The first one features a RISC-V 64-bit processor while the other one is ARM powered. These devices are combined with AI co-processors, or ASICs, with computer vision capabilities. Our benchmark consists of a simple classification task split into multiple versions. The results showed that the RISC-V inference machine had 4 times lower consumption while the ARM machine was up to 15 times faster in our largest network. We discuss the results in great detail while keeping our focus on all aspects equally. Finally, we make recommendations based on their usage and application. Index Terms—machine vision, edge ai, comparison, benchmark I. INTRODUCTION Deep Learning (DL) is a subcategory of Machine Learning (ML) that specializes in Artificial Neural Networks (ANNs). Deep Neural Networks (DNNs) are sub-type of ANNs but with multiple layers stacked on top of each other. Many applications, including but not limited to computer vision and speech recognition take advantage of them to achieve low latency and high accuracy. Examples of such networks are Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs). Their main purpose is to extract useful features from an input layer (e.g. an image). However, extracting more features comes with a great cost of higher power consumption and inference times. Many companies began releasing their own devices capable of computing DNNs efficiently and fast. These ASIC devices, also known as AI accelerators, usually come in the form of processing units. Compared to a Graphics Processing Unit (GPU), the performance per watt of an AI accelerator proves that it’s more efficient while a GPU is usually faster but more power-hungry. When combined with Internet of Things (IoT), these AI chips result in powerful AIoT devices able to satisfy the requirements of a variety of networks commonly used in the industry. As of late, hardware designers have also taken an interest in a relatively new processor architecture. Introduced in 2015, RISC-V [1] is an open standard Instruction Set Architecture (ISA) that’s based on Reduced Instruction Set Computer (RISC) architecture. It features a base instruction set along with many extensions. This modular approach allows companies to reduce manufacturing costs while improving the overall cost-effectiveness of each hardware package. Compared to ARM processors which follow RISC principles as well, it is an open ecosystem meaning that there are no license fees at all. Many research studies have worked on efficient ways to utilize AIoT systems, with the biggest one being deploying them on edge [2]. The edge is located between a server and the end user. The end-user sends his data to the edge, the AIoT devices do the computation, and then send the results to the central server. The prime advantage of this approach is that the servers are no longer required to handle AI requests since the computation is done elsewhere. The disadvantage of this approach is that we now have a new limiting parameter called network latency. Thankfully this issue could be overcome with compression algorithms and more sophisticated network designs like the one shown in [3]. Improving the architecture of each chip can also help in latency reduction as shown in [4] where researchers applied their edge routing methods in Network-on-Chip (NOC) designs. When handling demanding networks, edge devices can sometimes be too strict. Researchers have come up with ways to go around these restrictions with the most common one being quantization. Quantization, as stated in [5], is an algorithm that attempts to transform floating-point arithmetic into integer-only arithmetic while trying not to affect the accuracy too much. In most cases using Post-Training Quantization (PTQ) is more than enough to get great results. Another method is Weight Pruning [6] which is essentially another compression technique that ”kills” neurons if their existence doesn’t have a huge impact on the results. Other modern methods include Knowledge Distillation [7] in which a teacher model transfers its knowledge to a student model and Lowrank Factorization [8]. Our work does a comparative evaluation between two different architecture inference machines and gives recommendations based on their usage. These systems are the Sipeed Maixduino with its CNN accelerator, and the Raspberry Pi 4B combined with Google’s Coral USB Accelerator. We additionally provide results from the Raspberry Pi alone as a reference point for our comparison. II. RELATED WORK IoT devices like the Raspberry Pi are an industry standard for many applications thanks to their low power consumption and cost. Regarding data sharing, they cooperate surprisingly well which is why they are chosen for communication with AI co-processors as well. In general, there are many different types of configurations that can handle AI inference: •Single Board Computer (SBC) •System on Chip (SoC) devices •SBC + Application Specific Integrated Circuit (ASICs) •SBC + GPU •Field Programmable Gate Array (FPGA) boards A. SBC-Based Inference Recent studies have shown that standalone SBCs like the Raspberry Pi 4 (RPi4) can run AI without the aid of AI co-processors but are not as energy-efficient when compared to combinations with other ASICs. Amanatidis et al. in [9] used a standalone RPi4 for face-mask detection in a video dataset and received an energy-efficiency factor of about 2.5 FPS/Watt. When the RPi4 was combined with a Google Coral USB Accelerator the energy efficiency rose up to 10 FPS/Watt. A CPU-based inference was also attempted but managed to get no more than the other two. There is also a study that combined the RPi4 with Intel’s Neural Compute Stick 2 (NCS2) [10]. NCS2 is an AI accelerator that utilizes Intel’s Movidius Myriad X VPU and has an AI performance of around 1 TFLOP. Nvidia’s GPU-based AI accelerators were used in the benchmarks as well including the Jetson Nano and Jetson Xavier with an AI performance of 0.5 TFLOPs and 1.3 TFLOPs. The energy consumption of the RPi was higher than Jetson Xavier’s but less than Jetson Nano’s. The performance graphs on the other hand showed that Jetson Xavier is two times faster than the other two competitors, meaning that it’s way more efficient in terms of latency and power usage. Nair et al. in [11] did a practical comparison between three camera-equipped AI boards; JeVois A33, Sipeed Maix Bit, and OpenMV H7. Their task was grasp verification for robots, and based on the performance analysis of the object detection models, Sipeed had better throughput and latency than the Jevois thanks to its dedicated CNN accelerator. B. SoC-Based Inference Cantero et al. in [12] did a comparative evaluation between Google’s Coral Dev Board and iMX 8M Plus SoC. The benchmark consisted of multiple object detection models that were trained using the COCO [13] dataset. The results showed that Google’s Coral Edge-TPU was the quickest in most cases, but proved to be slightly less efficient than the iMX. C. FPGA-Based Inference When it comes to FPGAs, it is possible to design AI accelerators efficiently [14]. FPGAs mainly consist of programmable logic blocks that can be used to implement different logic functions. Since they are reprogrammable they can be configured to implement DNNs such as CNNs. Many reviews analyze and compare the potential of each FPGA board by doing AI inferences on the vast majority of CNN networks [15], [16]. According to the aforementioned research papers, the average power consumption lies between 15W-20W in some cases while the latency is so low that it is usually measured in microseconds instead of milliseconds. D. Inference at the Edge Machine learning at the edge covers a wide variety of problems. For instance, task offloading has already been covered in [17], where researchers considered both energy consumption and execution latency of the edge devices and trained the model to allocate mobile device tasks to multiple cell access points whenever possible. Another example is object detection. In [18], Nikouei et al. designed a real-time human detection system that was tested using multiple lightweight networks. Methods that incorporate efficient edge training techniques have been introduced as well. Federated learning [19] is a process in which a server coordinates the training of a large AI model by multiple user devices. Split learning [20] is yet another way that works by splitting the neural network into parts and applying partial training. Unlike federated learning, the training part here is done both by the server and the user’s device. III. METHODOLOGY Our test bench consists of three inference machines that belong to the edge category but one is used only as a reference. We had to convert our DNN model to a supported model type, execute it, and measure the results. Meeting the limitations of each edge-class device required proper handling as we didn’t want to affect our metrics disadvantageously. A. Edge-class Hardware In 2018, Canaan released a chip under the name of Kendryte K210. The K210 is a dual-core RISC-V 64-bit processor that comes with a KPU neural network accelerator and 8MB of SRAM. The KPU, which stands for Kendryte Processing Unit is a CNN accelerator able to run models with an inference speed of up to 1 TOPS and a power consumption of less than a watt. The development board we used is Sipeed’s Maixduino. It features the K210 chip, an ESP32 WiFi module, and 16MB of flash storage. The board costs 35$from Sipeed’s official website. In 2019, Google released the Coral USB Accelerator. As its name suggests, it’s a USB stick that provides ML acceleration by utilizing its Tensor Processing Unit (TPU) chip. This EdgeTPU could reach 4 TOPS while consuming 1 Watt per 2 TOPS. Works only with TensorFlow tensors and accelerates many DNNs such as CNNs. The USB costs around 75$MSRP from Coral’s official website. B. SBC Hardware The Raspberry Pi series is essentially a compact and costeffective SBC solution used in a wide range of IoT applications. Their latest models include the Raspberry Pi Pico and the Raspberry Pi 4B. For the inference, we preferred the 4B. The Raspberry Pi 4B takes advantage of Broadcom’s BCM2711, a quad-core ARMv8 64-bit processor. The total cost depends on the desired amount of RAM with our 4GB model being around 60$. Its performance combined with its low energy consumption makes it a really affordable choice for hobbyists, teachers, and industries. C. Model Conversion In most cases, AI accelerators require a compiled version of the neural network in order to work properly. The compilation process is done using chip-specific frameworks. The Kendryte processor supports only KModel model types. The tool that we used to convert from the original network is called ”NNCase”. It supports ONNX, TFLite, and Caffe models. NNCase v0.1 converts models to KModel v3 but it doesn’t accept quantized models as input so we had to provide samples of the dataset and let it perform its own unsigned 8-bit PTQ. After the compilation, we flash our model in the chip’s memory using another tool called ”KFlash”. Coral, on the other hand, has a less complicated conversion process. After acquiring the TFLite model we perform unsigned 8-bit PTQ via Tensorflow so we can use the ”Edge TPU compiler” tool to add Coral OP support. D. Model Inference Running the model on both platforms required different methods. The Maixduino board can work with and without firmware, but we preferred using MaixPy’s micropython firmware to simplify the inference with Python scripts. After flashing the minimal variant of the firmware, we connect our Maixduino to the MaixPy IDE for real-time graphs and quick code deployment. The script loads images from an SD card and passes them to the KPU to do the inference with the alreadyflashed model. Then it saves the results in a CSV file for later examination and analysis. The Coral is a USB device that can’t do anything on its own so we plugged it into our Raspberry Pi. After installing the appropriate drivers we had to make the TFLite Interpreter point to the accelerator by including it in the experimental delegates. The script used for the TensorFlow Lite inference is similar to the Maixduino; We pass images from the validation dataset to the Edge-TPU and save the results to CSV files. The inference here is done twice, with and without the Coral, so that our results include Raspberry Pi’s performance alone as well. E. Network Architecture The inference model copies MobileNet’s architecture [21] to take advantage of its depth-wise separable convolution network. Specifically, we chose MobileNet v1 with a depthwise width factor of 1.0, 0.75, 0.50, and 0.25. Because of Kendryte’s acceleration padding requirements, we had to train and redesign the entire model from scratch. It consists of multiple convolution layers; the standard convolutions, the depth-wise convolutions, and the point-wise convolutions. Each layer has a kernel size of 1x1 or 3x3, depending on its type, with a stride of 1 or 2 and ”same” padding. The expected image input size is 224x224 with 3 channels (RGB) and the output consists of all the classes alongside their confidence values. In Table I, we list the file sizes of our network models alongside their parameter count. TABLE I NETWORK SPECIFICATIONS OF ORIGINAL AND COMPILED MODELS MobileNet v1 Specification Depth Parameters Size TFLite Maixduino Coral 1.0 3.3M 3.5MB 3.3MB 3.6MB 0.75 1.9M 2.0MB 1.9MB 2.1MB 0.50 0.8M 1.0MB 0.9MB 1.1MB 0.25 0.2M 0.3MB 0.2MB 0.4MB F. Dataset The training and validation images belong to the CIFAR100 [22] visual database. It consists of 100 classes of common objects such as vehicles and other daily items. Since our model’s architecture requires an input of size 224x224 we had to upscale the whole dataset using the nearest-neighbor rescale method. CIFAR provides us with two types of labels: ”fine” and ”coarse”. Fine labels are the class labels while the coarse are the super-classes. We used fine labels for a more realistic evaluation. For the inference we randomly chose 500 samples for 2 different sets from the test dataset meaning that we passed 1000 images in total. G. Training We trained each model for 40 epochs with a learning rate of 0.001 and a batch size of 32. The training dataset was randomly augmented in each iteration to avoid overfitting early. The data augmentation included rotating and flipping the images. Additionally, the data values were normalized from unsigned 8-bit [0, 255] to float 32-bit [-1.0, 1.0]. However, because of the low epoch count, the network’s performance may not be optimal. IV. EVALUATION RESULTS To examine and compare the results we divided the evaluation into three parts; the latency comparison, the power draw comparison, and the network performance comparison. A. Metrics The evaluation metrics are as follows: •Inference Time: The average time it takes for each sample to be evaluated on the network. It’s measured in milliseconds (ms). •Images per Second: As its name suggests, it’s the number of images that get processed every second. This metric is calculated using Inference Time. •Energy Consumption: The average power draw of the inference device. It’s measured in Watts (W). •Recall: Measures the number of correct positive predictions made out of all positive predictions that could have been made. Recall =True Positive True Positive + False Negative •Precision: Measures the number of correct positive predictions made out of all instances classified as positive. P recision =True Positive True Positive + False Positive •F1-score: The harmony between Recall and Precision. It’s basically a combination of both metrics to make up one balanced score. F1 = 2∗P recision ∗Recall P recision +Recall B. Latency Latency-wise, it is obvious from Table II that as we reduce the depth of the network our inference machines perform tasks a lot faster. Starting from the full version of Mobilenet v1 with a depth value of 1.0 it’s clear that the Coral is the best performer here, being 15 times quicker than the Maixduino and 21 times more than the Raspberry Pi alone. This huge performance gap TABLE II INFERENCE LATENCY PER SAMPLE Inference latency Depth Set Maixduino Pi 4B Pi 4B w/Coral 1.0 1 60ms 84ms 4ms 2 59ms 83ms 4ms 0.75 1 37ms 53ms 3ms 2 36ms 54ms 3ms 0.50 1 19ms 29ms 2ms 2 19ms 28ms 2ms 0.25 1 7.7ms 12ms 1.2ms 2 8ms 11ms 1.2ms becomes smaller as we drop the number of parameters, for example in the 0.25 variant of our model the Coral is faster than the Maixduino by just 7ms. Both the Maixduino and the Coral execute the classification tasks fast and Table III reflects that. From 0.75 and below the Maixduino is already reaching real-time speeds while the stand-alone Pi reaches them from 0.50 and below. Even though the Coral is above 250 images per second, which is quite astonishing, this performance gap is not required in most classification applications. TABLE III INFERENCE IMAGES PER SECOND Average images per second Depth Maixduino Pi 4B Pi 4B w/Coral 1.0 16.8 11.9 250 0.75 27.3 18.6 333.3 0.50 52.6 35 500 0.25 127.3 86.9 833.3 Considering the architecture of both development boards it’s quite impressive how the Maixduino passed the Raspberry Pi in latency tests since RISC-V is a relatively new technology. We also provided a performance per price ratio plot, as shown in Figure 1, where we compare the value of each device. The reference costs are 35$for the Maixduino, 60$for the Raspberry Pi 4B (4GB), and 60+75$for the whole Pi w/Coral setup. The x-axis consists of the network models while the y-axis is the ratio. As expected, the stand-alone Raspberry Pi is the least cost-effective option since it doesn’t have any acceleration capabilities while the Maixduino and the Coral setup are a much better option. What’s more interesting is that the Maixduino keeps its cost-effectiveness close to the Coral (˜50%less) even when the performance gap is near 700 images per second. C. Power Consumption The results we got in Table IV indicate that the Coral and Pi setups have at least 4.5 times higher power usage than the Maixduino. Keep in mind that Coral’s energy usage is affected by the Raspberry Pi since it needs a host to work. If it was combined e.g. with a desktop computer, the difference would be marginally bigger but unfair at the same time. Fig. 1. Performance / Price plot TABLE IV BOARD AVERAGE POWER CONSUMPTION Power consumption Depth Set Maixduino Pi 4B Pi 4B w/Coral 1.0 1 1W 4.8W 4.7W 2 1.1W 4.9W 4.7W 0.75 1 1W 4.8W 4.6W 2 1W 4.8W 4.7W 0.50 1 1W 4.8W 4.6W 2 0.9W 4.8W 4.6W 0.25 1 0.9W 4.8W 4.6W 2 0.8W 4.8W 4.6W Edge-class devices are usually powered by batteries, so keeping the wattage at a minimum is important. Maixduino maintained ˜0.8 - 1 Watts at all times meaning that it could last a lot more when compared to the other two. However, after calculating the energy efficiency factor of each inference machine we noticed that the Maixduino had a hard time keeping up with the Coral as the network model grew larger. This difference is directly linked to the inference speeds of both devices as the Coral handled the model inference better under load. The energy efficiency plot is illustrated in Figure 2 with the x-axis being the network model and the y-axis being the ratio. D. Post-compilation Network Performance Since the evaluation sets were generated using randomly selected images, the class balance is not guaranteed. Therefore, we can only rely on recall and precision metrics that were made for this exact purpose. In Table V, we list the results of our evaluation sets from our testbench. The Maixduino performed worse compared to the other two, but this could be caused by KModel v3’s limited support for TFLite OPs. The Raspberry Pi on the other hand, had very similar results with and without the Coral accelerator. This is because the edge TPU compiler doesn’t add modifications to the network itself; it solely adds support for the TPU. Fig. 2. Performance / Watt plot TABLE V MODEL RECALL AND PRECISION PERFORMANCE Inference recall and precision Depth Set Maixduino Pi 4B Pi 4B w/Coral Recall Prec Recall Prec Recall Prec 1.0 1 0.5 0.55 0.55 0.59 0.56 0.6 2 0.48 0.54 0.54 0.59 0.54 0.59 0.75 1 0.45 0.48 0.53 0.56 0.53 0.55 2 0.44 0.5 0.51 0.58 0.5 0.57 0.50 1 0.35 0.42 0.4 0.55 0.4 0.57 2 0.35 0.49 0.37 0.56 0.36 0.57 0.25 1 0.32 0.45 0.34 0.49 0.34 0.49 2 0.29 0.37 0.37 0.47 0.35 0.46 In Table VI, we calculated the F1-score metric as well to combine both recall and precision. This helps us simplify the results but also allows us a better idea of how well the model performs. TABLE VI THE AVERAGE BETWEEN RECALL AND PRECISION Inference F1-Score Depth Set Maixduino Pi 4B Pi 4B w/Coral 1.0 1 0.52 0.57 0.58 2 0.51 0.56 0.56 0.75 1 0.46 0.54 0.54 2 0.47 0.54 0.53 0.50 1 0.38 0.46 0.47 2 0.41 0.45 0.44 0.25 1 0.37 0.4 0.4 2 0.33 0.41 0.4 E. Discussion The Pi 4B + Coral setup has a slight advantage over the other inference machines in all aspects of efficiency, but the actual difference depends on real-life scenarios. For example, if we want to design a classifier that can categorize flowers, the Maixduino is more than enough. Even in object detection cases such as security cameras a Maixduino would be an ideal choice as it can do real-time processing. The Coral is designed for deeper and more hardware-demanding networks that can achieve higher recall and precision. It’s also an ideal choice if the application has tight timing requirements. When the power consumption is more important, the Maixduino is a clear winner with 4 times less wattage compared to the other devices thanks to its RISC-V nature. The Raspberry Pi alone consumes as much power as the combination with the Coral (˜4.7W) so it makes sense to use the Coral whenever possible. When the inference speed is more important, the Coral is a lot faster as the network becomes larger. In our case, the overall latency of the Coral setup was less than 4ms while for the Maixduino and Raspberry Pi it was up to 60ms and 84ms respectively. As the model parameters decreased we could see that the Maixduino started performing better to the point where the difference with the Coral was less than 7ms, so the Maixduino is a good choice for simple real-time tasks. Regarding recall and accuracy, all devices had equal results but for the Maixduino they were slightly worse, possibly because of compiler limitations. V. CONCLUSION The demand for Edge AI applications is experiencing an exponential growth along with the need for efficient accelerators. In our research paper, we conducted an evaluative comparison between such AI accelerators. Our aim was to analyze and provide data that covers all aspects of efficiency in order to help industries enhance their services in a cost-effective manner. In future work, we could go a step further and make comparisons based on real-life scenarios. By integrating AIoT with current AI solutions we could save a significant amount of time and money while improving their daily usage, to say the least. ACKNOWLEDGMENT This work received funding from the European Union’s Horizon Europe Research and Innovation Programme under Grant Agreement No. 101070181. This paper only reflects the authors’ views, and the Commission is not responsible for any use that may be made of the information it contains. REFERENCES [1] A. Waterman, “Design of the risc-v instruction set architecture,” Ph.D. dissertation, EECS Department, University of California, Berkeley, Jan 2016. [Online]. Available: http://www2.eecs.berkeley.edu/Pubs/TechRpts/2016/EECS-2016-1.html [2] E. Li, L. Zeng, Z. Zhou, and X. Chen, “Edge AI: OnDemand Accelerating Deep Neural Network Inference via Edge Computing,” Oct. 2019, arXiv:1910.05316 [cs]. [Online]. Available: http://arxiv.org/abs/1910.05316 [3] T. Ogino, S. Kitagami, and N. Shiratori, “A multi-agent based flexible iot edge computing architecture and application to its.” J. Commun., vol. 14, no. 1, pp. 47–52, 2019. [4] E. Mueller, C. Lane, Y. Mai, P. Valencia, and N. Wang, “An efficient deterministic edge traffic distribution network-on-chip routing algorithm design.” J. Commun., vol. 13, no. 10, pp. 594–600, 2018. [5] B. Jacob, S. Kligys, B. Chen, M. Zhu, M. Tang, A. Howard, H. Adam, and D. Kalenichenko, “Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference,” Dec. 2017, arXiv:1712.05877 [cs, stat]. [Online]. Available: http://arxiv.org/abs/1712.05877 [6] S. Han, H. Mao, and W. J. Dally, “Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding,” Feb. 2016, arXiv:1510.00149 [cs]. [Online]. Available: http://arxiv.org/abs/1510.00149 [7] G. Hinton, O. Vinyals, and J. Dean, “Distilling the Knowledge in a Neural Network,” Mar. 2015, arXiv:1503.02531 [cs, stat]. [Online]. Available: http://arxiv.org/abs/1503.02531 [8] T. N. Sainath, B. Kingsbury, V. Sindhwani, E. Arisoy, and B. Ramabhadran, “Low-rank matrix factorization for Deep Neural Network training with high-dimensional output targets,” in 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. Vancouver, BC, Canada: IEEE, May 2013, pp. 6655–6659. [Online]. Available: http://ieeexplore.ieee.org/document/6638949/ [9] P. Amanatidis, G. Iosifidis, and D. Karampatzakis, “Comparative Evaluation of Machine Learning Inference Machines on Edgeclass Devices,” in 25th Pan-Hellenic Conference on Informatics. Volos Greece: ACM, Nov. 2021, pp. 102–106. [Online]. Available: https://dl.acm.org/doi/10.1145/3503823.3503843 [10] H. Feng, G. Mu, S. Zhong, P. Zhang, and T. Yuan, “Benchmark Analysis of YOLO Performance on Edge Intelligence Devices,” Cryptography, vol. 6, no. 2, p. 16, Apr. 2022. [Online]. Available: https://www.mdpi.com/2410-387X/6/2/16 [11] D. Nair, A. Pakdaman, and P. G. Pl¨ oger, “Performance Evaluation of Low-Cost Machine Vision Cameras for Image-Based Grasp Verification,” Mar. 2020, arXiv:2003.10167 [cs]. [Online]. Available: http://arxiv.org/abs/2003.10167 [12] D. Cantero, I. Esnaola-Gonzalez, J. Miguel-Alonso, and E. Jauregi, “Benchmarking Object Detection Deep Learning Models in Embedded Devices,” Sensors, vol. 22, no. 11, p. 4205, May 2022. [Online]. Available: https://www.mdpi.com/1424-8220/22/11/4205 [13] T.-Y. Lin, M. Maire, S. Belongie, L. Bourdev, R. Girshick, J. Hays, P. Perona, D. Ramanan, C. L. Zitnick, and P. Doll´ ar, “Microsoft COCO: Common Objects in Context,” Feb. 2015, arXiv:1405.0312 [cs]. [Online]. Available: http://arxiv.org/abs/1405.0312 [14] W. Ding, Z. Huang, Z. Huang, L. Tian, H. Wang, and S. Feng, “Designing efficient accelerator of depthwise separable convolutional neural network on FPGA,” Journal of Systems Architecture, vol. 97, pp. 278–286, Aug. 2019. [Online]. Available: https://linkinghub.elsevier.com/retrieve/pii/S1383762118304612 [15] A. G. Blaiech, K. Ben Khalifa, C. Valderrama, M. A. Fernandes, and M. H. Bedoui, “A Survey and Taxonomy of FPGA-based Deep Learning Accelerators,” Journal of Systems Architecture, vol. 98, pp. 331–345, Sep. 2019. [Online]. Available: https://linkinghub.elsevier.com/retrieve/pii/S1383762118304156 [16] A. Shawahna, S. M. Sait, and A. El-Maleh, “FPGA-based Accelerators of Deep Learning Networks for Learning and Classification: A Review,” IEEE Access, vol. 7, pp. 7823–7859, 2019, arXiv:1901.00121 [cs]. [Online]. Available: http://arxiv.org/abs/1901.00121 [17] T. Q. Thinh, J. Tang, Q. D. La, and T. Q. S. Quek, “Offloading in Mobile Edge Computing: Task Allocation and Computational Frequency Scaling,” IEEE Transactions on Communications, pp. 1–1, 2017. [Online]. Available: http://ieeexplore.ieee.org/document/7914660/ [18] S. Y. Nikouei, Y. Chen, S. Song, R. Xu, B.-Y. Choi, and T. R. Faughnan, “Real-Time Human Detection as an Edge Service Enabled by a Lightweight CNN,” Apr. 2018, arXiv:1805.00330 [cs]. [Online]. Available: http://arxiv.org/abs/1805.00330 [19] Q. Yang, Y. Liu, T. Chen, and Y. Tong, “Federated Machine Learning: Concept and Applications,” Feb. 2019, arXiv:1902.04885 [cs]. [Online]. Available: http://arxiv.org/abs/1902.04885 [20] Y. Matsubara, S. Baidya, D. Callegaro, M. Levorato, and S. Singh, “Distilled Split Deep Neural Networks for Edge-Assisted RealTime Systems,” in Proceedings of the 2019 Workshop on Hot Topics in Video Analytics and Intelligent Edges. Los Cabos Mexico: ACM, Oct. 2019, pp. 21–26. [Online]. Available: https://dl.acm.org/doi/10.1145/3349614.3356022 [21] A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications,” Apr. 2017, arXiv:1704.04861 [cs]. [Online]. Available: http://arxiv.org/abs/1704.04861 [22] A. Krizhevsky, “Learning multiple layers of features from tiny images,” University of Toronto, 05 2012.