Energy-Efficient and Data-Optimized Federated Learning for Distributed on-Device Intelligence

IEEE TRANSACTIONS ON CONSUMER ELECTRONICS 1 Energy-Efficient and Data-Optimized Federated Learning for Distributed On-Device Intelligence Ioannis Protogeros, Maria Diamanti, Member, IEEE, Dimitrios Spatharakis, and Symeon Papavassiliou, Senior Member, IEEE Abstract—The advent of Artificial Intelligence (AI), requiring large data from diverse devices, necessitates novel architectures that incorporate edge devices in the pipeline of AI operations. Federated Learning (FL) is becoming the state-of-practice for distributed, collaborative training of AI models at the network edge, respecting data privacy and anonymity concerns. However, implementing FL on battery-powered edge devices with limited computing capabilities necessitates innovative approaches to balance training accuracy and energy efficiency. In this work, we propose a framework for energy-efficient and data-optimized FL to facilitate on-device intelligence. The proposed framework jointly optimizes the (a) selection of the most important data samples from the devices’ local training datasets to maintain FL model accuracy, and their (b) computing frequency and (c) uplink transmission power to minimize the energy consumption during local computation and communication phases. A multiprocessor model is considered for each device, extending data selection to allocate data samples to each device’s processors. The initially formulated non-convex and combinatorial optimization problem is decomposed into sub-problems, each equivalently transformed into a convex form. An Alternating Optimization (AO) approach is then applied to the sub-problems, yielding a close-to-optimal solution to the original problem. The proposed algorithm is distinguished in terms of scalability, achieving at least 30% energy reduction on the MNIST and CIFAR10 datasets compared to the second-best benchmarking schemes, while maintaining FL model accuracy at nearly the same level. Overall, the proposed framework proves scalable and suitable for deployment in realistic resource-constrained edge environments. Index Terms—Federated learning, on-device intelligence, data selection, energy efficiency, transmission power control, computing frequency scaling, alternating optimization. I. INTRODUCTION The advent of cutting-edge Artificial Intelligence (AI) applications and Large Language Models (LLMs) [1] requires a vast amount of training data originating from diverse devices at the network edge. In this context, Federated Learning (FL) [2] has emerged as an effective technique for distributed and collaborative training of AI models from multiple sources of information. Unlike conventional cloud-based learning approaches, FL is typically conducted at the network’s edge, where data I. Protogeros, M. Diamanti, D. Spatharakis, and S. Papavassiliou are with the Institute of Communication and Computer Systems (ICCS), School of Electrical and Computer Engineering, National Technical University of Athens, Zografou, Greece, 15780, e-mail: [email protected]; [email protected]; [email protected]; papav[email protected]. This work was partly supported by Project 6G-LEADER funded by the European Union supported by Smart Networks and Services Joint Undertaking (SNS JU) (Grant 101192080). Manuscript received 14 June, 2025; revised 18 August, 2025. is generated. This minimizes communication delays caused by constant data exchange while preserving privacy by keeping sensitive data on the device [3]. Specifically, the edge devices update their local models using their local datasets and transmit the model parameters to a central entity (i.e., edge server), where the weights are aggregated to generate a global model. This novel architecture paves the way for a new generation of Intelligent Cyber-Physical Systems (ICPSs) [4], [5], characterized by on-device AI. Nevertheless, several challenges arise concerning the quality of FL training while meeting several key requirements, such as response time and energy efficiency. Implementing FL over wireless networks exacerbates these challenges by factors such as spectrum scarcity and the inherent battery limitations of edge devices [6]. These issues hinder the performance of distributed learning by causing latency and increasing interference during the transmission of local model parameters. Non-Orthogonal Multiple Access (NOMA) techniques can enhance spectrum efficiency by allowing simultaneous transmissions over the same time-frequency resources, differentiating devices by their power levels. Therefore, power control solutions must be concurrently considered to ensure feasible multi-device channel sharing while operating within the time limits imposed by the FL process [7]. Beyond efficient communication, the practical implementation of FL necessitates ensuring the sustainability of edge devices from the computing viewpoint, especially when considering batterypowered edge devices with limited computing capabilities [8]. Executing computationally-intensive AI workloads in parallel with frequent communication can be highly power-consuming, resulting in battery drainage. Consequently, it is essential to meticulously design solutions for energy-efficient AI operations that guarantee consistent performance and reliable model training while jointly minimizing the power consumption of the devices’ computing resources and data transmission. While several seminal works have proposed energy-efficient solutions for FL through combined radio and compute resource allocation [9], [10], the joint optimization of training data remains underexplored. Optimizing the size and quality of the local training data heavily affects overall FL performance in accuracy and energy consumption, being closely intertwined with computing resource allocation at the devices. Together with radio resource allocation, significant interdependencies are introduced among the optimization variables, resulting in a particularly intricate problem. A holistic framework that spans all phases of the FL pipeline, optimally handling data, computing, and radio resources while balancing model accuracy with total energy consumption, is currently missing. IEEE TRANSACTIONS ON CONSUMER ELECTRONICS 2 In this work, we aim to bridge the gap in the literature regarding joint energy-efficient and data-optimized FL operations. To this end, we focus on an FL scenario where a centralized edge server trains a global image classification model by aggregating parameters from edge devices. The edge devices use multiple processors for local training and communicate the local model parameters over the same time-frequency resources using the power-domain NOMA technique. In this multi-processor and shared-radio-resource setting, we jointly optimize data selection, uplink transmission power allocation, and computing frequency scaling at each device and processor. Contrary to similar studies that simplify specific model aspects and treat data selection and resource allocation unilaterally, we propose a single, low-complexity, and scalable framework that jointly addresses this multi-variable and highly non-convex optimization problem within the considered complex FL setting. The main contributions of this work are summarized as: •The joint problem of local training data selection for each device and its processors, uplink transmission power allocation, and frequency scaling for each device processor is formulated. The aim is to maximize FL efficiency and minimize energy consumption across all phases of the FL pipeline, including computing and communication. •The initially formulated highly non-convex problem is decomposed into three sub-problems, concluding close-tooptimal solutions for the data selection, computing, and radio resource allocation separately. The sub-problems are iteratively solved using Alternating Optimization (AO) until the original objective function converges. The proposed algorithm has a polynomial complexity to the number of edge devices, resulting in a scalable and computationally efficient solution. •The overall proposed energy-efficient and data-optimized FL framework is evaluated through modeling and simulation, demonstrating its scalability and energy efficiency in comparison to benchmark schemes for each individual sub-problem. The MNIST and CIFAR10 datasets are considered for local training, introducing diversity and enhancing the validity of the proposed framework. •The proposed framework achieves comparable FL model accuracy and energy consumption with off-the-shelf solutions, while reducing real execution time by 1.5 orders of magnitude. The data selection scheme achieves 95% and 83% sample reduction for the MNIST and CIFAR10 datasets, respectively. Overall, the framework results in at least 30% energy reduction on both the MNIST and CIFAR10 datasets, while maintaining FL model accuracy close to the second-best benchmarking schemes. The remainder of the paper is organized as follows. Section II summarizes the most relevant studies in the literature. In Section III, the system model is presented along with the formulation of the joint optimization problem. Section IV discusses the original problem decomposition, the solutions to each specific sub-problem, and the overall proposed AO framework. Section V presents the numerical evaluation of the proposed framework, and Section VI concludes the paper and discusses improvements and extensions of this work. II. RELATED WORK In this section, we discuss the most relevant and representative subset of works in the literature on data selection and resource allocation in FL systems. Interested readers can refer to the survey in [11], covering several aspects of on-device intelligence, such as deployment scenarios and model pruning, to facilitate FL in resource-constrained environments. A. Data Selection for FL Training data selection on edge devices is crucial to balance FL model accuracy and local computation energy consumption at, making it an actively evolving research area, e.g., [12]–[16]. For instance, the work in [12] presents one of the first attempts at modeling data importance in FL networks by proposing a data selection and communication resource allocation algorithm that leverages the squared estimated gradient norm to guide importance-aware data selection, thereby reducing end-to-end latency and improving learning efficiency in FL systems. At the same time, stochastic data selection techniques exist, such as Mercury in [13], which uses stochastic importance sampling to ensure unbiased convergence to the same solution as standard distributed Stochastic Gradient Descent (SGD), offering theoretical guarantees. Other works focus on adaptive data selection at each FL round to optimize specific performance metrics such as accuracy, energy efficiency, or communication overhead, e.g., [14], [15]. FedOL [14] Unmanned Aerial Vehicles (UAV) executing an FL process, while dynamically adjusting the size of local datasets during training rounds based on their real-time status and FL deadlines. The algorithm selects the most important samples by prioritizing newly added online data collected by the UAVs, assigning them the highest priority by setting their loss value to the maximum among existing samples. In this way, the UAVs focus on fresh data and avoid redundant training. Following a similar rationale, the authors in [15] propose an adaptive data selection strategy that enhances training efficiency while reducing the communication overhead due to redundant data transmissions. The employed selection strategy adaptively selects data by starting with a minimum sample size, gradually increasing it based on model performance, and updating the model in batches until a predefined accuracy threshold is reached. This ensures efficient training under dynamic resource conditions. For the interested reader, the survey in [16] presents a review of device selection methods with an emphasis on data selection in a comprehensive manner. B. Computing and Radio Resource Optimization in FL The performance of FL depends heavily on the allocation of computing and radio resources, especially when (i) training is performed by battery-powered edge devices and (ii) model updates are transmitted over wireless channels. For this reason, the problems of computing and radio resource allocation have gained much attention, being addressed both independently and jointly. Indicative works focusing exclusively on the computing aspect for FL include [17], [18]. The work in [17] considers a two-tier FL system, where Internet of Things (IoT) IEEE TRANSACTIONS ON CONSUMER ELECTRONICS 3 devices with multi-processor computing capabilities execute AI training tasks in parallel. The aim is to efficiently distribute training tasks across processors to balance the computational load and minimize energy consumption. Contrary to [17], the authors in [18] explore the performance trade-offs in FL between different computing resources, namely CPUs and GPUs, while dynamically allocating training tasks across both resource types, aiming to optimize accuracy and latency. While GPUs accelerate AI training with parallel processing, it is highlighted that CPUs offer more energy-efficient solutions. Considering wireless FL systems, radio resource allocation has been studied from various viewpoints, considering wireless channel reliability and key metrics, such as global model accuracy, training loss, and energy consumption, e.g., [19]– [21]. On the one hand, bandwidth or subchannel allocation alongside power control has been the focus of works such as [19] and [20], respectively, while modeling the probability of successful transmission over the wireless medium due to limited bandwidth. A different stream of research focuses on non-orthogonal multiplexing of the edge devices’ transmissions of local model updates to the server, e.g., [21], allowing for more efficient utilization of the available bandwidth. In this context, power control is critical to mitigate interference and ensure effective signal decoding while targeting FL-related metrics and energy efficiency. A remarkable amount of work can also be found on joint efforts regarding the allocation of compute and radio resources for FL. The most widely pursued optimization objective is to minimize the total energy consumption for both computing and communication operations while adhering to bandwidth and latency constraints (e.g, [22]) or learning performance constraints like the global FL loss (e.g., [23]). Both works in [22], [23] consider orthogonal sharing of the available bandwidth and tackle the problem of bandwidth or rate allocation along with transmission power control and computing frequency allocation. A similar energy consumption minimization objective is addressed in [24] with the difference that powerdomain NOMA is assumed for better resource utilization. Multi-objective optimization is also of interest, but it has been explored to a lesser extent. Indicatively, the minimization of global FL loss function and total energy consumption is tackled in [25]. Also, the work in [26] seeks to minimize both the time and energy overheads due to computing and communication, while optimizing the FL process’s convergence speed. Similar communication settings and optimization variables to [22], [23] are considered in [25], [26], neglecting to account for the scarcity of bandwidth resources via NOMA techniques. Moreover, none of the works in [22]–[26] models the multi-processor case from a computing perspective, while the problem of data selection is majorly overlooked under joint compute and radio resource optimization works in FL. III. SYSTEM MODEL We consider the distributed ICPS illustrated in Fig. 1, where a set of edge devices N={1,2, . . . , N}, with constrained energy and computing capabilities, perform FL tasks. The proposed ICPS optimizes the AI model training pipeline using Fig. 1: High-level Architecture of the ICPS. FL, by dealing with three main problems namely; (a) the data selection on each edge device (b) the frequency scaling for the multi-processors of the device, and (c) the power control for efficient data transmission. In this work, and for proof-ofconcept purposes, we consider a typical image classification FL task, which has broad industrial and practical applications in the ICPS domain, such as vehicular networks [4] or smart-city environmental monitoring systems using consumer devices [5]. The ICPS model can be extended to integrate edge devices with intermittent energy harvesting from ambient sources (e.g., solar, wind, radio frequency) [27]. In this case, ensuring continuous FL operations and long-term sustainability requires accounting for the interplay between harvested energy and the energy consumed for local training and uplink transmission, which further complicates data selection and resource allocation, and is part of our future work. Each device npossesses a labeled training dataset Dn= {xn,l, yn,l}Dn l=1, containing Dnsamples, where xn,l is the l-th input sample and yn,l is the corresponding class label in the FL task. In each FL round i, each edge device trains a local model with parameters wn. The goal of local training is to minimize the loss between the output of the on-device neural network and the target label yn,l for each input sample xn,l. Let Ψ(xn,l,wn)denote the output of the on-device neural network for input xn,l ∈ Bnwith parameter vector wn. Also, define as ℓ(Ψ(xn,l,w), yn,l)the loss function that quantifies the per-sample error after forward propagation. To reduce computational overhead and increase learning efficiency, local training is performed on a subset of the local dataset, Bn⊆ Dn,∀n, which comprises the most important samples for device n. To determine the subset Bn, the importance of each sample lof device nis first estimated using the squared norm of the gradient as follows [28]: σn,l =     ∂ℓ(Ψ(xn,l,w), yn,l) ∂xL n,l      2 2 ,(1) where ∥·∥2is the L2 norm and xL lis the input to the last layer’s activation function for sample l. This is a well-founded heuristic metric based on the assumption that samples with larger gradients contribute more significantly to model updates. Thus, a larger σn,l indicates that sample lhas a higher impact on the loss and can contribute to faster convergence. By sorting the IEEE TRANSACTIONS ON CONSUMER ELECTRONICS 4 samples xn,l ∈ Dnaccording to their importance, the subset Bnof most ones can be derived. Estimating sample importance during the forward pass allows performing backpropagation only on the selected samples. This reduces computation since backpropagation typically requires approximately twice as many FLOPs per sample compared to forward pass [29]. To further determine the specific number Bnof samples for each device (i.e., the cardinality of Bn), we define the data importance function g(Bn)of device nas follows: gn(Bn) = Bn X i=1 σn,i, Bn≤Dn.(2) This function captures the device’s contribution to the global loss reduction when selecting the top Bnsamples from Dn with the highest significance values, considering an ordered sequence σn,1≥σn,2≥ · · · ≥ σn,Dn, starting from the most important ones. Notably, gn(Bn)is concave with respect to Bn, as discussed in [12], when Bnis relaxed to a continuous variable. By incorporating the function gn(Bn)along with the system’s total energy consumption, we formulate and solve the joint problem of data selection and transmission power and computing frequency allocation toward energy-efficient and data-optimized FL operations, as detailed in Section IV. Having concluded the training subset Bn⊆ Dnof Bn samples, the local training loss of device nis calculated as: Ln(Bn,wn) = Bn X l=1 ℓ(Ψ(xn,l,wn), yn,l),∀n∈ N .(3) Toward minimizing the aforementioned loss, each device performs κlocal model updates, indexed by j. In each iteration j, the local model parameters wnfor each device nare updated using the gradient descent rule with learning rate η∈[0,1] as follows: wj+1 n=wj n−η∇Ln(Bn,wj n).(4) After local training, each device communicates its local gradient gn=1 Bn∇Ln(Bn,wi n)to the coordinating server. The server then aggregates these local gradients to produce a global gradient gi+1 used to update the global model wi+1 for the next FL round i+ 1. Specifically, the global gradient is computed as a weighted average of the local gradients: gi+1 =1 Pn∈N BnX n∈N Bngi n.(5) Using the global gradient gi, the global model for the FL task is updated for FL round i+ 1 as follows: wi+1 =wi−ηg.(6) In the remainder of Section III, we drop the index i associated with the FL rounds for notation simplicity. A. Computing Model Following the work in [30], the edge devices are equipped with a multiprocessing model of CPUs. In particular, each device is equipped with Qnprocessors which define a set Qn. Let fq n[FLOPs/sec] denote each processor’s scaling frequency that operates between a minimum and maximum value, i.e., fq n∈[fq,min n, fq,max n]. The edge devices model training is performed in parallel [31], meaning the computational workload is delegated to the available processors. We define the sets Bq n⊆ Bn, which represent the set of samples that each processor q∈ Qnwill handle. For the partitioning of the devices’ datasets to their processors, it holds that ∪Qn q=1Bq n=Bn. Accordingly, we define the workload (in FLOPs) of each processor as Wq n=Bq n·NF LOP S, where NF LOP S is the number of FLOPs needed for processing a sample. Therefore, for device n, the computing time required by each processor to process its workload is expressed as: Tcmp n,q =Wq n fq n ,∀q∈ Qn[sec].(7) As a result, the maximum time for the local model update for edge device nis: Tcmp n=Qn max q=1 Wq n fq n[sec].(8) Moreover, we define the power consumption for each processor following the standard Dynamic Voltage Frequency Scaling (DVFS) modeling utilized for CMOS circuits to express the power as a function of the CPU frequency, following [31]: Pcmp n,q =Cq n(fq n)3[Watt],(9) where Cq n[Watt/(FLOPs/sec)3] determines the efficiency of each processor, i.e., the power rate growth compared to the increase of the computing requirements. Given the duration of local training for each processor to complete the associated tasks as defined in Eq. (7), we can express the total energy consumption of edge device nas follows: Ecmp n= Qn X q=1 Cq nWq n(fq n)2[Joule].(10) It is noted that the proposed computing model could be further complemented with intelligent AI optimization techniques, such as federated dropout or pruning of models, that dynamically adapt model architecture to reduce computational complexity and preserve device energy [32]. B. Communication Model The users’ local gradients are transmitted to the coordinating server over the same time-frequency resources of total bandwidth B[Hz] using the power-domain Non-Orthogonal Multiple Access (NOMA) technique to ensure efficient spectrum reuse. Let Gndenote the channel gain between user nand the server. Without loss of generality, assume that the channel gains between users and the server are sorted in ascending order, G1≤···≤Gn≤···≤GN, and the decoding of the signals begins from the highest channel gain user using the Successive Interference Cancellation (SIC) technique. The achieved uplink data rate of user nto the server is calculated as: Rn=BW log2 1 + Gnpn Pn−1 n′=1 Gn′pn′+N0BW ![bps], (11) IEEE TRANSACTIONS ON CONSUMER ELECTRONICS 5 where pn[Watt] is the uplink transmission power of user nand N0[dBm/Hz] is the power spectral density of zeromean Additive White Gaussian Noise (AWGN). Aligned with common assumptions in the literature, we assume perfect Channel State Information (CSI) and ideal implementation of the SIC technique [24]. In case of imperfections, additional variables should be introduced to capture the uncertainties in the channel gains and the decoded interference, respectively. Accordingly, the transmission time of user nfor communicating its local gradient gnto the server is: Ttx n=V(gn) Rn [sec],(12) where V(gn)[bits] is the data size of the local gradient vector gn, which is equal for all users in the system. Moreover, the associated transmission energy consumption is given by: Etx n=V(gn)pn Rn [Joule].(13) C. Problem Formulation In this work, our objective is to maximize FL efficiency by optimizing the number of local training samples and minimizing the energy consumption in both local computation and communication of gradients to the server. To this end, we formulate the joint optimization problem of the number of local training samples Bnper device and the number of processed samples Bq nper device processor, the uplink transmission power pnof each device to the server, and the frequency scaling fq nfor each device processor. Specifically, in each FL round, we optimize the following objective function to balance data selection and energy efficiency across all edge devices: F(Bn, Bq n, fq n, pn) = N X n=1 gn(Bn)−y N X n=1 Ecmp n+Etx n. (14) The first term measures data importance, while the second term represents the total computation and communication energy consumption. The constant parameter y∈R+serves as a scaling factor to balance the tradeoff between these two terms. Hence, we define the following joint optimization problem: P: max {Bn,Bq n,fq n,pn}F(Bn, Bq n, fq n, pn)(15a) s.t. Tcmp n≤Tcmp max,∀n, (15b) Ttx n≤Ttx max,∀n, (15c) Tcmp n+Ttx n≤Tmax,∀n, (15d) fq,min n≤fq n≤fq,max n,∀n, q, (15e) 0≤pn≤pmax n,∀n, (15f) 0≤Bn≤Dn,∀n, (15g) Qn X q=1 Bq n=Bn,∀n. (15h) Constraints (15b) and (15c) ensure that the computing and communication times per device remain below their respective maximum thresholds. Constraint (15d) is the total time constraint for an FL round per device, calculated as the sum of their computing and communication times. Constraint (15e) ensures that the frequency of each processor operates between the minimum and maximum values. Constraint (15f) ensures that the uplink transmission power for each edge device is below the maximum level. Finally, Constraints (15g) and (15h) guarantee that the selected samples for local model updating will not exceed the samples of the local dataset and will equal the allocated workload to the available processors of each device. It should be noted that a continuous relaxation of the integer variables Bn, Bq n,∀q∈ Qn, n ∈ N is selected to tractably derive a solution. However, as demonstrated in the following sections, the corresponding decision variables are ultimately mapped back to the discrete space. Problem Pis non-convex due to the non-convexity of the objective function, while the optimization variables Bn, Bq n, fq nare highly coupled, complicating the derivation of a tractable solution. Therefore, it is challenging to obtain a global optimal in polynomial time. In the following, we outline the methodology to decompose the original highly non-convex problem into independent convex sub-problems, the solutions of which separately provide sub-optimal solutions to the joint energy-efficient and data-optimized FL optimization problem. IV. ENERGY-EFFICIENT AND DATA-OPTIMIZED FL SOLUTION In this section, the original optimization problem Pis decomposed into three independent sub-problems, namely the on-device data selection (Section IV-A), computing resource allocation (Section IV-B), and radio resource allocation (Section IV-C). Each sub-problem is analytically solved while treating the others as fixed, as analytically described in the respective subsections. Last, Section IV-D details the iterative approach based on the principles of AO [33], [34], according to which the solutions of the sub-problems are applied in an alternative manner until convergence of the objective function is achieved. A. On-Device Data Selection First, we delve into the problem of on-device data selection, aiming to determine the optimal number of samples Bnfor each device and Bq nfor each of its processors, while efficiently allocating the samples to the processors. By fixing the decision variables regarding the frequency scaling of all processors fn,q and the transmission power of each edge device pn, problem Pis reformulated as follows: P1 : max {Bn,Bq n} N X n=1 g(Bn)−y N X n=1 Ecmp n+Etx n(16a) s.t. Qn max q=1 Wq n fq n≤Tmax n,∀n, (16b) 0≤Bn≤Dn,∀n, (16c) Qn X q=1 Bq n=Bn,∀n. (16d) IEEE TRANSACTIONS ON CONSUMER ELECTRONICS 6 where Tmax n= min Tmax −V(gn) Rn, Tcmp maxis the maximum acceptable computing time per device, derived by jointly considering constraints (15c) and (15d). Through introducing the auxiliary variables tn,∀n, problem P1is equivalently transformed into problem P1′to remove the max operator and obtain a continuous expression of the original constraint (16b). Also, by substituting Wq n=Bq n· NF LOP S, problem P1is rewritten as: P1′: max {Bn,Bq n,tn} N X n=1 g(Bn)−y N X n=1 Ecmp n+Etx n(17a) s.t. Bq nNF LOP S fq n ≤tn,∀n, q (17b) tn≤Tmax n,∀n(17c) 0≤Bn≤Dn,∀n, (17d) Qn X q=1 Bq n=Bn,∀n. (17e) Problem P1′is a concave problem, consisting of a concave objective function with respect to Bn, Bq n,∀n∈ N,∀q∈ Qn, and convex sets of constraints. Thus, problem P1′can be efficiently solved using standard numerical methods such as the interior point method. However, in the following, we use Lagrange dual decomposition to derive closed-form expressions given the Lagrange multipliers, guaranteeing that the optimal solution is obtained within polynomial time [35]. The Lagrangian function (see [35], Theorem 2.1) of problem P1′is written as: L=− N X n=1 g(Bn) + y N X n=1 Ecmp n+Etx n+ N X n=1 αn(Bn−Dn) + N X n=1 Qn X q=1 λn,q Bq n·NF LOP S fq n −tn+ N X n=1 µn(tn−Tmax n) + N X n=1 βnBn+ N X n=1 νn Qn X q=1 (Bq n−Bn),(18) where αn, λn,q, µn, βn, vn≥0are the Lagrangian multipliers associated with the constraints (17b)-(17e). By calculating the KKT conditions, we obtain: ∀n:∂L ∂Bn =−∂g(Bn) ∂Bn +αn+βn−νn= 0 (19) ∀n:∂L ∂Bq n =y∂Ecmp n ∂Bq n +λn,q NF LOP S fq n +νn= 0 (20) ∀n:∂L ∂tn =− Qn X q=1 λn,q +µn= 0 (21) ∀n:λn,q Bq nNF LOP S fq n −tn= 0 (22) ∀n:αn(Bn−Dn)=0 (23) ∀n:µn(tn−Tmax n)=0 (24) ∀n:−βnBn= 0 (25) Given that problem P1′is concave, the KKT conditions are both necessary and sufficient for optimality. To obtain the optimal solution for the data selection problem, the following cases are considered. First, we investigate the general case that λn,q = 0, meaning that the time constraint (17b) is inactive. Then, by substituting stationary condition (19) to the stationary condition (20) yields: ∂g(Bn) ∂Bn −y∂Ecmp n ∂Bq n −αn−βn= 0.(26) If the complementary slackness condition (25) is active, i.e., β > 0, then the corresponding constraint (17e) must hold, i.e., B∗ n= 0 which is rejected. If the condition is inactive, i.e., β= 0, then by the complementary slackness condition (23) if αn>0then B∗ n=Dn, otherwise the optimal solution for B∗ nis obtained by solving Eq. (26): ∂g(Bn) ∂Bn =y∂Ecmp n ∂Bq n =y Qn X q=1 Cq nNF LOP S(fq n)2.(27) As discussed in Section III, function g(Bn)is concave and from Eq. (27) the value of ∂g(Bn) ∂Bnis known. Subsequently, we can determine the discrete value of B∗ nby utilizing the inverse function g−1 n(Bn)and mapping the result to the closest discrete value. In this way, the optimal number of selected samples B∗ nis determined for each device n. Furthermore, by solving Eq. (22), the optimal number of samples Bq∗ nfor each processor q∈ Qnof device nis derived. Having calculated both B∗ nand Bq∗ n,∀n, q, the allocation of samples to each device’s processors is then performed by sorting the processors according to their energy efficiency, as detailed in Algorithm 1. Algorithm 1 Data Allocation to the Device’s Processors 1: Bn= 0 and Q∗ n=∅ 2: Compute B∗ nvalue from Eq. (27). 3: Compute optimal value t∗ n= min Tmax −V(gn) Rn, Tcmp max 4: Sort processors of edge device naccording to Cq n(fq n)2 value to form set Q′ n. 5: for each processor q∈ {1,...,Q′ n}do 6: Solve Eq. (22) and obtain Bq∗ n=fq n NF LOP S t∗ n. 7: Bn+= Bq∗ n 8: Add qto the set of active processors Q∗ n=Q∗ n∪ {q}. 9: if Bn≥B∗ nthen 10: Bq′∗ n←0for all q′∈ {q+ 1, . . . , Q′ n} 11: break for loop; 12: end if 13: end for 14: return B∗ n,{Bq∗ n}, t∗,Q∗ n The overall procedure is repeated for all devices as described in Algorithm 1. Specifically, after calculating t∗ n(Step 3), the set Q′ nis constructed by sorting the processors according to the terms Cq n(fq n)2(Step 4) which indicates their energy efficiency regarding their chosen frequency. As a result, the algorithm starts allocating data samples to the most efficient processor until constraint (17b) is not violated (Step 6) for each processor. Moreover, the set of active processors Q∗ nis updated with this processor (Step 8). In case the total allocated data Bnon the device so far exceeds the optimal value B∗ n(Step 9), the rest of the processors are not assigned any data samples IEEE TRANSACTIONS ON CONSUMER ELECTRONICS 7 (Step 10) and we consider that they are not operational in this round. Otherwise, the algorithm proceeds with assigning the remaining data to the rest of the less efficient processors. The algorithm returns the binary values for B∗ n, Bq∗ nand the set of active processors for each device Q∗ n. The complexity of Algorithm 1 is mainly driven by the sorting operation of the processors at Step 4 and the iterations over all processors at Step 5. Specifically, sorting the processors of each device at Step 4 has a complexity of O(Qnlog(Qn)), using well-known sorting algorithms, e.g., Merge Sort. The loop in Step 5 involves algebraic calculations of O(1) complexity over at-worst Qniterations, leading to an overall complexity of O(Qn)for the loop. Consequently, the worst-case complexity of Algorithm 1 is O(Qnlog(Qn)). Finally, if λn,q >0meaning that the time constraint (17b) is active, then Eq. (22) yields Bq∗ n=tnfq n NF LOP S . Therefore, if µn>0then the complementary slackness condition (24) yields the optimal solution t∗ n=Tmax n,Bq∗ n=Tmax nfq n NF LOP S and using the primal feasibility condition (17e) we get B∗ n= PQn q=1 Bq∗ n. B. Computing Resource Allocation Given the data selection, the data allocation to each device’s processors and the set of active processors Q∗ n, the frequency allocation for the processors that will process data samples is optimized by solving: P2 : min {fq n} N X n=1 Q∗ n X q=1 Cq nBq nNF LOP (fq n)2(28a) s.t. Bq nNF LOP ≤fq nTmax n,∀n, q ∈ Q∗ n(28b) fq,min n≤fq n≤fq,max n,∀n, q ∈ Q∗ n,(28c) where Tmax n= min Tmax −V Rn , Tcmp max. Problem (28) is a quadratic problem with linear constraints, however strictly convex. The objective function is increasing with regards to the frequencies, and thus obtaining a closed-form solution is trivial: fq∗ n= max(fq,min n,(Bq nNF LOP )/T max n).(29) C. Radio Resource Allocation Given the data selection and frequency allocation solutions Bn, Bn,q, fn,q for each device and its respective processors, the uplink transmission power for communicating local gradients to the server for each device is optimized by solving: P3 : min {pn}Etx n= N X n=1 pnTtx n(30a) s.t. Ttx n≤Ttx,max n,∀n, (30b) 0≤pn≤pmax n,∀n, (30c) where Ttx,max n= min (Tmax −tn, Ttx max)is the maximum allowable transmission time of each device, derived based on the optimized computing time tnobtained through data selection (Section IV-A), the total time constraint Tmax for an FL round, and the predefined maximum transmission time Ttx max. Problem P3is challenging to solve due to the non-convexity of the objective function with respect to pn,∀n, and the interdependence of the devices’ transmission powers stemming from the shared communication channel and the resulting interference among them. To address this challenge, we leverage the fact that transmission time Ttx n,∀nis upper-bounded by constraint (31b). Following established approaches from the literature [36], we can fix Ttx n,∀nand reformulate the problem as: P3′: min {pn} N X n=1 pn(31a) s.t. V(gn) BW log21 + Gnpn Pn−1 n′=1 Gn′pn′+N0BW =Ttx,max n,∀n, (31b) 0≤pn≤pmax n∀n, (31c) where constraint (31b) replaces constraint (30b). This reformulation continues to contribute to minimizing transmission energy consumption, since the transmission energy Etx n= pnTtx n=pnV(gn) Rndecreases more rapidly with a reduction in pn(linear relationship) than with an increase in Rn(logarithmic relationship). Problem P3′is a Linear Programming (LP) problem, whose solution is derived by solving constraints (31b) for pn,∀n, leading to the following set of equations: p1=2V(g1)/(BW ·Ttx,max 1)−1 G1 ·N0BW p2=2V(g2)/(BW ·Ttx,mx 2)−1 G2 ·(N0BW +G1p1) . . . pn=2V(gn)/(BW ·Ttx,max n)−1 Gn · N0BW + n−1 X n′=1 Gn′pn′!,∀n. (32) D. Data-optimized and Energy Efficient Algorithm for FL The solution to the joint data-optimized and energy-efficient FL is analytically presented in Algorithm 2. Algorithm 2 is executed at each FL round i, optimizing the data selection and computing and radio resource allocations to maximize FL efficiency and minimize total energy consumption. Additionally, Algorithm 2 operates in iterations indexed by t, where the solutions of the sub-problems are updated until the original objective function Fconverges, meaning its value remains unchanged between consecutive algorithm iterations. To calculate the complexity of Algorithm 2, we first consider the complexity of deriving the solution to each subproblem. The complexity for determining the data selection for all devices using Algorithm 1 is O(Qnlog(Qn). The computing frequency allocation has a closed-form optimal solution for each device, yielding a total complexity of O(N) for all devices. The complexity of solving the system of linear equations to determine the uplink transmission powers IEEE TRANSACTIONS ON CONSUMER ELECTRONICS 8 of all devices is also O(N). Let Tdenote the number of iterations required for the AO approach to converge. Then, the overall algorithm’s complexity is O(T·(Qnlog(Qn)+N)). Numerical results regarding the number of AO iterations and real execution time required for Algorithm 1 to conclude a solution are presented in Section V, emphasizing its remarkably lower complexity compared to standard solvers available in optimization toolboxes. Algorithm 2 Overall Data-Optimized and Energy-Efficient FL Algorithm 1: Set AO iteration index t←0. 2: Initialize a feasible solution for problem P: {Bn(0), Bq n(0), fq n(0), pn(0)}∀(n,q). 3: repeat 4: t=t+ 1 5: Calculate {Bn(t), Bq n(t)}∀(n,q)based on Algorithm 1, for given {fq n(t−1), pn(t−1)}∀(n,q). 6: Calculate {fq n(t)}∀(n,q)based on Eq. (29), for given {Bn(t), Bq n(t), pn(t−1)}∀(n,q). 7: Calculate {pq n(t)}∀nby solving problem (32), for given {Bn(t), Bq n(t), fq n(t)}∀(n,q). 8: until |F(t)−F(t−1)|< ϵ, ϵ →0 9: return {B∗ n, Bq∗ n, fq∗ n, p∗ n}∀(n,q) V. EVALUATION AND RESULTS In this section, we evaluate the performance of the joint data-optimized and energy-efficient FL algorithm and solution via modeling and simulation. The simulation setup and optimization parameters are initialized as follows. We consider a wireless FL system deployed within a circular area of 200 m radius. The server is located at the center of the system and N= 10 edge devices are uniformly randomly distributed. The channel gain between the devices and the server is calculated according to the distance-based path loss model from 3GPP [22], [24], PL = 128.1 + 37.6 log(d), where dkm is the Euclidean distance between them. The total system bandwidth is BW = 20 MHz. The rest of the communication-related parameters are set as; pmax n= 24 dBm and N0=−134 dBm/Hz. Each edge device nis equipped with Qn= 4 processors, considering frequency scaling within the range fq,min n, fq,max n= [1,3] GHz. The capacitance coefficient of each processor qis uniformly distributed within the range Cq n∼[0.01,1] W(MFLOPs/s)−3. The maximum time threshold for an FL round is Tmax = 500 ms, with 20% allocated to communication (Ttx max = 100 ms) and 80% to computing (Tcmp max = 400 ms). The FL model is trained on the MNIST dataset [37], unless otherwise explicitly stated. The larger and more complex CIFAR10 dataset [38] is also adopted to enhance the validity and applicability of the proposed data selection scheme, while allowing direct comparison and reproducibility with other works in the literature. From the total of 60,000 samples of both datasets, 50,000 are used for training and the remaining 10,000 for testing. The training samples are evenly distributed among the devices to perform image classification, i.e., 5,000 samples/device. Note that effectively handling heterogeneous data distributions would benefit from an additional mechanism, beyond on-device data selection using the gradient norm, integrated into our framework to cumulatively evaluate data importance across all devices. The FL process is performed until model accuracy converges, while the learning rate of the local training is set equal to η= 10−3. The number of FLOPs required for processing one MNIST sample is NF LOP s = 106, while the size of the communicating gradients to the server is calculated as V(gn) = 723.04 Kbits. For the MNIST dataset, each node’s local model uses a Convolutional Neural Network (CNN) for image classification, consisting of two convolutional layers with 10 and 20 filters, respectively, each followed by a ReLU activation function and a2×2max pooling layer. The resulting feature maps are flattened and passed through a fully connected layer with 50 neurons. The output layer contains 10 neurons with softmax activation function, equal to the number of MNIST classes. The training is performed using the categorical cross-entropy loss function. For the CIFAR10 dataset, the designed CNN consists of two convolutional blocks with 32 and 64 filters, respectively, each followed by a ReLU activation function and a 2×2max pooling layer. The result is flattened and passed through a fully connected dense layer with 128 neurons and ReLU activation. The output layer contains 10 neurons using a Softmax activation, corresponding to the classes of the CIFAR10 dataset. A. Evaluation of Overall Algorithm’s Convergence and Performance First, we examine the convergence behavior of the proposed Algorithm 2 with respect to the number of AO iterations Trequired to conclude the solution for the joint problem. Fig. 2a depicts the total data importance, calculated using the function in Eq. (2), and total energy consumption resulting from both computing and communication as a function of the AO iterations. The results have been normalized relative to their initial values at the start of Algorithm 2 to more accurately reflect the tradeoff between data importance and energy consumption. The results reveal that after approximately 30 AO iterations, the overall algorithm converges to the final solution. More importantly, though, it is observed that energy consumption decreases more rapidly than the total data importance in the system, highlighting the effectiveness of the proposed solution in utilizing energy more efficiently for handling fewer yet significant local data samples. Fig. 2b and 2c further demonstrate the convergence of each device’s transmission and computing energy, respectively. Both figures show that the solution converges to lower energy consumption values from both computing and communication perspectives, although this trend is more pronounced for computing energy due to its larger scale and range of values compared to transmission energy. Furthermore, to evaluate the effectiveness of Algorithm 2, we compare its performance to the Trust-Constr algorithm [39] used for solving complex non-linear constrained optimization problems. The Trust-Constr algorithm is widely implemented IEEE TRANSACTIONS ON CONSUMER ELECTRONICS 9 10 20 30 40 50 AO Iterations 0 0.5 1 Normalized Total Data Importance / Energy Total Data Importance Total Energy (a) 10 20 30 40 50 AO Iterations 0 0.2 0.4 0.6 Transmission Energy [J] N=10 (b) 10 20 30 40 50 AO Iterations 0 0.5 1 1.5 2 2.5 Computing Energy [J] N=10 2468 1 2 First 8 AO Iterations (c) Fig. 2: Convergence analysis of the (a) total importance of selected data and energy consumption, (b) computing energy per device, and (c) transmission energy per device, over AO iterations. 5 10 15 20 Number of Devices 0.835 0.84 0.845 0.85 0.855 0.86 Accuracy Proposed Trust-Constr Algorithm (a) 5 10 15 20 Number of Devices 0 2 4 6 Total Energy [J] Proposed Trust-Constr Algorithm (b) 5 10 15 20 Number of Devices 0 0.5 1 1.5 Proposed-to-Trust-Constr Objective ratio (c) 5 10 15 20 Number of Devices 0 50 100 150 Execution Time [s] Proposed Trust-Constr Algorithm (d) Fig. 3: Performance analysis of the Proposed and Trust-Constr algorithms, in terms of (a) accuracy, (b) total energy consumption, (c) proposed-to-trust-constr objective ratio, and (d) real execution time, for different numbers of devices. by various optimization toolboxes, such as Python’s scipy, and, in this particular evaluation scenario, is used to solve the original problem P. The aim of this experiment is to compare the tradeoff between performance and scalability achieved by the proposed and Trust-Constr algorithms in terms of FL model accuracy, energy consumption across the system, and real execution time. To this end, we consider an increasing number of devices in the system from N= 5 to N= 20. Throughout this experiment, we consider that each device possesses 500 MNIST samples, such that the cumulative knowledge in the system increases with the addition of more devices. To ensure the feasibility of the total FL round constraint with double the maximum number of devices in the system, we also doubled the total time threshold to Tmax = 1000 ms. Fig. 3a and Fig. 3b illustrate the achieved FL model accuracy and total energy consumption, respectively, as a function of the number of devices for both algorithms. The results reveal that the two algorithms achieve nearly identical global FL model accuracy. However, the proposed algorithm manages to conclude lower total energy consumption for the system while this performance gap between the algorithms becomes more pronounced as the number of devices increases. This outcome is expected, as for smaller-scale problems, the solver can explore the solution space more thoroughly within a reasonable time. Fig. 3c summarizes the performance gain of the proposed algorithm compared to the Trust-Constr solver by depicting the ratio of the objective function value achieved by each algorithm. The performance improvement consistently exceeds 25% and grows as the number of devices increases. Notably, what particularly distinguishes the proposed solution is the time required to execute the algorithm, presented in Fig. 3d. The real execution time required by the Trust-Constr algorithm to converge is approximately 1.5orders of magnitude higher than that of the proposed Algorithm 2, rendering the former significantly impractical for real FL deployments. This trend is consistent with the derived complexity OT· (Qnlog(Qn)+N)of Algorithm 2 (see Section IV-D), where the execution time grows at most linearly with N, thus explaining the observed near-constant or mildly increasing runtime with N. In contrast, the off-the-shelf Trust-Constr algorithm solves iterative quadratic subproblems, the size of which grows with the number of optimization variables and constraints (both proportional to Nin our setting). Summarizing, the proposed algorithm based on conventional optimization and AO techniques strikes a good balance between FL efficiency, total energy consumption, and real execution time. B. Evaluation of On-Device Data Selection In this section, we evaluate the performance of data selection for each device and the allocation to the device’s respective processors using the solution described in Section IV-A and Algorithm 1. To this end, we first examine the pure performance of data selection in Fig. 4, followed by a comparative evaluation against benchmark data selection schemes in Fig. 5. Aiming to assess the validity and applicability of our proposed on-device selection scheme across varying levels of classification task complexity, experiments on both the MNIST and CIFAR10 datasets were conducted.