On the Use of Artificial Neural Networks for the Automated High-Level Design of ΣΔ Modulators

Depósito de investigación de la Universidad de Sevilla https://idus.us.es/ This is an Accepted Manuscript of an article published by IEEE in IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS I-REGULAR PAPERS on 2024, available at: https://doi.org/10.1109/TCSI.2023.3338056 “© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other Works” 1 On the Use of Artificial Neural Networks for the Automated High-Level Design of Σ∆ Modulators Pablo D´ ıaz-Lobo, Gustavo Li˜ n´ an-Cembrano and Jos´ e M. de la Rosa, Fellow, IEEE Abstract—This paper presents a high-level synthesis methodology for Sigma-Delta Modulators (Σ∆Ms) that combines behavioral modeling and simulation for performance evaluation, and Artificial Neural Networks (ANNs) to generate high-level designs variables for the required specifications. To this end, comprehensive datasets made up of design variables and performance metrics, generated from accurate behavioral simulations of different kinds of Σ∆Ms, are used to allow the ANN to learn the complex relationships between design-variables and specifications. Several representative case studies are considered, including single-loop and cascade architectures with single-bit and multi-bit quantization, as well as both Switched-Capacitor (SC) and Continuous-Time (CT) circuit techniques. The proposed solution works in two steps. First, for a given set of specifications, a trained classifier proposes one of the available Σ∆M architectures in the dataset. Second, for the proposed architecture, a Regression-type Neural Network (RNN) infers the design variables required to produce the requested specifications. A comparison with other optimization methods – such as genetic algorithms and gradient descent – is discussed, demonstrating that the presented approach yields to more efficient design solutions in terms of performance metrics and CPU time. Index Terms—Design Automation, Optimization, Neural Networks, Analog-to-Digital Converters, Sigma-Delta Modulation. I. INTRODUCTION SIGMA-Delta Modulators (Σ∆Ms) are one of the best techniques to implement Analog-to-Digital Converters (ADCs) in a wide range of applications – from instrumentation, biomedical devices and automotive sensors to communications. In spite of their potential advantages in terms of robustness and efficiency, the design of high-performance Σ∆Ms involves a number of tasks carried out in a topdown/bottom-up design flow from systems to chip, which requires a certain degree of know-how and expertise. In most cases, the main design bottleneck is at the system level, where finding the best Σ∆M architecture for a given set of specifications, and mapping those specifications onto circuitlevel electrical design parameters become crucial to get a good design choice [1], [2]. Over the years, a number of design methodologies and CAD tools have been presented to automate the design of Σ∆Ms. These tools are mostly focused on optimizing system-level Manuscript submitted August 16, 2023; reviewed October 7, 2023. This work was supported in part by Grants PID2019-103876RB-I00, PID2022-138078OB-I00, funded by MCIN/AEI/10.13039/501100011033, by the European Union ESF Investing in your future, by ERDF A way of making Europe, and by ”Junta de Andaluc´ ıa” under Grant P20-00599. Pablo D´ ıaz-Lobo, Gustavo Li˜ n´ an-Cembrano and Jos´ e M. de la Rosa are with the Institute of Microelectronics of Seville, IMSE-CNM (CSIC/University of Seville), Parque Tecnol´ ogico de la Cartuja, C/ Am´ erico Vespuccio 28, 41092 Sevilla, SPAIN, e-mail: [pabdiaz,jrosa]@imse-cnm.csic.es, gustav[email protected] tasks – such as architecture selection, loop-filter design, behavioral modeling, simulation and sizing, as well as electrical design and validation [3]–[9]. The most general approach is based on the so-called optimization-based synthesis method, where an optimizer is combined with a performance evaluator (usually a simulator) to find the optimum design. A number of optimization algorithms have been proposed to this end, including genetic, simulated annealing, or multi-objective evolutionary Pareto fronts, to cite a few [3], [4], [7], [10]. Recent works demonstrate that Artificial Intelligence (AI) algorithms can be applied to automate, or to assist, analog circuit design [11]–[19]. Their use in Σ∆Ms has been employed to improve the performance metrics of Σ∆Ms and other ADCs by means of linearization or calibration techniques based on Artificial Neural Networks (ANNs) [20], [21]. Some authors have proposed using ANNs in an optimization-based synthesis methodology [11]–[13], [15], [20], [22]. In some of them, the ANN has been trained to replace the simulator, while other approaches consider ANNs as an optimization engine. In the latter case, the ANN is trained to size a given system for a set of specifications. Thus, the ANN should be trained with sized solutions known from prior optimized designs. Once the ANN is trained, it is able to automate the sizing process and generate optimum design solutions for new sets of specifications which were not considered in the training dataset. This method has been successfully applied to design essential analog circuits such as operational amplifiers [11]. In our initial publication on this topic [23], we investigated the use of manually designed neural networks in deriving high-level design variables for some Σ∆Ms architectures. Here we present the evolution of this initial concept into a self-contained comprehensive framework for automatic architecture selection and high-level design. In this framework, neural networks with optimized architectures, combined with a library of classifiers topologies, are the core of a methodology that, starting from very high-level and general specifications, can swiftly provide users with a suitable topology and its corresponding high-level design variables in negligible time (less than 50 milliseconds in our simulations). We demonstrate the versatility of our approach, applicable to either single-bit or multi-bit quantization, single-loop or cascade configuration, discrete or continuous time operation, with a case study that embeds an extensive dataset encompassing various Σ∆M architectures, including 2nd-order SC-Σ∆M, 3rd- and 4th- order cascade 2-1 and 2-1-1 SC-Σ∆M, as well as a 2nd- order Gm-C Σ∆M. The networks have been automatically optimized and trained to learn the complex mapping from specifications onto high-level design variables, enabling them to predict the most suitable set of design variables for a 2 Modulator Specifications Resolution ? Bandwidth? Power cons.? Behavioral Simulation & Optimization Circuit Design and Implementation Nonlinear Equations Heuristic Know-How Heuristic Know-How Sigma-Delta Synthesis Tool Schreier’s Toolbox Architecture Selection (L, OSR, B) ? ADC DAC ! g1 g'1 ! g2 g'2 - + + - - + + - + (GBW, SR, Adc,…) ? Optimiser SIMSIDES Fig. 1. Usual design flow diagram applied to the design of Σ∆Ms. given set of specifications. The results achieved with our proposed method are compared with those obtained using other optimization algorithms, demonstrating very significant improvements in terms of the CPU time. The paper is organized as follows. Section II revisits prior art on design methodologies and EDA tools for Σ∆Ms, with emphasis on the optimization-based system-level sizing procedure. Section III describes the proposed ANN-based highlevel synthesis methodology. Section IV presents the results for some case studies encompassing four different modulator architectures. Section V discusses the comparison with other existing approaches. Finally, conclusions are drawn in Section VI. II. BACKGROUND AND PRIOR ART ON OPTIMIZATION-BASED SYNTHESIS OF Σ∆MS The design methodology of Σ∆Ms follows the well-known hierarchical top-down/bottom-up approach. In this divide-and- conquer strategy, a Σ∆M is partitioned into several abstraction levels (system-level, building block-level, circuit-level and device/physical-level), so that at each abstraction level, a design (or sizing) process takes place, to transmit (or to map) the system specifications in a hierarchical way – from Σ∆M architecture to circuit [24]. This process is conceptually depicted in Fig. 1 for the different tasks involved in the system-level design of Σ∆Ms: from specifications to building-block/circuit-level implementation. Starting from the Σ∆Mspecifications; i.e. Effective Number of Bits (ENOB) and signal bandwidth (BW), the first design problem consists of finding out the best modulator architecture that meets such specifications with the minimum power consumption. At the first step, ideal design equations – without including the effect of circuit errors – of the Noise Transfer Function (NTF) and ENOB are used to get an estimation of main Σ∆M system-level parameters, i.e. OverSampling Ratio, OSR, loop-filter order, L, and number of bits of the embedded quantizer, B. Once these parameters are known, the Σ∆M is analyzed by using more accurate nonlinear model equations. Schreier’s MATLAB®Σ∆-toolbox is widely used to do this task [25]. This toolboox is a collection of MATLAB scripts that include nonlinear models of Σ∆Ms, which can be applied to automatically synthesize a Σ∆M Loop Filter (LF), providing the desired noise shaping while keeping system stability [1], [9]. Once the Σ∆M architecture has been selected, the next design task is to analyze the influence of circuit nonidealities to obtain the specifications of main Σ∆M building blocks, i.e. the amplifiers, comparators, switches, etc. To this purpose, an accurate – but computationally efficient – performance evaluator, usually a simulator, is used. The modulator needs to be modeled at a high abstraction level, albeit taking into account the effect of nonideal physical phenomena at circuit and device level. An efficient technique to do this is the so-called behavioral modeling approach, which allows to simulate Σ∆Ms in a fast but precise way [4]. Fig. 1 highlights some exemplary behavioral simulators which are normally used by Σ∆M designers. One of them is the socalled SIMSIDES [8]– a time-domain simulator developed in the MATLAB/SIMULINK environment and available at www.imse-cnm.csic.es/simsides. This tool is able to simulate any arbitrary Σ∆M, by considering the effect of main circuit nonidealities. Behavioral models included in SIMSIDES have been verified by electrical simulations and experimental measurements from state-of-the-art chips in order to guarantee the accuracy of the simulations. Behavioral models of building blocks are implemented in C-coded MATLAB S-functions to keep high accuracy and high computational efficiency – typically a 216 clock-cycle simulation takes a few seconds CPU time [4]. Another widely used system-level simulation approach – mostly focused on CT-Σ∆Ms – is based on the so-called lifting method [7]. This approach allows to speed up the simulation by using Discrete-Time (DT) solvers and hardware (GPU) acceleration, drastically reducing the simulation time – typically hundreds or thousands of simulations are done within seconds – while keeping high accuracy. Based on this technique, a web-based tool named Sigma-Delta Synthesis Tool [26] is available at www.sigma-delta.de. This tool also provides a design automation environment for CT-Σ∆Ms, including Signal Transfer Function (STF) engineering and automated LF scaling. Behavioral simulators are usually guided by an optimization engine to explore the multi-dimensional design space and automate the high-level sizing process. The final objective is to find the optimum solution, i.e. the best set of design variables (building-block performance parameters), that satisfy the Σ∆M ADC specifications with the minimum power 3 dissipation. This paper proposes replacing the optimizer– behavioral simulator system with neural networks that will learn, from extensive datasets, how to map a point P within the specifications space onto a point Q in the high-level designvariables spaces. This work assumes that the first steps of the synthesis procedure shown in Fig. 1, i.e. the internal architecture of a given modulator type, the scaling of loopfilter coefficients, Out-of-Band Gain (OBG), etc. are given. However, ANNs might also be applied to this purpose. The procedure – beyond the scope of this paper – would essentially be the same as that followed in this work but applying the sizing process at a higher abstraction level. In such a case, the design variables would be the loop-filter coefficients, and the Σ∆M subcircuits (integrators, resonators, comparators, etc.) should be considered as ideal blocks. III. PROPOSED METHODOLOGY A. Problem Definition As stated in previous section, the high-level design of a Σ∆M involves solving two types of problems. First, for a given vector of specifications, ¯ Γ, a suitable architecture has to be selected from a family of alternatives {Aj}, where jrepresents available topologies. In our case, specifications typically involve ENOB, BW, Signal-to-Noise Ratio (SNR), Signal-to-Noise and Distortion Ratio (SNDR), Total Harmonic Distortion (THD), a Figure of Merit (FOM)1, etc., but they might also include constrains imposed by higher hierarchy considerations if the modulator is to be embedded in a more complex system (e.g. forcing discrete-time vs. continuous time operation or vice-versa, defining a maximum area occupation, setting a maximum power consumption, etc.), and even include constrains imposed by application-related scenarios (i.e. military, industrial, space, automotive, mass market,...). Based on the selected topology, a vector of architecture-dependent design variables ¯ϵ(Aj)needs to be found in such a way that specifications ¯ Γare not only met but also optimized according to some predefined metrics. B. Proposed Solution For the first problem, finding a suitable architecture from the information in our dataset, we propose to consider it as obtaining an inference from a trained classifier, C, which maps system performance metrics (¯ Γi) into a categorical variable Aj(the selected architecture). The process of selecting and training the classifier is described in detail in Section III-D. The second problem must yield the design variables which guarantee that the received set specifications are met for the chosen Σ∆M architecture. Here we propose utilizing Regression-type Neural Networks (RNNs), trained on comprehensive datasets, to infer these design variables. Hence, our methology, detailed in the flow charts in Fig. 2, can be basically described by these two operations: Aj=C(¯ Γi);¯ϵk=RNN(¯ Γi, Aj)(1) 1The Schreier’s Figure of Merit – denoted as FOMS [1] and widely adopted by Σ∆M community – will be used in this work. Where C(·)is a call to the classifier to obtain an architecture, and RNN(·)is a call to the neural network inference process to generate the design variables. Section III-E provides detailed information on how we propose to optimally design and train the regression neural networks. Utilizing pre-trained classifiers and networks offers the significant advantage of reducing the inference time, i.e., the time required to select a modulator architecture and obtain its high-level design variables from a new set of specifications, to just a few milliseconds. Furthermore, memory limitations and CPU time-related concerns are alleviated as they pertain to the dataset generation and training processes, which need to be executed only once or infrequently when incorporating new datasets/architectures. However, the primary drawback of this approach is that the quality of the obtained solution heavily relies on the accuracy (in a broad sense) of both the classifier and the networks, which, in turn, are highly dependent on the quality of the available dataset [27]. C. Designing the Dataset Our proposal is to format every input diin the dataset as a triplet of the form {Ci,¯ Γi,¯ϵi}where Ciis a categorical variable which defines the architecture of the modulator, ¯ Γi is a vector of Σ∆M performance metrics, (i.e., a point in the specifications space) and ¯ϵiis a vector which contains the design variables (i.e., a point in the design-variables space) that produced such metrics in a behavioral simulation of this modulator architecture using SIMSIDES [2], [4]. Regardless of the specific modulator architecture, ¯ Γimust always be made up of the same performance metrics for all the architectures being incorporated into our framework. This is simply justified by the fact that the classifier needs to operate on ¯ Γacross the whole dataset in order to find the best-suited solution for a particular set of specifications ¯ Γi. However, different modulators will require different high-level design variables to be found. Thus, if a unified ¯ϵiformat is to be used for all the architectures under consideration, we will need to define the ¯ϵivector with as many elements as there are different design variables across all the modulators in our dataset, forcing us to pad with ”N/A” (Not Applicable), and handle it, all inputs that do not correspond to the specific modulator in each particular dataset entry. Though this solution offers the advantage of using a single dataset file, it lacks scalability as it necessitates recreating this file every time a new architecture is added to our framework and retraining the classifier and the RNN from scratch. Alternatively, one can choose to create a specific dataset in the form of {Ci,¯ Γi,¯ϵi}for each modulator, where Ciremains the same for all inputs in each particular dataset. Individual networks can then be tailored for each case, optimizing them specifically for their corresponding architectures. This approach enables us to optimize a network for each modulator, training it solely on specific {¯ Γi,¯ϵi}data, rather than seeking an optimum and more complex solution that handles the entire dataset encompassing all architectures. Thus, (1) suffers an apparently simple but conceptually crucial transformation becoming: Aj=C(¯ Γi);¯ϵk=RNNAj(¯ Γi)(2) 4 Arch.1 Arch.2 Arch.N Family of Architectures Generate design var vectors (thousands) Datasets Dataset Arch. 1 Dataset Arch. 2 Dataset Arch. N Behavioral Simulation SIMSIDES Classif. 1 Classif.2 Classif.K Family of Classifiers For each architecture obtain metrics For each Classifier Train & Evaluate Accuracy Downsample & Create dataset for the classifiers Select Best Classifier Classifier For each Dataset Use NAS to optimize RNN architecture RNN training & validation Save to file Save model to file RNN for this SDM Arch. Dataset for this Architecture Save to file (a) Training procedure {Γk, Ck} {CN,¯ ΓN,¯ǫN} (b) Inference procedure Specifications Classifier Obtain Architecture Select RNN for this architecture RNN Model Files RNN N RNN 2 RNN 1 RNN k Select Best FOM DESIGN Save to file Design vars Run inference Fine Tuning Behavioral Simulation SIMSIDES Generate small variations Fig. 2. Flow diagrams for the proposed methodology. (a) Training and dataset preparation. (b) Inferring a design from specifications. where, now, the regression operation RNN(¯ Γi, Aj)in (1) has been transformed in RNNAj(¯ Γi), i.e, the role of the classifier is selecting a specific network from a library of already trained ones, and none of the networks receive the categorical architecture variable as input. As far as the training of the classifier is concerned, it will only require reading the Ci,¯ Γiinformation for each modulator in the dataset and simply concatenate the information before training. In our opinion, having specific datasets for each architecture is a better solution, considering both scalability and neural network performance. Thus, adding a new modulator architecture to the framework only requires creating its particular dataset in the behavioral simulator, training the classifier, and optimizing and training its associated network, which is preferable to creating a completely new dataset file by augmenting the number of columns and adding ”N/A” in all the previously existing columns, and then training again a single network whose complexity grows as new architectures are added. D. Considerations on the Classifier Definition Selecting an appropriate architecture for a classifier is a crucial step in machine learning and data analysis. The choice of architecture directly impacts the model’s performance and its ability to generalize well on unseen data. However, determining the optimal classification technique is not a one-size- fits-all process. It heavily depends on various factors, including the characteristics of the dataset, statistical properties of the data, and the specific problem at hand [28]. The dataset plays a central role in architecture selection, as its size, complexity, and distribution of classes influence the model’s behavior. Imbalanced datasets, where certain classes are significantly more abundant than others, pose a challenge as they can lead to biased predictions towards the majority classes [29]. In our case, we propose to always feed the classifier with a subsampled version of the data available for each of the architectures in our framework in such a way that all classes will have same presence. Moreover, the statistical properties of the data, such as feature correlation, co-linearity, dimensionality, and noise, also influence the architecture selection. Since we are not apriori imposing any limitation on which data is to be present in the metrics vector {¯ Γ}(until a case study is presented in Section IV), we have implemented a procedure which explores different classifier types and returns the classifier featuring a better confussion matrix for the particular dataset under consideration. Our solution currently covers: Quadratic and Linear Discriminant Analysis [30], Support Vector Machines SVM [31] with linear, polynomial, and radial-basis function kernels, Gaussian and Multinomial Naive-Bayes [28], Decission Trees [32], Random Forest [33], Gradient Boosting Classifiers [34], and neural network-based classifiers [35]. E. Network Architecture Search and Model Optimization The backbone of our solution consists of utilizing an optimized multivariate regression network [36] for each modulator architecture to map system metrics (specifications) onto highlevel design variables, making the network to play the role 5 Fig. 3. Network Template Architecture which is optimized using NAS of the optimizer in the conventional approach depicted in Fig. 1. However, the problem of manually defining a network architecture which minimizes the regression error2usually implies a tedious and time-consuming process where network parameters such as number of layers, neurons per layer, etc., are varied in an iterative way until a solution is accepted. To overcome this, we have integrated Neural Architecture Search (NAS3) [37] techniques in our design framework to automatically explore the hyper-parameters space and identify a network architecture for each modulator using the Keras tuner API [38], [39]. Our solution first defines a parameterized network template, shown in Fig. 3, and a search space for the following hyper-parameters: •Number on neurons (units) per layer: ∈[32,64] step 4. •Number of additional layers after the concatenation block in Fig. 3: ∈[1,6] step 1. •Activation function: ∈[relu, tanh], where relu stands for Rectified-Linear Unit and tanh stands for hyperbolic tangent. •Dropout layers [40]: Whether to insert dropout layers between dense layers or to bypass them. •Optimizer: ∈ {Adam, SGD, RMSprop, Adadelta}. and then runs the grid-search optimization until the search space has been exhausted or an early-stop condition4occurs. Once the hyper-parameters for each network are available, the fitting process takes place using a 80% - 20% division for the training and validation subsets (TS, VS). For any existing {¯ Γk,¯ϵk}pair in the training dataset (TS) of a particular Σ∆M architecture, the RNN produces a regression output ¯ϵ′ ksuch that, overall, during the training process, a loss function (MSE in our case) ET S =f( ¯ϵk,¯ϵ′ k), k ∈T S is minimized. Then, the network validation process checks for over-fitting5by evaluating this same metric Eover the unseen data in the validation set (VS); EV S =f(¯ϵj,¯ϵ′ j), j ∈V S, letting us guarantee that whenever |ET S −EV S|< δ, with delta a predefined threshold, the network has been able to generalize the problem with an accuracy EV S. 2Assuming a dataset with j= 1...M inputs, and a i= 1...N-elements target yiand output vectors ˆyi, then MSE =1 N·MPN i=1 PM j=1(yi,j −ˆyi,j )2. 3NAS is a technique that automatically explores and finds the bestperforming neural network architecture for a specific task. 4Meeting specifications or not improving the solution in a predefined number of iterations known as the patience parameter. 5It occurs when a neural network becomes overly specialized in capturing noise and random fluctuations in the training set so that it performs exceptionally well on its training data but fails to generalize to new, or unseen data. F. Cross-Validation and Result Improvement Unfortunately, using the MSE to quantify the accuracy of the networks only guarantees that the errors between the inferred design variables vector ¯ϵ′ kand the target design variables ¯ϵkare, on average, bounded by the MSE. However, due to the highly non-linear nature of Σ∆Ms, it does not provide a clear insight about how closely the performance metrics ¯ Γ′ kobtained for the inference design variables ¯ϵ′ kmatch the requested ones ¯ Γk. To address this concern, our methodology incorporates a final step which involves using the behavioral simulator [4] not only for cross-validation but also for result improvement. In order to do so, we first create a small number (niters) of slightly varied versions of the solution provided by he network ¯ϵ′ kwhere each of its components (DVn, n = 1...Nvars)6is randomly modified by adding a small amount taken from an uniform distribution within ±r(%) ×100 range: DVn,a =DVn(1 + αn,a), where a= 1 . . . niters n= 1 . . . Nvars αn,a ∼U(−r, r) (3) Essentially, this cloud of points represents a random sampling of a small, well-defined region within the design-variables space around the point selected by the network where the average distance of these additional points to the original solution is constrained to be less than the MSE. Afterwards, we employ the behavioral simulator to evaluate the resulting performances not only for the point predicted by the network ¯ϵ′ kbut also for the cloud of random points around it.{¯ϵ′ k,a}a=0...niters : {¯ Γ′ k,a}=BS(Aj,{¯ϵ ′ k,a})(4) where Ajis the selected modulator architecture, BS(·)is a call to the behavioral simulator, and {¯ϵ′ k,0} ≡ ¯ϵ′ k. Finally, we select the high-level design point which produces the best Schreier’s FOM [1]. Conceptually, our approach can be described as a two-step process. First, the neural network provides a coarse solution for the design variables which serves as a starting point. In the second step, a small random search is conducted around this initial solution to identify similar solutions featuring better FOM. Clearly, this second step is open to further development by leveraging advanced optimization solutions, such as gradient-driven methods or simulated annealing, among others, representing an excellent opportunity for future improvements of our design framework. 6DV stands for Design Variable and Nvars is the number of design variables. 6 (a) (c) (b) (d) Fig. 4. SIMSIDES models of the Σ∆Ms used as case studies: (a) 2nd-order SC-Σ∆M (I). (b) 3rd-order cascade 2-1 SC-Σ∆M (II). (c) 4th-order cascade 2-1-1 SC Σ∆M with 3-bit quantization (III). (d) 2nd-order Gm-C Σ∆M with 3-level quantization (IV). (All models include main circuit nonidealities such as limited input/output swings, thermal noise, finite DC gain, GBW, maximum output current, nonlinear transconductance, etc. [2]). IV. CASE STUDY The versatile design methodology proposed in this paper can be applied to any arbitrary Σ∆M. Without loss of generality, let us consider four case-study architectures which include single-loop and cascade loop-filter topologies, singlebit and multi-bit quantization as well as switched-capacitor and continuous-time circuit techniques. Fig. 4 shows the SIMSIDES diagrams of the Σ∆Ms under study: (I) a 2nd-order SC-Σ∆M (Fig. 4(a)), (II) a 3rd-order cascade 2-1 SC-Σ∆M (Fig. 4(b)), (III) a 4th-order cascade 2-1-1 SC-Σ∆M with 3-bit quantization (Fig. 4(c)) and (IV) a 2nd-order 3-level CT-Σ∆M based on Gm-C integrators. Regardless of the architecture, the specifications vector ¯ Γcomprised the required modulator resolution, in terms of the SNR, the signal bandwith BW (expressed by the OSR), and the power consumption (Pow) ¯ Γk≡ {SNRk,OSRk,Powk}. Conversely, the design variables vector ¯ϵkwere obviously different for each architecture. In order to generate this case study’ dataset, we ran behavioral simulations of each architecture under consideration using the design variables and ranges listed in Table I for five values of the OSR, namely: OSR= 32,64,128,256,512. For two of the cases, the 2nd-ord Gm-C-Σ∆M and the 3th-ord 2-1 SC- Σ∆M, the variables space was covered using nested loops, 4th-Ord 2-1-1 SC 2nd-Ord Gm-C 2nd-Ord SC 3rd-Ord 2-1 SC Predicted Label 4th-Ord 2-1-1 SC 2nd-Ord Gm-C 2nd-Ord SC 3rd-Ord 2-1 SC True Label 0.95 0.00 0.02 0.03 0.00 1.00 0.00 0.00 0.02 0.00 0.94 0.04 0.05 0.00 0.09 0.86 GB 0.0 0.2 0.4 0.6 0.8 1.0 Fig. 5. Confusion Matrix for the Gradient Boosting (GB) Classifier. whereas for the other two cases design variables were generated randomly within meaningful predefined ranges. Moreover, for the 2-1-1 cascade architecture (case III), we considered the design variables of the first amplifier, and defined the other three amplifiers’ parameters as a scaled version of those of the first one. The dataset generation involved evaluating more than 200,000 random designs. The combined use of behavioral simulation (SIMSIDES), GPU acceleration and the MATLAB Parallel Computing ToolboxTM allowed us to obtain results at a rate of about 10,000 simulations per hour of CPU time7. Needless to mention, since we were generating random designs, not all combinations produced simulation results that were valid for classifiers and networks training as they featured poor SNR. Thus, prior to the training process, the dataset was filtered to remove designs where SNR<50dB. The final number of elements in each dataset is listed in the Valid Points column in Table I, amounting to nearly 120,000 {Ck,¯ Γk,¯ϵk} (Class, Specifications, Design Vars.) entries. A. Training Results The training involved executing two independent processes (see Fig. 2(a)). First we trained the classifiers. There, in order to avoid classimbalance related problems, we down-sampled the dataset to obtain an equal representation of all classes (Σ∆M architectures). We set a 80%-20%division for Training Set (TS) and Validation Set (VS), obtaining the results in Table II for the four best performing options. The Gradient Boosting classifier, whose confusion matrix is depicted in Fig.5, provided the best accuracy over the Validation Set (i.e. over unseen previous data) and we selected it as the best option for our case study. Second, each modulator’s neural network topology and hyperparameters were optimized using NAS, and trained afterwards. The dataset was also split in 80%-20% subsets for training and validation. 7All simulations have been carried out in a PC with a 5.0 GHz, i9-12900F CPU, 64-GB RAM and NVIDIA®GeForce RTX™3060TI GPU. 7 TABLE I DESIGN PARAMETERS AND RANGES USED TO GENERATE Σ∆MDATASETS Σ∆M Arch. Parameter(s) Description Range (Units) Scale Points Scheme Valid P. 2nd-Ord SC aoFinite DC Gain (10,1000) Log. 5×104Random 23,432 gmAmp. Transconductance (10µ, 1m), (A/V) Log. 5×104 ioMax. Amplifier Output Current (100µ, 10m), (A) Log. 5×104 VnInput-Referred Noise (10p,100n), (V/√Hz) Log. 5×104 2nd-Ord Gm-C ao{1,2}OTA1,2Finite DC Gain (10,1000) Log. 40, 40 Nested 16,334 GBW{1,2}OTA1,2GBW (1,1000), MHz Log. 25, 25 Loops IIP3,{1,2}Integrators1,23rd-order intermod. prod. (-10, 50), (dBm) Lin. 10 IIPF F FeedForward path integrators IIP3. (-10, 50), (dBm) Lin. 10 3rd-Ord 2-1 SC ao{1,2}Stage1,2Amplifiers Finite DC Gain (100,10000) Log. 50, 50 Nested 58,570 gm{1,2}Stage1,2Amplifiers Transconductance (10µ, 10m), (A/V) Log. 25, 25 Loops io{1,2}Stage1,2Max. Amplifier Output Current (100µ, 10m), (A) Log. 10, 10 4th-Ord 2-1-1 SC ao1First Stage Amplifiers Finite DC Gain (1,1000) Log. 5×104Random 19,769 gm1First Stage Amplifier Transconductance (10µ, 1m), (A/V) Log. 5×104 io1First Stage Max. Amplifier Output Current (100µ, 10m), (A) Log. 5×104 k2,3,4Scaling factors for 2nd, 3rd, and 4th (0.2,1) Lin. 5×104 Stage Amps {ao, gm, io}with respect to Amp1 We applied the grid-search NAS algorithm for each modulator and fine-tuned the resulting network template for either 2000 epochs or until an early-stopping condition was met8.Table III summarizes the training results. Notably, the MSE values for the training and validation sets were very similar in all cases, demonstrating that none of the networks experienced overfitting. The NAS CPU time varied between 1 hour and 53 minutes and 6 hours and 4 minutes due to the differences in number of output variables, optimal number of layers and dataset dimensions. Once the optimal network architecture was determined by the NAS algorithm, the finetuning of each network took between 60 and 397 seconds to complete —almost negligible when compared to the NAS CPU time. Finally, the validation took less than 0.125 seconds in the worst case (2nd- ord. SC). Note that, during this time, this network generated 4687 designs, i.e. it only required 26µs to produce the high-level design variables for every set of specifications B. Validating Results To fully validate our methodology, we randomly selected 1000 {¯ Γk,¯ϵk}pairs from the validation set (250 per architecture) and computed the relative deviation vector9¯ρkbetween the obtained specification vector, ¯ Γ′ k, and its requested value ¯ Γk. ¯ρk=¯ Γ′ k−¯ Γk ¯ Γk (5) 8The early-stopping condition was defined as no significant improvement in the loss function over the last 50 epochs. 9Here, vector division is defined as component-wise division. TABLE II CLASSIFIER TRAINING RESULTS Classifier AccT S (%) AccV S (%) CPU Time(s) Grad. Boost. 94.4 93.7 6.47 Rand. Forest 99.9 92.4 0.62 Dec. Tree 100 91.4 0.11 SVM RBF 82.8 82.9 16.09 To obtain the resulting specifications ¯ Γ′ k, we employed the method described in Section III-F, equation (3), with niters = 10 and α= 5%. Table IV shows the resulting FOM for the four architectures in our case study. Observe that the mean value of the deviation, i.e., the difference between the obtained FOM and the requested one, is very close to zero but positive in three of the cases and just slightly negative in the case of the cascade 2-1-1 SC. This result shows that the networks produced centered designs, in other words, designs meeting specifications, demonstrating that they learned to implement the specifications−→high-level design variables mapping. Besides, the probability of producing designs with better than required specifications were of at least 68.5% in the worst case (2nd-ord Gm-C) and peaked to 78.5% for the cascade 2- 1 SC. Finally, notice how the use of the small random search method, significantly improves the resulting FOM. V. DISCUSSION We have compared our solution against alternative more ”conventional” optimizers incorporated in the SIMSIDES environment [41] available in MATLAB®, namely Genetic, Gradient Descent, and Positive Basis Np1. Before entering into the comparison results it is crucial to highlight the fact that our solution does not have any optimization capability. Every network has just learned how to map specifications onto design variables that guarantee (up to a level of accuracy) that these specifications are met, whether these specifications are optimal in any way or not, is completely beyond the network. This is an important concept; the optimization that occurs during the neural network training refers to the minimization of the error in generating this mapping not to how optimal the specifications are because specifications are inputs for the network. In order to obtain the specifications to pass to the networks, we fixed OSR= 128, and commanded the optimizers to maximize SNR for each of the modulator architectures in our case study. Power consumption was estimated by the 8 TABLE III RNN TRAINING RESULTS Case Study NLayers N(a) Units Dropout Optimizer [38] # Train # Validation MSETS MSEVS T(b) NAS T(c) Train T(d) Val 2nd-ord. SC 5 32 No adam 18,746 4,687 0.034 0.033 2h.32min 63.81s 0.123s 2nd-ord. Gm-C 5 32 No adam 13,068 3,267 0.075 0.073 1h53min 105s 0.094s 3rd-ord. 2-1 SC 6 32 No RMSProp 47,862 11,965 0.031 0.028 6h4min 397s 0.025s 4th-ord. 2-1-1 SC 1 32 Yes RMSProp 15,816 3,954 0.016 0.017 2h40min 60.07s 0.088s aNumber of neurons per layer. bNAS GPU Time(s). cRNN GPU Training Time(s). dRNN GPU Validation Time(s). TABLE IV VALIDATION RESULTS Architecture: 2nd-ord SC Result (a)ρF OM0,0 (b)ρF OM10,5% Boost(%) Mean Value 0.0059 0.0184 211.9 (c)P(ρ > 0) 0.6331 0.7470 18.0 (d)Percentile(5) -0.0976 -0.0826 15.4 Percentile(50) 0.0100 0.0196 96.0 Percentile(95) 0.0814 0.0960 17.9 Architecture: 2nd-ord Gm-C Mean Value 0.0215 0.0267 20.9 P(ρ > 0) 0.6580 0.6850 4.1 Percentile(5) -0.0572 -0.0550 3.8 Percentile(50) 0.0219 0.0269 28.1 Percentile(95) 0.1201 0.1253 4.3 Architecture: 2-1 Cascade SC Mean Value 0.0638 0.0741 16.1 P(ρ > 0) 0.6979 0.7850 12.5 Percentile(5) -0.1847 -0.1656 10.3 Percentile(50) 0.0196 0.0272 38.8 Percentile(95) 0.3881 0.4018 3.5 Architecture: 2-1-1 Cascade SC Mean Value -0.0773 -0.0622 19.5 P(ρ > 0) 0.6536 0.6997 7.1 Percentile(5) -0.5917 -0.5825 1.6 Percentile(50) 0.0088 0.0102 15.9 Percentile(95) 0.0558 0.0575 3.0 aρF OM0,0is the relative deviation (see eq.(5)) for the FOM when using only the design variables inferred by the RNN. bρF OM10,5% is the relative deviation (see eq.(5)) for the FOM when improving the RNN result using the small random variations method (10 samples, ±5% of random variation). cP(ρS>0) is the probability of a given relative deviation ρSto be larger than 0. dPercentile(x)=T: When sorted, (1 −x)% of the points are above T. behavioral simulator SIMSIDES from the biasing currents obtained for the amplifiers during the SNR optimization. Then, for each architecture, we selected the best performing solution provided by the optimizers in terms of the FOM, and passed its results (SNR, OSR, Power) to the corresponding neural network to infer the design variables. Table V presents the results of this process. First, we can highlight a noteworthy aspect: our solution obtained a CPU time improvement of at least ×60 across the various modulators when compared to the available optimizers in SIMSIDES. Clearly, this is due to the fact that the optimizers do not incorporate any a priori knowledge about the problem whereas the networks, during the training phase, did learn how to precisely map specifications into design variables. Moreover, our networks are not too complex, involving just a few hundreds of neurons and a relatively low number of layers, consequently inference time is reduced to just a few milliseconds. Indeed, most of the time required to obtain the TABLE V COMPARISON WITH OTHER OPTIMIZATION ALGORITHMS Architecture: 2nd-ord SC Algorithm CPU Time (min) SNR (dB) P(mW) FOM(dB) This work 0.05 86.01 10.16 149.0 Gradient 9 85.8 10.80 148.5 Genetic 18 88.1 18.41 148.5 Np1 5 86.6 20.00 146.6 Architecture: 2nd-ord GmC This work 0.05 75.19 0.07 182.6 Gradient 388.6 0.8 185.5 Genetic 129 90.2 0.8 187.0 Np1 10 88.0 0.4 187.9 Architecture: 3rd-ord Cascade 2-1 SC This work 0.05 115.8 2.7 181.5 Gradient 11 115.7 8.0 176.7 Genetic 145 116.2 4.1 180.1 Np1 12 115.9 8.0 176.9 Architecture: 4th-ord Cascade 2-1-1 SC This work 0.05 143.44 0.6 241.5 Gradient 4 144.32 1.8 237.9 Genetic 90 144.32 13.6 229.0 Np1 10 143.9 1.5 238.55 design variables was spent in the final search among the randomly generated points which involved using the behavioral simulator 10 times –as detailed in Section IV-B. As a byproduct, due to the fine-tuning phase, our solution produced, in all cases, designs with a smaller power consumption thus yielding to better FOMs in three out of the four case studies. Probably, the most important contribution provided by our approach is the very short time required to obtain valid designs from new sets of specifications. Once the training phase is over, our framework is able to choose a modulator architecture and obtain its design variables from specifications in less than 50 milliseconds. However, we cannot neglect the fact that it requires a significant training time. In our opinion, this must be considered an initial cost that gets amortized proportionally based on the number of times the network is used. Another important concern arises from the abundance of specification-design variable pairs within the datasets: why not utilize this information directly, treating it like a Look- Up-Table (LUT) where valid designs can be readily found? In other words, when new specifications are required, why not identify the nearest corresponding point in the dataset and simply select its associated design variables as the solution? The answer to this question is far from being simple as it heavily depends on how accurate are the networks producing design vars from specs as compared to the resolution of the data in the LUT, i.e. how densely the space of variables was explored when producing the dataset. Let us employ the twodimensional example in Fig.6 to illustrate this. There, assume