Breaking bottlenecks: CPU optimization through architectural and neuromorphic techniques

 Corresponding author: Karan Diwan. Copyright © 2025 Author(s) retain the copyright of this article. This article is published under the terms of the Creative Commons Attribution License 4.0. Breaking bottlenecks: CPU optimization through architectural and neuromorphic techniques M L Sharma, Neelam Sharma, Sunil Kumar, Karan Diwan *, Vibhore Agarwal, Ansh Pathak, Shubham Gupta, Shreshth Jain and Ram Katara Department of Electronics and Communication Engineering, Maharaja Agrasen Institute of Technology, Delhi, India. World Journal of Advanced Research and Reviews, 2025, 26(02), 190-204 Publication history: Received on 17 March 2025; revised on 26 April 2025; accepted on 29 April 2025 Article DOI: https://doi.org/10.30574/wjarr.2025.26.2.1463 Abstract This research explores two different approaches to improving how computers process information efficiently. The first part uses the Gem5 simulator to test and compare three types of CPU designs—Timing Simple CPU, Minor CPU, and O3CPU—by running a basic program. We looked at how features like pipelining, caching, and branch prediction affect how fast the program runs and how efficiently the CPU works. The second part focuses on recognizing handwritten digits from the MNIST dataset using two types of AI models. One model is a traditional neural network (MLP) that runs on a standard computer setup (Von Neumann architecture), and the other is a spiking neural network (SNN) that runs on a neuromorphic system, which mimics how the human brain works. Overall, this study shows how both architectural improvements and brain-inspired computing can help solve performance and efficiency issues in modern computing systems. Keywords: CPU Optimization; Bottlenecks; Pipelining; Neuromorphic Computing; Spiking Neural Networks 1. Introduction 1.1. Method 1 CPU optimization using architectural methods The Central Processing Unit (CPU) is often referred to as the brain of the computer. It is a multipurpose, register-based, programmable, clock-driven electronic device that reads binary instructions from memory, accepts binary data, and processes the data according to those instructions. The CPU plays a central role in communicating with both memory and input/output (I/O) devices. However, the Control Unit within the CPU manages the timing and coordination of these communication processes. With advancements in integrated circuit (IC) technology, the CPU was eventually built onto a single chip, leading to the development of the microprocessor. Despite the impressive speed and capabilities of modern CPU’s, they operate below their maximum potential due to various bottlenecks, factors that limits the overall speed of execution, despite CPU’s high clock rate. One common form of a bottleneck arises due to the difference between CPU processing speed and slower speed of data transfer which is especially common in systems that follow Von Neumann architecture. In such systems, instructions and data share the same data bus leading to delays when CPU has to fetch them one after another. This part of the research focuses on understanding these bottlenecks and exploring CPU optimization techniques such as caching, pipelining, branch prediction and out of order execution. Using gem5 simulator we evaluate three built in CPU modelsTimingSimpleCPU, Minor CPU and O3 CPU each implementing different types of optimization techniques. World Journal of Advanced Research and Reviews, 2025, 26(02), 190-204 191 By running a similar workload on each model and observing their behaviour, we aim to understand how these techniques affect execution performance. 1.2. Method 2 CPU optimization using neuromorphic computing Neuromorphic computing, inspired by the structure and functioning of the human brain, presents a radically different approach to processing information. In this research, a Spiking Neural Network (SNN) is implemented on a neuromorphic system, offering an event-driven architecture that mimics the brain’s neural processes. This contrasts with the traditional Von Neumann architecture, where computation and memory are separate, creating bottlenecks in processing speed and power efficiency. The SNN model takes advantage of these neuromorphic principles to improve efficiency in tasks such as handwritten digit classification, using the MNIST dataset. Unlike conventional neural networks, SNNs process information in discrete events, allowing them to operate with lower power consumption and memory usage, making them highly suitable for real-time and resource-constrained applications. This approach explores the trade-offs in accuracy, memory usage, training time, and inference latency compared to the MLP model on the Von Neumann architecture. Despite slightly lower accuracy, the SNN model significantly reduces memory usage and latency, highlighting the potential of neuromorphic systems to overcome the limitations of traditional computing architectures, particularly in AI tasks. 2. Material and Methods 2.1. Method 1 architectural methods 2.1.1. Bottlenecks The Central Processing Unit (CPU) plays a crucial role in everyday computing—from running applications to browsing the internet and processing documents. Despite rapid advancements in processor technology, users often encounter slow or unresponsive systems, especially during multitasking or heavy workloads. This slowdown is commonly due to CPU bottlenecks—situations where the processor’s potential is limited by slower components or architectural constraints. Bottlenecks prevent the CPU from executing instructions at its maximum efficiency, leading to performance degradation. One prominent type is the Von Neumann bottleneck, which arises when the CPU must fetch both instructions and data over a single shared bus, causing delays. Other bottlenecks include memory latency, cache misses, poor branch prediction, and lack of pipelining, all of which stall the instruction pipeline or reduce throughput. Understanding these bottlenecks is key to developing architectural improvements like caching, pipelining, and out-oforder execution—techniques that aim to minimize idle CPU cycles and maximize performance. 2.2. Various types of bottlenecks are mentioned below 2.2.1. Instruction Fetch Bottleneck Happens when the CPU cannot fetch instructions fast enough, often due to a shared bus or limited instruction throughput. 2.2.2. Memory Bottleneck Occurs when memory access is slower than CPU execution speed. Leads to stalls as CPU waits for data to load. 2.2.3. Cache Bottleneck When the working set of data doesn't fit in cache, resulting in frequent cache misses and slow memory accesses. 2.2.4. Branch Prediction Bottleneck Happens when the CPU incorrectly predicts a branch, wasting cycles on wrong instruction paths. World Journal of Advanced Research and Reviews, 2025, 26(02), 190-204 192 2.2.5. Pipeline Bottleneck (or Stalls) In pipelined CPUs, hazards like data dependencies, control hazards (branches), or structural hazards can cause delays. 2.2.6. In-order Execution Bottleneck In in-order CPUs, instructions must complete in order, so one slow instruction can block all others. 2.2.7. Resource Contention When multiple instructions compete for the same execution resource (ALUs, buses, etc.). 3. Methodology To analyze the impact of different CPU optimization techniques on execution performance, we utilized the gem5 simulator, a widely used open-source platform for computer architecture research. In this study, we selected three built-in CPU models from gem5’s X86 architecture: • Timing Simple CPU – a basic in-order CPU model with no pipelining or caching. • Minor CPU – an in-order pipelined CPU that includes basic branch prediction and support for caches. • O3CPU – a complex out-of-order CPU with aggressive pipelining, advanced branch prediction, and full cache hierarchy. To ensure consistency, the same workload was executed on all three CPU models: A simple "Hello World" program compiled for the x86 architecture. The primary parameter recorded for performance comparison was execution time, measured in simulation ticks. Assuming a default CPU frequency of 1 GHz in gem5, we converted the number of ticks to seconds using the formula: Execution time (seconds) = ticks ÷ 1,000,000,000,000 This allowed for direct comparison of the time taken by each CPU model to complete the same task. The goal was to see how the inclusion of pipelining, caching, and branch prediction affects performance, thereby offering insight into the practical impact of each optimization technique. 3.1. Experiments 3.1.1. Hello World program The “Hello, World!” program is a simple application that performs a basic output operation without involving complex data processing or extensive memory usage. It typically consists of a single function call to print a short string to the console. Since the program does not involve loops, large data structures, or dynamic memory allocation, the number of instructions executed is minimal, and memory access is limited primarily to fetching the instructions themselves and accessing a small amount of data (i.e., the string "Hello, World!"). This simplicity makes it an ideal workload for isolating and analyzing the raw execution efficiency of different CPU models without interference from higher-level software behavior. 3.1.2. Timing Simple CPU Timing Simple CPU is the most basic CPU model available in the gem5 simulator. It represents a non-pipelined, in-order processor where instructions are fetched, decoded, executed, and retired sequentially. Unlike more advanced models, Timing Simple CPU does not support pipelining, branch prediction, or cache mechanisms by default. Each instruction must fully complete before the next one begins, making this model simple but inherently slow and inefficient for complex or high-throughput tasks. However, its straightforward design makes it an excellent reference point for understanding how different architectural enhancements impact CPU performance. Due to the absence of overlapping instruction stages, any delay—such as memory access latency or control hazards—stalls the entire processor, resulting in longer execution times. This model serves as a baseline in our study to highlight the performance gains achieved through pipelining and other optimizations in more advanced CPUs. World Journal of Advanced Research and Reviews, 2025, 26(02), 190-204 193 3.1.3. Possible Bottlenecks in Timing Simple CPU: Lack of Pipelining: Without pipelining, each instruction must fully complete before the next begins, leading to underutilization of CPU resources. Memory Latency: Without a cache, the CPU experiences longer wait times for memory operations, as each access must retrieve data directly from main memory. BASH COMMAND: ./build/X86/gem5.opt configs/deprecated/example/se.py -cpu-type=Timing Simple CPU -c tests/test-progs/hello/bin/x86/linux/hello OUTPUT: Exiting@ 454646000 ticks which means the execution time of Timing Simple CPU to run Hello World program is 0.454646 milliseconds. 3.1.4. X86Minor CPU Minor CPU is an in-order pipelined CPU model in gem5 that simulates a more realistic processor compared to TimingSimpleCPU. It breaks instruction execution into multiple pipeline stages—fetch, decode, execute, and writeback—allowing multiple instructions to be in different stages of execution simultaneously. This pipelining improves throughput by overlapping operations and reducing idle cycles. Minor CPU also introduces basic caching mechanisms and simple branch prediction, making it capable of handling moderate workloads with better performance than a non-pipelined architecture. 3.2. CPU Optimization Techniques in Minor CPU Pipelining is a fundamental technique used in modern CPUs to increase instruction throughput. It works by breaking down the execution of instructions into several distinct stages, such as fetch, decode, execute, and writeback. Instead of waiting for one instruction to fully complete before starting the next, pipelining allows the CPU to work on multiple instructions at once—each at a different stage. This overlapping of operations significantly improves performance by keeping different parts of the CPU active simultaneously. Branch Prediction is used to overcome one of the key challenges of pipelining—control hazards. When the CPU encounters a conditional branch (e.g., an "if" statement), it must decide which instruction path to load next. Waiting for the condition to resolve would stall the pipeline, so instead, branch prediction guesses the likely outcome of the branch. Cache is a small, high-speed memory unit located close to the CPU that temporarily stores copies of frequently accessed data and instructions. Its main purpose is to reduce the time it takes for the CPU to retrieve information from the main memory (RAM), which is much slower. When the CPU needs data, it first checks the cache. If the data is found there (called a cache hit), it is accessed quickly 3.3. Possible Bottlenecks in X86Minor CPU: 3.3.1. In-order Execution Minor CPU executes instructions sequentially. If an instruction stalls due to a data dependency or memory latency, all subsequent instructions are delayed, reducing overall efficiency. 3.3.2. Simple Branch Prediction Minor CPU employs a basic branch prediction mechanism. In programs with complex control flows, frequent mispredictions can lead to pipeline flushes and wasted cycles. 3.3.3. Limited Cache Hierarchy Although Minor CPU uses caching, the cache is relatively small and basic. High cache miss rates force the CPU to fetch data from slower main memory, increasing execution time. • BASH COMMAND./build/X86/gem5.opt configs/deprecated/example/se.py -cpu-type=X86Minor CPU -- caches -c tests/test-progs/hello/bin/x86/linux/hello • OUTPUT: Exiting@ 28096500 ticks which means the execution time of X86Minor CPU to run Hello World program is 0.0280965 milliseconds. Minor CPU performs better than Timing Simple CPU, Minor CPU is approximately 16 times faster than Timing Simple CPU when running the same "Hello World" program, primarily because it employs a pipelined architecture that allows World Journal of Advanced Research and Reviews, 2025, 26(02), 190-204 194 multiple instructions to be processed simultaneously in different stages of execution. In contrast, Timing Simple CPU is a simple, non-pipelined processor that executes instructions sequentially—fetching, decoding, and executing one instruction at a time, and only starting the next instruction after the current one has fully completed. This sequential nature leads to inefficiencies, especially when memory access delays occur. On the other hand, Minor CPU can overlap instruction execution, improving throughput and reducing idle CPU cycles. Additionally, Minor CPU includes basic cache support and branch prediction mechanisms, which further help in minimizing delays due to memory access and control flow changes. These architectural improvements result in better performance, even for simple programs like "hello world." 3.4. O3CPU O3CPU, or Out-of-Order CPU, is the most advanced CPU model available in gem5. Unlike in-order CPUs like Timing Simple CPU and Minor CPU, O3CPU is capable of executing instructions out of their original program order to maximize performance. This architecture allows the CPU to bypass stalled instructions (such as those waiting for memory) and continue executing independent instructions, thereby improving instruction-level parallelism. These features collectively reduce idle time, make better use of execution units, and minimize performance bottlenecks caused by data or control dependencies. Due to its complexity, O3CPU is best suited for simulating high-performance processors. 3.4.1. CPU Optimization Techniques in O3CPU Out-of-Order Execution allows the CPU to execute instructions as their operands become available, rather than strictly following the original program order. This helps avoid delays caused by instruction dependencies and keeps execution units busy. Speculative Execution and Branch Prediction allows O3CPU to guess the outcome of branches. It then speculatively executes instructions along the predicted path. If the prediction is correct, this saves time, if not, it discards the results and rolls back. Pipelining is Similar to Minor CPU but deeper and more dynamic, pipelining enables multiple instructions to be in different stages of execution at the same time, improving instruction throughput. Caching in O3CPU utilizes a hierarchical caching system, including L1 and L2 caches, to reduce memory latency. This speeds up data access and helps avoid frequent stalls caused by slow memory. 3.4.2. Possible Bottlenecks in O3CPU Cache Misses, if data is not found in the cache, it must be fetched from slower main memory, causing delays. Mispredicted Branches, Although the prediction is sophisticated, incorrect guesses result in costly rollbacks and wasted cycles. Pipeline Stalls, Due to complex instruction dependencies or resource conflicts, parts of the pipeline can still stall. • BASH COMMAND: ./build/X86/gem5.opt configs/deprecated/example/se.py --cputype=O3CPU --caches -c tests/test-progs/hello/bin/x86/linux/hello • OUTPUT: Exiting@ 16119500 ticks which means the execution time for O3CPU to run Hello World program is 0.0161195 milliseconds. The execution of the “hello world” program across three different CPU models in gem5—Timing Simple CPU, Minor CPU, and O3CPU—demonstrates the significant impact of architectural optimizations on performance. Timing Simple CPU, which lacks pipelining and executes instructions strictly in order, recorded the highest number of ticks at 454,646,000, indicating slowest performance. Minor CPU, with its in-order pipelined design and basic cache and branch prediction support, showed a substantial improvement, completing execution in 28,096,500 ticks. O3CPU, featuring out-of-order execution, register renaming, speculative execution, and advanced branch prediction, delivered the best performance with just 16,119,500 ticks. These results confirm that architectural enhancements like pipelining, caching, and out-oforder execution greatly reduce execution time, even for simple workloads. The execution speed of the "Hello World" program across the three CPU models in Gem5 reveals significant performance differences. In Gem5, simulation time is measured in ticks, where 1 tick equals 1 picosecond (1e-12 seconds). The Timing Simple CPU took approximately 454,646,000 ticks, which converts to 0.000454 seconds. Minor CPU executed the same program in around 28,096,500 ticks (0.000028 seconds), and O3CPU completed it in just 16,119,500 ticks (0.000016 seconds). These results World Journal of Advanced Research and Reviews, 2025, 26(02), 190-204 195 demonstrate that while all models completed the simple task within a fraction of a millisecond, the degree of architectural optimization greatly influenced performance. TimingSimpleCPU, being a simple non-pipelined model, had the slowest execution. Minor CPU, which incorporates pipelining and limited branch prediction, showed a substantial speed improvement. O3CPU, which supports out-of-order execution and more aggressive optimizations, outperformed both, highlighting the impact of advanced CPU features even on lightweight workloads. 3.5. Method 2: Neuromorphic methods 3.5.1. Neuromorphic computing Neuromorphic computing is inspired by the human brain's architecture, aiming to create systems that mimic biological neural networks. Unlike traditional computing, which relies on the Von Neumann architecture, neuromorphic systems use Spiking Neural Networks (SNNs). In SNNs, data is represented by spikes, mimicking the way neurons in the brain communicate through electrical impulses. The key benefits of this approach are energy efficiency and real-time processing. SNNs activate only when necessary, reducing power consumption compared to traditional models that continuously process data. This makes them ideal for resource constrained environments like embedded systems and edge computing. To analyze the impact of neuromorphic computing on performance and efficiency, we employed the Brian2 simulator, a widely used framework for simulating spiking neural networks (SNNs) and neuromorphic systems. For testing, we used the popular MNIST dataset, which contains 70,000 images of handwritten digits (from 0 to 9). Each image is 28x28 pixels. Before feeding them into the network, we normalized the pixel values and converted them into spike trains using a simple rate-based method — where brighter pixels cause neurons to spike more frequently. The network used in this experiment was an SNN with three layers: • Input Layer: 784 neurons (one for each pixel). • Two Hidden Layers: These helped the network learn patterns in the digits. • Output Layer: 10 neurons, one for each digit from 0 to 9. The neuron that spiked the most was considered the network’s guess. The network was trained using a brain-inspired rule called STDP (Spike-Timing Dependent Plasticity), which adjusts the strength of connections based on the timing of neuron spikes. We recorded the following values to measure performance: Accuracy, latency and memory usage. We then compared these results with a traditional neural network (MLP) that was trained on the same dataset using PyTorch on a Von Neumann system. This helped us see how the brain-like SNN model performs differently from standard models in terms of speed, memory, and accuracy. 3.6. Experiments 3.6.1. MNIST dataset For testing, a standard handwritten digit recognition task was used — the MNIST dataset — which is widely accepted for evaluating classification models. It consists of 70,000 grayscale images (28x28 pixels) of digits from 0 to 9. This dataset was chosen because it is simple, yet effective for comparing the learning ability, speed, and efficiency of different neural network approaches. Both the traditional MLP and the brain-inspired SNN were trained and evaluated on this dataset to ensure a consistent basis for comparison. 3.6.2. Classifying digits from the MNIST Dataset (Von Neumann) This code trains and tests an AI model (a simple multi-layer perceptron neural network) to recognize handwritten digits. It measures how fast, how accurate, and how much memory it uses and as well as how long it takes to make predictions. It basically takes a 28x28 pixel image of a digit (like ‘4’) and flattens it into a 1D vector. It passes it through two hidden layers with 512 and 256 neurons. World Journal of Advanced Research and Reviews, 2025, 26(02), 190-204 196 3.6.3. CODE: World Journal of Advanced Research and Reviews, 2025, 26(02), 190-204 197 World Journal of Advanced Research and Reviews, 2025, 26(02), 190-204 198 3.6.4. OUTPUT: