Task-Aware Differential Privacy and Modular Structural Perturbation for Secure Fine-Tuning of Large Language Models

Innovative Applications of Artificial Intelligence and Computer Science | Vo. 4, No. 7, 2024 ISSN: 2998-8780 https://pspress.org/index.php/ Pinnacle Science Press Task-Aware Differential Privacy and Modular Structural Perturbation for Secure Fine-Tuning of Large Language Models Yilin Li Carnegie Mellon University, Pittsburgh, USA [email protected] Abstract: This paper addresses the risk of privacy leakage during the fine-tuning of large language models in sensitive scenarios by proposing a differential privacy mechanism that integrates task-aware perturbation and modular structural injection. The mechanism consists of two components: Task-aware Differentially Private Fine-tuning (TDPF) and Modular Privacy-aware Injection (MPI). TDPF dynamically adjusts the intensity of gradient perturbation based on semantic sensitivity scoring, guiding the model to adaptively optimize its update path under differential privacy constraints. MPI injects structured noise into key substructures of the model and uses modulation factors to precisely control the perturbation intensity across different modules, thereby enhancing semantic consistency while maintaining structural stability. A series of systematic experiments is conducted to evaluate the proposed method across multiple dimensions, including privacy budget sensitivity, injection frequency, and modulation strength. The results show that the method significantly improves multi-task adaptability and semantic representation integrity while maintaining privacy budget efficiency. It effectively alleviates the performance-structure conflict present in traditional differential privacy strategies, demonstrating advantages in structural friendliness, controllable performance, and robust privacy protection. Keywords: Differential perturbation strategy, structural injection mechanism, semantic preservation, and interference control capability 1. Introduction With the continuous advancement of large-scale pre-trained language models, their application value in highsensitivity domains such as healthcare, finance, and public services has become increasingly prominent. As pre-trained models transition from static corpora to task-specific fine-tuning, a key challenge arises: how to ensure efficient task adaptation while protecting user data privacy. In multi-turn continual learning and multisource heterogeneous data settings, privacy risks introduced during fine-tuning may compromise model security and trustworthiness. These risks also directly affect system compliance and ethical responsibilities[1]. Therefore, building secure fine-tuning mechanisms that balance representation capacity and privacy control is essential for deploying large language models in real-world, high-risk scenarios[2]. However, most existing fine-tuning strategies focus on parameter efficiency, computational cost, or transfer generalization. They often overlook the semantic exposure and re-identification risks of original samples during fine-tuning. Some studies have adopted differential privacy mechanisms, but most apply static perturbation schemes[3]. These fail to dynamically adjust noise based on task semantics, leading to performance degradation and inefficient privacy budget usage. Moreover, current differential privacy methods lack structural sensitivity modeling for complex and highly semantic model architectures. This limits their ability to control semantic flow at a fine-grained level and reduces the practicality and scalability of differential privacy in large model fine-tuning[4,5]. To address these issues, this study proposes a differential privacy fine-tuning mechanism that integrates taskaware perturbation control and modular structural injection[6]. At the global level, the method introduces a semantic sensitivity-guided perturbation strategy along the gradient path. At the structural level, it applies modular perturbation adjustment and injection control within the model. This joint approach enables coordinated optimization of privacy protection and representation preservation during fine-tuning. The design does not rely on additional label supervision and enhances representational stability and structural robustness in multi-task continual learning. The method is scalable and compatible with various Transformer-based models. It can be flexibly deployed in task systems of different scales and complexities[7]. The main contributions of this work include two aspects. First, a Task-aware Differentially Private Finetuning (TDPF) mechanism is proposed. It dynamically adjusts the noise intensity for each sample based on semantic gradient sensitivity scoring, enabling differentiated optimization paths under privacy constraints. Second, a Modular Privacy-aware Injection (MPI) structure is designed. It constructs structure-aware perturbation pathways within the model to ensure local representational stability and semantic coherence during fine-tuning. These two mechanisms form a dual-layer privacy control framework, from global optimization to local representation. The proposed approach provides a new technical pathway and theoretical support for controllable fine-tuning of large language models in privacy-sensitive tasks[8,9]. 2. Related work 2.1 Privacy-Preserving Fine-tuning of Large Language Models As large language models are increasingly deployed in real-world applications, privacy concerns during the fine-tuning phase have emerged as a key research focus. In domains such as finance, healthcare, and law, training data used in fine-tuning often contains private user information[10]. The model may unintentionally memorize sensitive content during learning, which can later be exposed through fragments or semantic cues in downstream inference. This issue is more severe in large models due to their vast parameter space and powerful representation capacity, making it easier for them to reconstruct input data with high fidelity. Traditional fine-tuning methods generally lack structural constraints on data privacy. They cannot prevent the model from retaining unnecessary information while adapting to tasks. As a result, the model may achieve strong generalization but still carry significant risks of privacy leakage[11,12]. To address these risks, recent studies have begun exploring the integration of privacy-preserving techniques into the fine-tuning process. One common approach is to constrain the model's parameter updates to avoid direct exposure of sensitive gradient information[13]. These methods often apply gradient perturbation, weight clipping, or low-rank approximation to reduce reliance on individual samples while maintaining task performance. Some works have also proposed replacing conventional parameter tuning with lightweight adaptation methods. This includes inserting micro-modules or freezing parts of the model to reduce invasiveness[14]. While these strategies enhance privacy control to some extent, they usually lack formal guarantees and remain vulnerable to active or inference-based attacks. Beyond parameter constraints and structural adaptations, some approaches aim to improve privacy protection by redesigning the training process. For example, in multi-task learning or domain adaptation tasks, sample reweighting or task selection mechanisms are used to reduce overfitting on outliers. This can indirectly lower the model's sensitivity to specific samples[15,16]. Other studies introduce techniques such as model distillation and pseudo-label learning. These leverage auxiliary models or unlabeled data to optimize training without direct access to sensitive inputs. Although these methods provide new directions for privacy-aware training, they often lack rigorous privacy metrics and fail to establish verifiable safety boundaries in practice[17]. To overcome these limitations, differential privacy has emerged as a promising formal framework for secure fine-tuning of large language models. Some studies have embedded differential privacy into training and fine-tuning workflows by injecting noise into gradient updates. This helps prevent the model from memorizing individual samples from a statistical perspective. However, practical implementations still face several challenges. These include performance degradation, slower convergence, and increased tuning complexity. In multi-turn or dynamic task fine-tuning scenarios, it is particularly difficult to manage privacy budget consumption and control gradient drift. Balancing task utility and strong privacy protection has therefore become a central challenge in designing privacy-preserving fine-tuning mechanisms[18]. 2.2 Efficient Differential Privacy Mechanisms in Deep Learning Differential privacy is one of the most theoretically grounded privacy protection mechanisms in deep learning. Its core idea is to introduce random noise during training to limit the model's dependence on any single training sample. This prevents attackers from inferring whether a specific sample was used by observing model outputs. The mechanism provides mathematically verifiable privacy guarantees and has been widely applied across tasks. It shows strong adaptability in high-sensitivity environments. However, applying differential privacy directly to deep neural networks remains highly challenging. The complexity and high dimensionality of deep models make it difficult to balance privacy budget control and performance degradation. In addition, the non-convex optimization and rapid parameter shifts during training may lead to instability or even training failure when noise is added. Therefore, efficient differential privacy mechanisms tailored to deep structures are urgently needed to maintain privacy control and model performance simultaneously[19,20]. To address these challenges, various optimization strategies based on differential privacy have been proposed. Among them, the most representative is the differentially private stochastic gradient descent (DP-SGD) algorithm. This algorithm clips the gradient of each mini-batch and adds Gaussian noise to reduce the influence of individual samples on the update direction. However, DP-SGD has high computational costs in large-scale models and multi-round training[21,22]. The frequent gradient updates significantly increase the demand for memory and computation. Privacy accounting mechanisms are also required to monitor and manage the overall privacy budget, further increasing implementation complexity. In natural language processing tasks, applying DP is even more difficult due to the discrete nature of text and the nonlinear semantic relationships. Noise injection may conflict with semantic consistency and functional expression. Designing differential privacy strategies suitable for structured language modeling remains a key research challenge in private deep learning. In practice, some studies aim to improve the efficiency and usability of differential privacy by introducing ideas such as structural compression and local perturbation. These methods inject noise only into sensitive parameters or critical modules, reducing the impact on the entire network. They often combine parameter sharing, low-rank projection, or modular training to reduce noise propagation and computational redundancy. At the same time, they preserve performance on the main task. Other works design DP-friendly training architectures, such as hierarchical perturbation, task-aware clipping, and learnable noise injection modules. These methods enable fine-grained privacy control and adaptive tuning[23]. They improve noise efficiency in theory and enhance the privacy-performance trade-off in large-scale training in practice. This helps bridge the gap between differential privacy as a theoretical concept and its real-world engineering applications. At the same time, increasing demand for privacy protection is pushing differential privacy mechanisms toward task-adaptive and context-aware designs. Current studies are introducing scene-aware privacy modeling. This includes task relevance analysis and data sensitivity metrics to guide the intensity and strategy of noise injection. The goal is to enable customized and differentiated privacy protection. Emerging paradigms such as multimodal learning, federated optimization, and generative modeling further expand the scope of differential privacy. These approaches allow deployment beyond single-task and single-model settings. They enable flexible privacy protection across heterogeneous data sources and complex communication structures. These technical developments are moving differential privacy from algorithm- level improvement toward system-level and task-level collaborative design. This provides a forward-looking direction for privacy protection in deep learning systems[24]. 3. Method This paper proposes a secure fine-tuning framework for large language models by integrating differential privacy mechanisms, aiming to address the conflict between privacy protection and performance degradation during model adaptation. The approach introduces two core innovations. First, a Task-aware Differentially Private Fine-tuning (TDPF) mechanism dynamically adjusts the intensity and frequency of gradient perturbations based on task-specific privacy risk assessment. It incorporates semantic sensitivity, module dependency, and contextual importance of training samples to map privacy budget to task contribution. This improves the controllability and adaptability of privacy protection. Second, a Modular Privacy-aware Injection (MPI) structure injects differential noise into specific sub-networks such as attention layers, embedding layers, or adapter modules. It avoids unnecessary disturbance to the global parameter space and enhances the transparency and stability of privacy injection. The modular design supports flexible deployment across models of various sizes. Through the synergy of TDPF and MPI, the proposed framework reduces reliance on sensitive data while improving security and privacy robustness in real-world applications. The overall model architecture is shown in Figure 1. Figure 1. Overall model architecture 3.1 Task-aware Differentially Private Fine-tuning This study proposes a Task-aware Differentially Private Fine-tuning (TDPF) mechanism designed to enhance the adaptability and representational capacity of large language models while ensuring the privacy of training data. The mechanism introduces task-related sensitivity scheduling and perturbation control strategies during training, making differential privacy an active component of the model optimization process rather than a passive protection tool. Unlike traditional static noise injection methods, TDPF dynamically adjusts perturbation intensity based on the gradient contribution and semantic importance of each input sample. This enables fine-grained privacy control and task-aware optimization path selection. The mechanism avoids excessive perturbation of irrelevant features and preserves the modeling of critical semantic paths without significantly increasing computational overhead. TDPF also exhibits strong compatibility with modular injection structures and can be flexibly applied to various large model finetuning scenarios. The overall architecture of the mechanism is illustrated in Figure 2. Figure 2. TDPF Model Architecture First, the standard differential privacy mechanism can be expressed as the following perturbative gradient update form: ),0(),( ~22 ICNCgClipg tt   Where t g represents the original gradient of the step t , C is the gradient clipping threshold, N is the zero-mean Gaussian noise distribution, and  controls the privacy strength. In the task-aware scenario, we introduce a sample-level weight factor i  to weight the gradient according to the impact of the sample on task performance. The expanded weighted perturbation form is: ),0(),( ~22 1 ICNCgClipg N i iit     To measure the task contribution of each sample, this paper designs a sensitivity scoring function based on semantic gradient alignment:       |||||||| , taski taski iLg Lg Where ， represents the vector dot product, Ltask is the current task objective function, and  is the smoothing factor. This scoring function can effectively identify key samples that contribute significantly to model optimization in the current training round and provide stronger perturbation protection during privacy injection. In the overall optimization process, the objective function of TDPF consists of task loss and privacy regularization term, which are defined as follows: DPtaskTDPF RLL   Where  is the balance coefficient, and the privacy regularization term DP R represents the penalty term of the overall perturbation scale in the parameter space, which is specifically expressed as:   2 2 || ~ || ttDP ggER  This loss function plays a key structural role in TDPF. It guides the distribution of privacy perturbation through semantic alignment strategies, preventing damage to core features. It also uses a regularization term to dynamically constrain the magnitude of noise, ensuring training stability and controllability. The mechanism works closely with Modular Privacy-aware Injection (MPI). While TDPF constructs the global perturbation strategy and gradient optimization path, MPI applies structured noise injection within specific submodules. Together, they form a unified fine-tuning framework that balances task adaptability and differential privacy protection. 3.2 Modular Privacy-aware Injection This study proposes a Modular Privacy-aware Injection (MPI) mechanism to address the structural interference caused by uniform noise injection in differential privacy training. The mechanism introduces localized perturbations into key functional modules of large language models, such as attention mechanisms, feed-forward networks, and embedding substructures. It builds a more perceptive privacy control path from a structural perspective. Compared with traditional uniform perturbation strategies, MPI allocates perturbations differently based on each module's structural characteristics and semantic sensitivity. This enhances privacy protection while preserving the stability and interpretability of internal representations. A modulation factor is used to dynamically adjust the perturbation intensity for each submodule, aligning the perturbation behavior with the needs of semantic modeling and avoiding excessive disruption to critical information flow. MPI is highly modular and pluggable, allowing flexible deployment across various finetuning paradigms. It complements task-aware mechanisms and forms a coordinated structure. The overall architecture of the mechanism is illustrated in Figure 3. Figure 3. MPI Model Architecture Specifically, consider a modular neural architecture where the submodule output of a layer l is represented as: );( llll xFh   To achieve module-level perturbation injection, we embed noise into the output of each submodule. The perturbation form is defined as: ),0( ~2INhh lll   Where l  represents the local privacy strength control parameter corresponding to the module l , which is dynamically adjusted by the task-related sensitivity. To avoid excessive amplification of perturbations during forward propagation, MPI introduces a structural weighting mechanism and a structural modulation factor l  for each module, resulting in the modulated expression: ttt hh ~ ˆ  The structural modulation factor   1,0 t  is set according to the importance of the module to the target task. Important modules retain a larger proportion of the original expression, and non-core modules increase the disturbance injection. To further constrain the distribution stability of module perturbations in the semantic space, a module regularization term is introduced:    Sl ttMPI hhR 2 2 || ~ || Where S is the set of modules that inject disturbances. This regularization term can effectively penalize perturbations that cause excessive deviations in module expression, ensuring the controllability and consistency of model expression. Finally, the objective function of MPI integrates the module injection expression, structural regularization, and original task loss, and the complete optimization objective is defined as: MPItaskMPI RLL    is a structural regularization factor that balances task performance with module injection stability. This loss function plays a key structural regulation role in MPI, not only explicitly controlling the perturbation injection amplitude of each module but also providing guidance for optimizing cross-module expression coordination. This mechanism synergizes with the aforementioned TDPF: TDPF dynamically adjusts the privacy budget and perturbation strategy within the global optimization path, while MPI implements finegrained differential privacy embedding within the structural dimension, achieving a complete, integrated privacy control solution from the global path down to the module unit. 4. Experimental Results 4.1 Dataset This study uses the publicly available WinoBias Coreference Dataset, which can be accessed on the Kaggle platform. The dataset is specifically designed for coreference resolution tasks and includes a large number of annotated pairs of gender-neutral pronouns and antecedents. It is suitable for evaluating a model's ability to understand semantic referential relationships. In cross-task fine-tuning or incremental learning scenarios, WinoBias provides clear binary labels indicating whether a coreference exists. Its well-defined semantic structure and balanced class distribution make it ideal for observing whether a model can retain robust recognition of coreference patterns from previous tasks when new tasks are introduced. Each sample consists of a sentence with an ambiguous pronoun and candidate antecedents, with a label indicating whether the model should resolve the pronoun correctly. The WinoBias dataset also emphasizes gender bias in real-world language use. It highlights how models may exhibit bias when processing socially grounded semantic expressions. Since fine-tuning may cause improper generalization of style, register, or gender preference, this dataset helps examine whether the model maintains its ability to recognize pronoun resolution patterns from earlier semantic structures after being trained on new tasks. The dataset includes diverse examples, covering various sentence types, pronoun forms, and referential ambiguities. These complex contexts effectively support the evaluation of task-aware mechanisms such as TDPF, particularly in adjusting sensitivity to semantically important samples. This allows validation of task-specific scoring and perturbation strategies. In addition, WinoBias includes hierarchical and subset partition structures, such as groupings based on occupation categories or semantic templates. This structure is naturally compatible with staged training and evaluation in task-incremental learning. The proposed Modular Privacy-aware Injection (MPI) can leverage these subsets to perform differential privacy noise injection and modulation control within different submodules. This ensures that handling a specific semantic subtask does not degrade the model's ability to process others. By using phased training and localized injection, the model achieves privacy protection for new tasks while preserving performance on existing coreference patterns. 4.2 Experimental setup All experiments in this study were conducted in a standardized deep learning environment. The setup was based on Ubuntu and ran on a server equipped with two NVIDIA A100 40GB GPUs, 1TB of memory, and an Intel Xeon Gold 6348 CPU. To ensure training stability and reproducibility, all experiments were implemented using the PyTorch framework. CUDA and cuDNN versions were kept compatible. Mixed precision training (AMP) was used to improve resource efficiency. Multi-GPU distributed synchronization was enabled to accelerate fine-tuning convergence. For the base model, this study adopts the open-source ChatGLM2-6B, which has strong capabilities in Chinese understanding and generation. It supports reasoning, instruction following, and context modeling. The model is compatible with various parameter-efficient fine-tuning methods. INT4 quantization was used to reduce memory usage during model loading. Full parameter fine-tuning was applied to preserve model expressiveness and integrate with the task-aware privacy mechanism. All tokenizer settings and model initialization parameters remained consistent to avoid preprocessing bias. During training, the initial learning rate was set to 2e-5. The AdamW optimizer was used with a linear learning rate decay schedule. The gradient clipping threshold was 1.0. The batch size was set to 16, with a maximum of 10 epochs. After each epoch, validation was performed, and the best model state was saved for final inference. Noise scale parameters for the privacy mechanism were selected from {0.5, 1.0, 1.5} and consistently applied to both task-aware gradient paths and modular injection structures. All random seeds were fixed at 42 to ensure reproducibility. 4.3 Experimental Results 1) Comparative experimental results This paper first conducts a comparative experiment, and the experimental results are shown in Table 1. Table 1: Comparative experimental results Method PBE↑ RA↑(%) TIR↓(%) SPS↑ FLOPs↓ TDPF + MPI (Ours) 0.92 85.3 3.40 0.81 68.2 DP-Finetune [25] 0.77 79.8 6.70 0.68 84.5 LoRA [26] 0.54 83.1 9.10 0.63 45.3 FedNLP [27] 0.69 76.4 5.50 0.71 93.8 D3PM[28] 0.74 81.7 4.90 0.72 76.0 As shown in Table 1, overall, the proposed TDPF + MPI method demonstrates clear advantages across all key evaluation metrics, reflecting its effectiveness in balancing privacy protection and structural stability. In terms of Privacy Budget Efficiency (PBE), the method achieves a score of 0.92, significantly higher than all baseline methods. This indicates that the task-aware mechanism can accurately control the location and intensity of perturbations while efficiently utilizing the privacy budget. The high efficiency of privacy scheduling also offers broader flexibility for modular injection strategies. These results confirm the feasibility of our proposed two-level mechanism in layered semantic protection. For the Retained Accuracy (RA) metric, TDPF + MPI maintains a performance level of 85.3 percent, outperforming LoRA and D3PM. This suggests that the coupling design between differential perturbation and semantic modeling enhances task adaptability. In contrast, traditional methods such as DP-Finetune apply static noise and often impair semantic representation. Our method leverages gradient alignment and sensitivity-based weighting to target high-risk areas only, preserving representational integrity along core semantic paths and maintaining generalization performance across tasks. In terms of Task Interference Rate (TIR) and Structural Perturbation Stability (SPS), our method achieves 3.4 percent and 0.81, respectively, outperforming competing approaches. These results show that the modular injection mechanism plays a key role in limiting the spread of perturbation and preserving semantic consistency within substructures. Compared with methods like FedNLP and LoRA, which lack structural alignment in local parameter updates, our MPI module modulates channel activations and perturbation directions to prevent semantic drift and structural degradation. This ensures the stable coexistence of tasks in continual learning settings. Regarding computational cost, our method controls FLOPs at 68.2 ×10 ⁹while maintaining strong performance. This is lower than many privacy-enhanced approaches, such as DP-Finetune and D3PM. The result is attributed to the suppression of redundant gradient perturbations by the task-aware strategy and the improved sparsity from the modular design. This demonstrates a more efficient use of structure in large-scale fine-tuning. Overall, the proposed approach achieves a balanced optimization across privacy protection, structural control, and computational overhead, providing strong technical support for controllable finetuning of large language models in privacy-sensitive environments. 2) Ablation Experiment Results To evaluate the actual contribution of each module to overall performance, ablation studies are widely used to assess the impact of different design components. By incrementally adding or removing specific modules, the role of each mechanism in model performance, stability, and resource consumption can be identified. This process helps reveal the effectiveness and necessity of key structural elements. Table 2 presents the experimental results under different module combinations. The first row shows the baseline model, followed by the sequential introduction of the two core modules, and finally the complete method. The changes in each metric demonstrate the synergistic effects and performance gains of the proposed components.