Adversarial Feature Disentanglement for Bias-Invariant Prediction of Zigong Lantern User Preferences

Founded by Simpy Hope Limited. Natura Humanitas Journal Homepage: https://www.naturahumanitas.com/ ISSN 3106-731X Original Research Adversarial Feature Disentanglement for Bias-Invariant Prediction of Zigong Lantern User Preferences Zhang Haoyang a*, Zhang Luokuan b aSchool of Fine Arts, Sichuan University of Science & Engineering, Zigong, China bFaculty of Business and Economic, University of Malaya, Kuala Lumpur, Malaysia *Corresponding author: zhanghaoy[email protected] Article History Received: 12 October 2025 Revised: 30 October 2025 Accepted: 28 November 2025 Published: 5 December 2025 Keywords Zigong Lanterns; User Preference Prediction; Bias Mitigation; Data Augmentation; Generative Adversarial Network Abstract We propose a novel Adversarial Feature Disentanglement Framework to address bias-invariant prediction of user preferences for Zigong lanterns, a cultural artifact where traditional methods often fail to disentangle confounding biases from genuine preference signals. The framework integrates a Transformer-based feature extractor with adversarial learning to generate bias-invariant latent representations, while a bias-conflicting augmentation module synthesizes adversarial examples to mitigate data imbalance. The feature extractor employs multi-head selfattention to capture complex relationships in raw input features, and an adversarial discriminator with gradient reversal layers enforces invariance to bias attributes such as demographic factors. Furthermore, the augmentation module dynamically perturbs underrepresented preference patterns, improving robustness against spurious correlations. The proposed method jointly optimizes preference prediction accuracy and bias invariance through a dual adversarial objective, enabling base models to operate on disentangled features for more generalizable predictions. Experiments on real-world Zigong lantern preference datasets demonstrate significant improvements in fairness and accuracy compared to conventional approaches. Our work contributes a systematic solution to bias mitigation in cultural preference modeling, with broader applicability to other domains where feature bias and data imbalance pose critical challenges. The integration of adversarial learning with dynamic augmentation offers a principled approach to disentangling predictive features from confounding factors, advancing the state-of-the-art in unbiased recommendation systems. https://doi.org/10.5281/zenodo.17926650 1 Introduction Cultural heritage artifacts such as Zigong lanterns represent complex intersections of aesthetic, historical, and personal preference factors. Traditional recommendation systems for such artifacts often suffer from inherent biases in user preference data, where demographic attributes, temporal trends, or regional influences may disproportionately affect predictive models [1]. These biases manifest as spurious correlations that reduce model generalizability across diverse -1- Natura Humanitas (2025), 1(1): 1-10 Zhang Haoyang & Zhang Luokuan user groups. Existing approaches to preference modeling either ignore such biases or apply post-hoc correction techniques, which frequently degrade predictive performance while attempting to achieve fairness [2]. The challenge of disentangling genuine preference signals from confounding biases shares conceptual parallels with domain adaptation problems in machine learning [3]. Recent advances in adversarial learning have demonstrated promising results for bias mitigation by forcing feature representations to become invariant to protected attributes [4]. However, these methods typically assume static bias definitions and balanced training data—conditions rarely met in real-world cultural preference datasets. Moreover, conventional adversarial approaches often struggle with high-dimensional feature spaces common in user behavior modeling, where nonlinear interactions between preferences and biases complicate disentanglement. We address these limitations through a bias-aware feature engineering framework that integrates three key innovations: (1) a dual-network architecture with gradient reversal for simultaneous preference prediction and bias suppression, (2) dynamic augmentation of bias-conflicting samples inspired by adversarial example generation [5], and (3) attention mechanisms to isolate bias-prone feature interactions. Unlike prior work in fair representation learning [1], our method explicitly models the relationship between bias attributes and preference features through adversarial training, enabling the discovery of latent bias dimensions beyond predefined protected attributes. This proves particularly valuable for Zigong lantern preferences, where biases may emerge from subtle interactions between lantern craftsmanship styles and user cultural backgrounds. The proposed framework contributes to both machine learning and cultural heritage informatics. From a methodological perspective, we advance adversarial debiasing by introducing adaptive feature masking—a technique that selectively suppresses bias-related activations in the latent space based on their contribution to prediction errors across demographic subgroups. This differs fundamentally from invariant risk minimization approaches [3] by conditioning the debiasing strength on local feature importance rather than applying global invariance constraints. For practical applications, our system enables museums and cultural platforms to generate personalized lantern recommendations that remain robust against shifts in user demographics or seasonal preference variations. Empirical validation on real-world datasets shows that our framework reduces demographic disparity in recommendation accuracy by 38% compared to baseline models while maintaining 92% of their overall predictive performance. These improvements stem from the framework’s ability to identify and disentangle bias patterns that correlate weakly with explicit user attributes but strongly influence conventional model outputs. For instance, we found that certain lantern color combinations were systematically under-recommended to older users despite individual preferences, a bias eliminated through our adversarial feature learning approach. The remainder of this paper is organized as follows: Section 2 reviews related work in adversarial debiasing and cultural recommendation systems. Section 3 formalizes the bias disentanglement problem and introduces necessary preliminaries. Section 4 details our framework’s architecture and training dynamics. Sections 5 and 6 present experimental setup and results, respectively. Section 7 discusses implications and future directions, followed by conclusions in Section 8. 2 Related Work The challenge of mitigating biases in predictive models has garnered significant attention across multiple domains, with adversarial learning emerging as a particularly promising approach. Prior work in this space can be broadly categorized into three research directions: adversarial debiasing methods, cultural preference modeling, and feature disentanglement techniques. 2.1 Adversarial Debiasing Methods Recent advances in adversarial learning have demonstrated its effectiveness for bias mitigation through gradient reversal layers and competing objectives. The BiasAdv method [6] pioneered the use of adversarially attacked images as additional training data, showing that synthetic bias-conflicting samples could improve model fairness. This approach shares conceptual similarities with our augmentation module, though we extend it to handle structured preference data rather than image inputs. Federated learning environments have also adopted adversarial principles, as seen in Federated Adversarial DEbiasing [7], which maintains fairness across distributed data sources. While these methods focus on global bias mitigation, our framework introduces local adaptive masking to address bias patterns that vary across feature subspaces. 2.2 Cultural Preference Modeling The preservation and analysis of cultural artifacts through machine learning presents unique challenges in bias identification and mitigation. Studies on batik motif synthesis [8] have shown how generative adversarial networks can capture cultural aesthetics while avoiding stereotypical representations. In the Chinese cultural context, adversarial dialogue systems [9] have demonstrated the importance of culturally-aware preference modeling. These works highlight -2- Natura Humanitas (2025), 1(1): 1-10 Zhang Haoyang & Zhang Luokuan the need for domain-specific bias handling that our Zigong lantern framework addresses through its attention-based feature extractor. Unlike generic debiasing approaches, our method explicitly models the relationship between lantern craftsmanship features and demographic factors that influence preference expressions. 2.3 Adversarial Debiasing Methods Feature disentanglement has proven valuable for separating explanatory factors in high-dimensional data. The concept activation vector methodology [10] provides a mechanism for isolating bias-related features through partial learning. Our framework builds upon this idea but introduces dynamic augmentation to handle cases where bias dimensions cannot be predefined. For sequential data, transformer architectures have shown superior disentanglement capabilities compared to convolutional networks [11], particularly when combined with adversarial objectives. We adapt these insights to the structured preference prediction domain through our multi-head self-attention mechanism. The proposed framework advances beyond existing methods by combining three key innovations: (1) dynamic generation of bias-conflicting samples specific to cultural preference patterns, (2) local adaptive masking of bias-related features based on their contribution to subgroup prediction errors, and (3) integrated training of feature extraction and debiasing components through a dual adversarial objective. This holistic approach addresses limitations of prior work that either treated biases as static or required predefined protected attributes. Our experiments demonstrate that this combination yields superior performance in preserving genuine preference signals while suppressing spurious correlations, particularly for complex cultural artifacts like Zigong lanterns where biases manifest through subtle feature interactions. 3 Background and Preliminaries Understanding the mechanisms of bias propagation in machine learning models and the fundamentals of deep learning architectures forms the foundation for our proposed framework. This section systematically outlines these core concepts, establishing the theoretical basis for subsequent methodological developments. 3.1 Bias and Fairness in Machine Learning Machine learning models often inherit and amplify biases present in training data, particularly when predicting human preferences or behaviors. The mathematical formulation of bias in predictive modeling can be expressed as: Bias =E[ˆ Y] −E[Y](1) where ˆ Yrepresents model predictions and Ydenotes ground truth values. This difference in expectations quantifies the systematic deviation introduced by the learning process [12]. In cultural preference modeling, such biases frequently manifest as disparities in recommendation accuracy across demographic subgroups. Fairness metrics provide quantitative measures to assess and mitigate these biases. A commonly used group fairness criterion evaluates the difference in model outcomes between protected attribute groups: Fairness =1−E[ˆ Y|B=1] −E[ˆ Y|B=0](2) where Brepresents a binary protected attribute [13]. This formulation becomes particularly relevant for Zigong lantern recommendations, where we aim to minimize prediction disparities across age, gender, and regional user groups without compromising overall accuracy. The challenge intensifies when biases operate through latent feature interactions rather than explicit protected attributes. Traditional fairness-aware learning methods often assume direct access to bias-inducing variables [14], whereas real-world cultural preference data frequently contains implicit bias channels that require more sophisticated detection and mitigation approaches. 3.2 Deep Learning Fundamentals Modern deep learning architectures provide powerful tools for modeling complex relationships in high-dimensional data. Neural networks learn hierarchical representations through successive nonlinear transformations: h𝑙=𝜎(W𝑙h𝑙−1+b𝑙)(3) where h𝑙denotes the activation at layer 𝑙,W𝑙and b𝑙are learnable parameters, and 𝜎represents a nonlinear activation function [15]. This compositional structure enables the automatic discovery of relevant features from raw input data. -3- Natura Humanitas (2025), 1(1): 1-10 Zhang Haoyang & Zhang Luokuan Adversarial learning extends this paradigm by introducing competing objectives that shape the learned representations. The minimax formulation: min 𝜃max 𝜙L𝑎𝑑𝑣 (𝜃, 𝜙)(4) creates an adversarial dynamic between the main prediction task (parameterized by 𝜃) and an auxiliary discriminator (parameterized by 𝜙) [16]. In our context, this framework forces the model to develop bias-invariant features by making them indistinguishable across protected attribute groups while maintaining predictive accuracy. Transformer architectures have recently demonstrated superior performance in modeling complex relationships through self-attention mechanisms. The scaled dot-product attention computes: Attention(Q,K,V)=softmax QK𝑇 √𝑑𝑘V(5) where Q,K, and Vrepresent queries, keys, and values respectively, and 𝑑𝑘denotes the dimension of keys [17]. This mechanism proves particularly valuable for cultural preference modeling, as it can automatically identify and weight relevant feature interactions while potentially isolating bias-prone relationships. 4 Bias-Aware Feature Engineering Framework The proposed framework addresses bias mitigation through an integrated approach combining adversarial learning, dynamic sample augmentation, and attention-based feature extraction. The architecture systematically disentangles predictive features from confounding biases while preserving genuine preference signals in Zigong lantern recommendation tasks. 4.1 Framework Architecture Figure 1: Detailed Architecture of the Bias-Aware Feature Engineering Framework -4- Natura Humanitas (2025), 1(1): 1-10 Zhang Haoyang & Zhang Luokuan The framework consists of three core components: a Transformer-based feature extractor, an adversarial discriminator with gradient reversal, and a bias-conflicting sample augmentation module. The feature extractor 𝑓𝜃processes raw input features Xto generate latent representations Z: Z=𝑓𝜃(X)(6) where 𝜃denotes the learnable parameters of the Transformer encoder. The adversarial discriminator 𝑔𝜙then attempts to predict bias attributes Bfrom these latent features: ˆ B=𝑔𝜙(Z)(7) The gradient reversal layer (GRL) between 𝑓𝜃and 𝑔𝜙enforces feature invariance to bias attributes by reversing gradient directions during backpropagation. This creates the adversarial dynamic that drives the feature extractor to suppress bias-related patterns while maintaining predictive utility. 4.2 Component Details and Configuration The Transformer encoder employs multi-head self-attention to model complex feature interactions while allowing selective suppression of bias-correlated attention heads. Each attention head computes: head𝑖=Attention(XW𝑄 𝑖,XW𝐾 𝑖,XW𝑉 𝑖)(8) where W𝑄 𝑖,W𝐾 𝑖,and W𝑉 𝑖are projection matrices for the 𝑖-th head. The adversarial loss regularizes these attention patterns to attenuate bias-related feature interactions. The bias-conflicting augmentation module synthesizes adversarial examples ¯ Xby perturbing original samples to maximize prediction divergence from bias-conditional expectations: ¯ X=X+𝜖·sign(∇XLconflict)(9) where 𝜖controls perturbation magnitude and Lconflict measures deviation from expected bias-conditional predictions. These augmented samples explicitly represent underrepresented preference patterns that contradict typical bias correlations. 4.3 Training and Optimization Strategy The framework jointly optimizes three objectives: prediction accuracy, bias invariance, and augmentation consistency. The complete loss function combines: L=Lpred +𝜆1Ladv +𝜆2Lconflict (10) where Lpred measures prediction error for the main task, Ladv quantifies the discriminator’s ability to predict bias attributes, and Lconflict ensures augmented samples maintain label consistency. The hyperparameters 𝜆1and 𝜆2control the trade-off between these competing objectives. The adversarial component follows a minimax optimization: min 𝜃max 𝜙Lpred (ℎ𝜓(𝑓𝜃(X)),Y) −𝜆1Ladv (𝑔𝜙(𝑓𝜃(X)),B)(11) where ℎ𝜓denotes the prediction head for the main task. This formulation forces the feature extractor to develop representations that simultaneously maximize prediction accuracy and minimize bias predictability. 4.4 Application to Cultural Preference Analysis For Zigong lantern preferences, the framework specifically addresses biases arising from regional aesthetics and demographic factors. The Transformer encoder learns to isolate craftsmanship-related features (e.g., material, color combinations) from demographic-correlated patterns. The attention mechanism automatically identifies and weights relevant visual and cultural attributes while suppressing spurious correlations. The adversarial training adapts to implicit bias dimensions that may not correspond to predefined protected attributes but still influence recommendation patterns. This proves particularly valuable for cultural artifacts where biases often emerge through complex interactions between multiple subtle features rather than single explicit variables. The augmentation module generates samples that represent rare but valid preference combinations, such as younger users preferring traditional designs or regional variations contradicting demographic expectations. These synthetic examples prevent the model from overfitting to majority preference patterns while maintaining the authentic characteristics of Zigong lantern aesthetics. -5- Natura Humanitas (2025), 1(1): 1-10 Zhang Haoyang & Zhang Luokuan 5 Experimental Setup To validate the effectiveness of our proposed framework, we designed comprehensive experiments comparing its performance against conventional approaches on real-world Zigong lantern preference datasets. This section details the experimental configuration, including dataset characteristics, baseline methods, evaluation metrics, and implementation specifics. 5.1 Dataset Description We collected and analyzed user preference data from three major Zigong lantern exhibitions spanning different regions and time periods. The dataset comprises 12,894 anonymized user records, each containing: User Attributes: Age group (5 categories), gender (3 categories), and regional origin (8 Chinese provinces). Lantern Features: 42-dimensional vector encoding material, color scheme, craftsmanship style, and historical period. Interaction Data: Binary preference labels (like/dislike) and detailed viewing duration logs. To assess model robustness across demographic groups, we stratified the dataset into subgroups based on intersecting protected attributes (age ×gender ×region), ensuring each subgroup contained sufficient samples for meaningful evaluation. The data was split into training (70%), validation (15%), and test (15%) sets while maintaining subgroup proportions. 5.2 Baseline Methods We compared our framework against four categories of baseline approaches representing state-of-the-art methods in bias-aware recommendation systems: Traditional Recommenders: Matrix Factorization [18] and LightGCN [19] Bias-Aware Models: FairGo [20] and FairRec [21] Adversarial Debiasing: ADV-FM [22] and FairAdv [23] Feature Disentanglement: DVBPR [24] and CFC [25] Each baseline was implemented using their original architectures and optimized for our specific task through hyperparameter tuning on the validation set. For fairness comparison, all methods were constrained to use identical input features and evaluation protocols. 5.3 Evaluation Metrics We employed a dual evaluation strategy assessing both recommendation accuracy and fairness metrics: Accuracy Metrics: •Precision@K (K=5, 10) •Recall@K (K=5, 10) •Normalized Discounted Cumulative Gain (NDCG@K) Fairness Metrics: •Demographic Parity Difference (DPD) •Equal Opportunity Difference (EOD) •Subgroup AUC Variance (SAV) The fairness metrics were computed across all demographic subgroups to quantify performance disparities. For example, DPD measures the maximum difference in positive prediction rates between any two subgroups: DPD =max 𝑖, 𝑗 |𝑃(ˆ 𝑌=1|𝐺=𝑖) − 𝑃(ˆ 𝑌=1|𝐺=𝑗)| (12) where 𝐺denotes subgroup membership. -6- Natura Humanitas (2025), 1(1): 1-10 Zhang Haoyang & Zhang Luokuan 5.4 Implementation Details Our framework was implemented in PyTorch with the following configuration: Feature Extractor: 6-layer Transformer with 8 attention heads, hidden dimension 256 Adversarial Discriminator: 3-layer MLP with ReLU activation and dropout (p=0.2) Optimization: Adam optimizer with initial learning rate 1e-4, batch size 128 Hyperparameters: 𝜆1=0.5(adversarial weight), 𝜆2=0.3(augmentation weight) Training: Early stopping with patience=10 epochs on validation NDCG@10 The gradient reversal layer used a gradually increasing schedule for the adversarial weight 𝜆1, starting from 0 and reaching its maximum value over the first 20% of training iterations. This annealing strategy helped stabilize early training phases [26]. For the bias-conflicting augmentation, we set the perturbation magnitude 𝜖=0.1through grid search on the validation set. The augmentation module generated 3 synthetic samples per original instance, effectively increasing the training set size by 300% while maintaining label consistency. All experiments were conducted on NVIDIA V100 GPUs with 32GB memory, with each model run repeated 5 times with different random seeds to account for initialization variance. Statistical significance of performance differences was assessed using paired t-tests at p<0.05 level. 6 Experimental Results The comprehensive evaluation of our bias-aware feature engineering framework demonstrates significant improvements in both recommendation accuracy and fairness metrics compared to existing approaches. This section presents quantitative results across multiple dimensions, followed by detailed analysis of the framework’s behavior under different bias conditions. 6.1 Overall Performance Comparison Table 1 compares the proposed framework against baseline methods on the Zigong lantern preference dataset, evaluating both accuracy and fairness metrics. Our approach achieves superior performance across all metrics, particularly in reducing bias while maintaining high recommendation quality. Table 1: Performance comparison on Zigong lantern preference dataset Accuracy Metrics (↑) Fairness Metrics (↓) Method NDCG@10 Precision@10 Recall@10 DPD EOD SAV Matrix Factorization 0.612 0.538 0.487 0.218 0.195 0.041 LightGCN 0.647 0.572 0.526 0.187 0.163 0.036 FairGo 0.634 0.561 0.508 0.142 0.121 0.028 FairRec 0.628 0.554 0.502 0.135 0.118 0.025 ADV-FM 0.659 0.584 0.539 0.153 0.134 0.031 FairAdv 0.663 0.589 0.543 0.126 0.112 0.023 DVBPR 0.671 0.597 0.551 0.118 0.105 0.021 CFC 0.678 0.604 0.558 0.107 0.097 0.019 Proposed 0.703 0.628 0.582 0.082 0.074 0.014 The proposed framework achieves a 3.7% improvement in NDCG@10 over the best baseline (CFC) while reducing demographic parity difference (DPD) by 23.4%. This demonstrates our method’s ability to simultaneously enhance recommendation quality and fairness. The equal opportunity difference (EOD) of 0.074 represents a 38% reduction compared to conventional adversarial debiasing approaches like FairAdv. -7- Natura Humanitas (2025), 1(1): 1-10 Zhang Haoyang & Zhang Luokuan 6.2 Subgroup Performance Analysis To understand how the framework performs across different demographic segments, we analyze recommendation accuracy for each protected attribute group. Figure 2 illustrates the distribution of NDCG@10 scores across age, gender, and regional subgroups. Figure 2: Distribution of recommendation accuracy (NDCG@10) across demographic subgroups The proposed framework shows significantly reduced performance variance across subgroups compared to baseline methods. The standard deviation of NDCG@10 scores decreases from 0.042 (LightGCN) to 0.018 (Proposed), indicating more equitable performance distribution. Notably, traditionally disadvantaged groups such as older users (65+ years) and rural residents show 12-15% improvement in recommendation accuracy compared to baseline methods. 6.3 Bias Mitigation Effectiveness The adversarial component’s effectiveness in suppressing bias-related patterns becomes evident when examining latent feature representations. Figure 3 compares the t-SNE projections of latent features before and after adversarial training. Figure 3: t-SNE visualization of latent feature distributions with respect to protected attributes -8- Natura Humanitas (2025), 1(1): 1-10 Zhang Haoyang & Zhang Luokuan The baseline model (without adversarial training) shows clear clustering of features by demographic attributes, indicating strong bias encoding. After applying our framework, these demographic-specific clusters become indistinguishable while maintaining separation based on genuine preference patterns. Quantitative analysis using the Mutual Information Gap (MIG) metric [27] confirms this observation, showing a 67% reduction in bias-related information in the latent space. 6.4 Ablation Study To understand the contribution of each framework component, we conducted systematic ablation tests by removing individual elements while keeping other parameters constant. Table 2 presents the results of this analysis. Table 2: Ablation study of framework components Configuration NDCG@10 DPD (↓) EOD (↓) Full Framework 0.703 0.082 0.074 w/o Adversarial Discriminator 0.681 0.132 0.121 w/o Gradient Reversal 0.673 0.154 0.138 w/o Bias-Conflicting Augment 0.692 0.097 0.088 w/o Attention Masking 0.697 0.091 0.083 The ablation study reveals that each component contributes significantly to the framework’s overall performance. The adversarial discriminator has the most substantial impact on fairness metrics (35.6% increase in DPD when removed), while the bias-conflicting augmentation shows the strongest effect on recommendation accuracy (1.6% drop in NDCG@10 when removed). The attention masking mechanism provides balanced improvements to both accuracy and fairness. 6.5 Robustness to Varying Bias Levels To evaluate the framework’s adaptability to different bias conditions, we conducted experiments on synthetically biased versions of the dataset. Figure 4 shows how performance metrics vary with increasing bias severity (controlled by correlation strength between protected attributes and preference labels). Figure 4: Framework performance across varying levels of synthetic bias The proposed framework maintains stable performance across all bias levels, with less than 5% variation in NDCG@10 -9-