Emulating Public Opinion: A Proof-of-Concept of AI-Generated Synthetic Survey Responses for the Chilean Case

Emulating Public Opinion A Proof-of-Concept of AI-Generated Synthetic Survey Responses for the Chilean Case Bastián González-Bustamante Nando Verelst Carla Cisternas Empiria Lab Method Series (ISSN pending) Issue 1 ·September 2025 Emulating Public Opinion A Proof-of-Concept of AI-Generated Synthetic Survey Responses for the Chilean Case Bastián González-Bustamante Nando Verelst Carla Cisternas Recommended Citation González-Bustamante, B., Verelst, N., & Cisternas, C. (2025). Emulating Public Opinion: A Proof-of-Concept of AI-Generated Synthetic Survey Responses for the Chilean Case. Empiria Lab Method Series, 1. https://doi.org/10.5281/zenodo.17077752. Empiria Lab Method Series (ISSN pending) Issue 1 ·September 2025 Emulating Public Opinion: A Proof-of-Concept of AI-Generated Synthetic Survey Responses for the Chilean Case Bastián González-Bustamante1 2 3∗, Nando Verelst1 2†, and Carla Cisternas1 3‡ 1Empiria Lab 2Universidad Diego Portales 3Leiden University Abstract Large Language Models (LLMs) offer promising avenues for methodological and applied innovations in survey research by using synthetic respondents to emulate human answers and behaviour, potentially mitigating measurement and representation errors. However, the extent to which LLMs recover aggregate item distributions remains uncertain and downstream applications risk reproducing social stereotypes and biases inherited from training data. We evaluate the reliability of LLM-generated synthetic survey responses against ground-truth human responses from a Chilean public opinion probabilistic survey. Specifically, we benchmark 128 prompt-model-question triplets, generating 189,696 synthetic profiles, and pool performance metrics (i.e., accuracy, precision, recall, and F1-score) in a meta-analysis across 128 question-subsample pairs to test for biases along key sociodemographic ∗Research Coordinator, Empiria Lab; Associate Professor, Faculty of Administration and Economics, Universidad Diego Portales; Postdoctoral Researcher in Computational Social Science, Faculty of Governance and Global Affairs, Leiden University. †Associate Researcher, Empiria Lab; PhD Researcher, Faculty of Administration and Economics, Universidad Diego Portales. ‡Director and Associate Researcher, Empiria Lab; PhD Researcher, Faculty of Humanities, Leiden University. 1 dimensions. The evaluation spans OpenAI’s GPT family and o-series reasoning models, as well as Llama and Qwen checkpoints. Three results stand out. First, synthetic responses achieve excellent performance on trust items (F1-score and accuracy >0.90). Second, GPT-4o, GPT-4o-mini and Llama 4 Maverick perform comparably on this task. Third, synthetic-human alignment is highest among respondents aged 45-59. Overall, LLM-based synthetic samples approximate responses from a probabilistic sample, though with substantial item-level heterogeneity. Capturing the full nuance of public opinion remains challenging and requires careful calibration and additional distributional tests to ensure algorithmic fidelity and reduce errors. Keywords: synthetic samples; survey data; LLMs; benchmark; public opinion 1 Introduction Traditional public opinion surveys face a number of challenges and risks related to measurement and representation dimensions, including, for example, coverage error due to incomplete frames and hard-to-reach groups, sampling error resulting from finite samples and complex designs, nonresponse error stemming from low participation and interview fatigue, measurement error introduced by questionnaire wording, and processing errors in coding and post-survey adjustments, among others (Groves,1989;Groves and Lyberg, 2010;Weisberg,2005). These errors could be amplified by substantial financial, human, and logistical demands, such as time spent on instrument design, piloting, and fieldwork that often forces a cost-quality trade-off that may distort population inferences. Consequently, there is a growing demand in the social sciences and market research for methods that reduce burden and cost while maintaining and improving overall data quality. Against this backdrop, Large Language Models (LLMs), trained extensively on vast and diverse data, emerge as promising alternatives for new research possibilities and applied research, including handling the abovementioned survey research limitations and measurement and representation errors. Indeed, recent advances in generative artificial intelligence (AI) suggest LLMs could serve for a number of classification tasks, including the creation of synthetic samples, providing simulated responses reflective of broader societal attitudes and behaviours (Argyle et al.,2023;Gilardi et al.,2023;GonzálezBustamante,2024). The synthetic samples specifically may leverage the ability of LLMs to generate contextually informed responses based on individual-level demographic characteristics and attitudes, and, in this way, potentially emulate public opinion without direct interaction with human respondents. This methodological innovation opens new avenues for rapid data collection, experimentation with sensitive topics, and a deeper understanding of complex public opinion dynamics that complement or even partially 2 substitute for traditional surveys. Thus, the primary objective of this working paper is to evaluate the effectiveness and reliability of LLM-generated synthetic survey responses in reflecting real-world public opinion in Chile. Specifically, we aim to assess the predictive accuracy of a number of state-of-the-art private and open-source LLMs by comparing their synthetic respondents against human probabilistic responses.1Furthermore, we examine the impact of incorporating a variety of contextual information, including demographic and attitudinal variables, on the predictive performance of the models. In doing so, we contribute to the understanding of the feasibility and potential downstream applications of synthetic survey methodologies and LLMs in public opinion research. 2 From Synthetic Data to Synthetic Publics: LLMs in Public Opinion Research Synthetic data have a long tradition in the social sciences, initially used to deal with nonresponse using multiple imputation (Rubin,1993; see also Little,1993;Little and Rubin,1987;Raghunathan et al.,2003). However, while the current research in synthetic data is mainly motivated by harvesting considerable data to train machine learning models, some government agencies are exploring these approaches to broaden public access to their resources (Drechsler and Haensch,2023). Indeed, there is potential related to privacy-preserving and cost-efficient substitutes for microdata, aiming to reproduce joint distributions without exposing individual, sensitive records. Although there are several applications, from patient information and health records (Choi et al.,2018;Yahi et al., 2017) to environmental monitoring (Allken et al.,2019), all of them share a common goal: the synthetic data must be similar to the results obtained using actual data (Drechsler and Haensch,2023). In this sense, synthetic data may not be merely a way to deal with missingness or privacy concerns for survey-oriented applications; when properly constructed and validated, they can act as stand-ins for hard-to-obtain samples, enabling faster iteration and lower-cost implementation. A number of early applications originated from techniques developed to address class imbalance and small-Nconstraints. Methods like Synthetic Minority Oversampling Technique (SMOTE) and its variants showed that augmenting minority classes can reduce the 1By a synthetic respondent, we mean an AI-generated persona used to simulate a survey participant. We create these personas by giving an LLM basic traits, as we further detail in the method section. Then, we ask it to answer as if it were that person. 3 accuracy paradox2and improve sensitivity in rare event settings.3Subsequent research generalised this intuition to broader data scarcity, showing that generators which preserve covariance structure and local dependencies (e.g., optimised multivariable Kernel Density Estimation, KDE, for small biomedical datasets or fuzzy diffusions via k-nearest neighbours) can expand datasets while maintaining multivariate relationships (Fowler et al., 2020;Sivakumar et al.,2022). For survey research, the relevance of these advances lies less in their original domains than in the transferable principle that credible synthetic data must respect the joint distribution of key covariates rather than merely reproduce marginals. Modern methods have been increasingly tackling the challenge of data scarcity and sparsity, moving beyond the focus on class distribution. Fowler et al. (2020), for example, developed a method using optimised multivariate KDE to generate entire synthetic populations from small biomedical datasets. Their objective was not to balance classes but to prevent overfitting by meticulously replicating the complex, correlated covariance of the original data, thereby creating a sufficiently large and representative dataset for modelling where none existed before. This focus on the small data problem is also evident in the work of Sivakumar et al. (2022), who created the kNN Mega-Trend Diffusion (kNNMTD) method. This approach is agnostic to the learning task, whether regression or classification, and is designed explicitly for tiny datasets. Indeed, the kNNMDT method is able to generate plausible synthetic data that faithfully maintains the original data correlation structure by applying a fuzzy-set-based information diffusion technique locally via k-nearest neighbours, preventing overestimation. Parallel advances in deep generative modelling, such as Generative Adversarial Networks (GANs; see Goodfellow et al.,2014) and diffusion models, extended synthetic data beyond tabular settings to sequential and textual modalities.4Beyond their use in time series (see de Souza et al.,2023) and text (see Tolba et al.,2021), GANs have been effectively applied to augment structured survey data in the social sciences. Ruiz-Gándara 2Accuracy, as a classic machine learning indicator, reflects the proportion of correct predictions of a model in the test set, however, it could be misled by data imbalance because the model is predicting the majority class and is not able to identify underrepresented categories (González-Bustamante,2023, 2025b). 3The foundational work in synthetic sampling, notably the SMOTE, was indeed primarily designed to mitigate class imbalance by generating examples for the minority class. SMOTE remains effective nowadays. For instance, de Blasio et al. (2022) demonstrated that it was important for transforming a model with zero sensitivity for detecting rare instances of municipal corruption into a viable tool for policy intervention. Without synthetic oversampling, their model failed to identify positive cases, a classic example of the accuracy paradox. Similarly, Walid et al. (2022) and Tchokote and Tagne (2025) found that advanced SMOTE variants were effective for enhancing model robustness and prediction accuracy in educational data mining and multimodal hate speech detection tasks, respectively. 4Moving beyond simple tabular data, for example, some techniques could generate synthetic instances in sparse regions of the feature space and assign a new label distribution by interpolating the ones of neighbouring instances (González et al.,2021). Other progress has been made in the integration of synthetic sampling and multimodal pipelines (see Tchokote and Tagne,2025). 4 et al. (2025) demonstrated that GANs could generate synthetic survey responses that maintained the original data’s statistical behaviour, spurring the development of novel harmonic mean-based indices for quality evaluation. In addition, comparative studies have emphasised that quality matters more than quantity (de Souza et al.,2023), proposing evaluation frameworks that benchmark not only predictive accuracy, but also distributional similarity based on measures such as Maximum Mean Discrepancy (MMD) and PCA-based (i.e., based on Principal Component Analysis) diagnostics (Fowler et al., 2020), along with correlation-preservation metrics such as Pairwise Correlation Difference (PCD; see Sivakumar et al.,2022).5 Against this methodological backdrop, a new strand evaluates LLMs directly as synthetic respondents. The core design emulates survey interviews by conditioning an LLM on respondent profiles based on demographics and, when available, attitudinal traits (Argyle et al.,2023;Kim and Lee,2024). Evidence is mixed but promising: LLMs can recover aggregate distributions for specific items, reproduce well-known covariates-attitude associations, and display sensitivity to profile granularity and prompt wording (Argyle et al., 2023;Bisbee et al.,2024). Shrestha et al. (2024) explored the use of GPT-4 to generate synthetic data for policy surveys across diverse cultural contexts, reaching a reasonable aggregate alignment with human responses, however, they also uncovered systematic positive biases and cultural variability in the accuracy of synthetic data. Boelaert et al. (2025) indicated, on the other hand, that current models could hardly emulate human respondents because of machine bias. In sum, this mixed evidence highlights both the potential of LLMs to create large-scale synthetic data and the inherent risk of introducing biases that may be difficult to detect because of the intrinsic patterns from pretraining data and instruction tuning datasets (Geng et al.,2024;González-Bustamante,2024). Indeed, the risk for downstream applications based on LLMs could be associated with encoding social stereotypes or political biases, yielding subgroups distortions or directional tilt on contentious issues (Morris,2025;Santurkar et al.,2023;Qu and Wang, 2024). For this reason, the level of algorithmic fidelity, which reflects how LLMs can replicate effectively socio-cultural contexts and nuanced viewpoints of subpopulations, is quite relevant (Argyle et al.,2023;Ma et al.,2025). In this sense, safeguarding algorithmic fidelity should motivate evaluation practices that move beyond headline accuracy by focusing on subgroups, evaluation of prompt perturbations (see Bisbee et al.,2024), among other innovative approaches. For survey emulation, validating the synthetic distribution against probabilistic samples should be considered as a starting point to test the quality of the AI-generated synthetic profiles. 5Alternative options for exploring the distribution are measures such as Earth-Mover’s Distance (EMD; see Boelaert et al.,2025;González-Bustamante,2019;Lupu et al.,2017). 5 3 Methodology 3.1 Survey Emulation and Probabilistic Sample For survey emulation and validation of the synthetic distribution, we rely on the probabilistic sample survey conducted by the Centro de Estudios Públicos (CEP) in Chile, widely recognised for its rigorous methodology and comprehensive coverage of sociodemographic and attitudinal variables. The CEP survey programme has served since 1986 as a barometer of public opinion on political and economic views, key societal concerns, and citizens’ evaluations of government performance and other major actors. The target population of the CEP survey comprises residents aged 18 and older across Chile, excluding provinces classified as difficult to access by the National Statistics Institute (INE in Spanish), namely Easter Island, Palena, and Antarctica. The sample design is probabilistic, stratified by region and by urban-rural area, with random selection at three stages: block, household, and respondent. The instrument is a structured questionnaire incorporating programmed skip patterns, administered face-to-face using electronic devices such as tablets or smartphones to enhance accuracy and consistency in data collection. In addition, survey results are weighted and adjusted post-survey to account for selection probabilities and nonresponse, thereby correcting underor over-representation of specific groups. Specifically, we employed the probabilistic survey number 92, carried out between August 2 and September 12, 2024.6This survey comprised 1,482 face-to-face interviews conducted across 127 Chilean municipalities, yielding a response rate of 61.3% from an initial sample of 2,416. The sample error was ±2.8with a 95% confidence interval. 3.2 LLMs, Prompting Strategies and Performance We tested the performance of a number of state-of-the-art private and open-source LLMs. Our proof-of-concept included flagship OpenAI GPTs such as GPT-4o (2024-11-20), GPT-4o-mini (2024-07-18), GPT-4.5-preview (2025-02-27), GPT-4.1, 4.1-mini and 4.1nano (2025-04-14), as well as reasoning models like o1-mini (2024-09-12) and o3-mini (2025-01-31). We also tested the latest Llama 4 Maverick (400B) and Scout (107B), Llama 3.3 (70B), Qwen 2.5 (32B) and Gemma 3 (12B).7 6We created the synthetic profiles by the end of April 2025. By that time, CEP number 92 was the latest released survey. Further information is available on https://www.cepchile.cl/opinion-publica. 7We tried to consider o1 (2024-12-17) and o4-mini (2025-04-16), however, being reasoning models, the tasks required considerable time. For future applications, Qwen 3 (0.6B, 1.7B, 4B, 8B, 14B, 30B, 32B and 235B), released the last week of April 2025, may be a suitable alternative. In addition, more recent models, such as GPT-5 or the novel, very first open-source OpenAI model GPT-OSS (20B and 120B), released in August 2025, should be good options. It is relevant to note that there are a variety of stateof-the-art closed and open-source flagship models developed by Antrophic, Bespoke Labs, DeepSeek-AI, 6 We employed two distinct prompting strategies to generate synthetic responses. The first strategy utilises exclusively demographic information (i.e., region, urban or rural area, age, sex, educational level, socioeconomic group, and occupation). The second one incorporates attitudinal dimensions, such as individuals’ political interests, ideological identification, and previous demographic characteristics. This dual approach enables us to investigate the effect of varying contextual information levels on the accuracy of LLMs. Therefore, we created 1,482 synthetic profiles per prompting strategy, which implies 38,532 synthetic profiles (1,482 respondent profiles ×2 prompting strategies × 13 LLMs) for the proof-of-concept, focusing on the question: “According to the following scale, how would you rate the country’s current economic situation? Very bad, bad, neither good nor bad, good, very good.” We then benchmarked the LLMs’ predictive performance by estimating standard evaluation metrics, including accuracy, precision, recall, and F1-score (Biecek and Burzykowski, 2021;González-Bustamante,2023,2025b). We reduced the list of models for benchmarking based on the proof-of-concept indicators8provided in the Appendix in order to conduct an error-rate analysis using the questions in Table 1 as ground-truth. Consequently, we tested GPT-4o (2024-11-20), GPT-4o-mini (2024-07-18), Llama 4 Maverick (400B), and Qwen 2.5 (32B), which involved the creation of 189,696 synthetic profiles (1,482 respondent profiles ×2 prompting strategies ×4 LLMs ×16 ground-truth questions, including the proof-of-concept question and the expanded ones for benchmark).9This implies that we have 128 observations for the prompting-LLM-question triplet. 3.3 Meta-Analysis of Bias We focus GPT-4o-mini (2024-07-18) on 16 ground-truth questions crossed with eight sociodemographic subsamples to analyse potential bias in 128 combinations for the questionsubsample pairs (16 questions ×8 subsamples). The subsets were elaborated based on a number of relevant sociodemographic variables, including area (urban vs. rural), gender (female vs. male), and age groups (i.e., 18-29, 30-44, 45-59, and 60 and older). We focused on the F1-score, which is appropriate given that the items’ responses can be imbalanced, and then fit random-effects meta-regressions to separate sampling error from Mistral, among other providers, that are worthy of trying (see González-Bustamante,2025a). 8The goodness-of-prediction indicators of the proof-of-concept against economic perception did not show the best performance, however, as we will discuss in the results for the final benchmark, some model ×question pairs show relatively high performance with F1-scores above 0.9, particularly for questions related to trust. This suggests that there is considerable heterogeneity not only depending on the model but also in relation to the question. 9By a ground-truth question, we mean the human responses to a specific survey item from our reference probabilistic sample. We use this observed distribution as the benchmark to evaluate LLMgenerated answers. This ground-truth is an empirical measure, not normative, and is subject to the survey sampling error indicated above. 7 References Allken, V., Handegard, N. O., Rosen, S., Schreyeck, T., Mahiout, T., and Malde, K. (2019). Fish species identification using a convolutional neural network trained on synthetic data. ICES Journal of Marine Science, 76(1):342–349. https://doi.org/10.1093/icesjms/fsy147. Argyle, L. P., Busby, E. C., Fulda, N., Gubler, J. R., Rytting, C., and Wingate, D. (2023). Out of One, Many: Using Language Models to Simulate Human Samples. Political Analysis, 31(3):337–351. https://doi.org/10.1017/pan.2023.2. Biecek, P. and Burzykowski, T. (2021). Explanatory Model Analysis. Chapman and Hall/CRC, New York. Bisbee, J., Clinton, J. D., Dorff, C., Kenkel, B., and Larson, J. M. (2024). Synthetic Replacements for Human Survey Data? The Perils of Large Language Models. Political Analysis, 32(4):401–416. https://doi.org/10.1017/pan.2024.5. Boelaert, J., Coavoux, S., Ollion, E., Petev, I., and Präg, P. (2025). Machine Bias. How Do Generative Language Models Answer Opinion Polls? Sociological Methods & Research, 54(3):1156–1196. https://doi.org/10.1177/00491241251330582. Choi, E., Biswal, S., Malin, B., Duke, J., Stewart, W. F., and Sun, J. (2018). Generating Multi-label Discrete Patient Records using Generative Adversarial Networks. Preprint, arXiv, Cornell University. https://doi.org/10.48550/arXiv.1703.06490. de Blasio, G., D’Ignazio, A., and Letta, M. (2022). Gotham city. Predicting ‘corrupted’ municipalities with machine learning. Technological Forecasting and Social Change, 184:122016. https://doi.org/10.1016/j.techfore.2022.122016. de Souza, M. D., Junior, C. R. S., Quintino, J., Santos, A. L., da Silva, F. Q. B., and Zanchettin, C. (2023). Exploring the Impact of Synthetic Data on Human Activity Recognition Tasks. Procedia Computer Science, 222:656–665. https://doi.org/10.1016/j.procs.2023.08.203. Drechsler, J. and Haensch, A.-C. (2023). 30 Years of Synthetic Data. Preprint, arXiv, Cornell University. https://doi.org/10.48550/arXiv.2304.02107. Fowler, E. E., Berglund, A., Schell, M. J., Sellers, T. A., Eschrich, S., and Heine, J. (2020). Empirically-derived synthetic populations to mitigate small sample sizes. Journal of Biomedical Informatics, 105:103408. https://doi.org/10.1016/j.jbi.2020.103408. 14 Geng, M., He, S., and Trotta, R. (2024). Are Large Language Models Chameleons? Preprint, arXiv, Cornell University. https://doi.org/10.48550/arXiv.2405.19323. Gilardi, F., Alizadeh, M., and Kubli, M. (2023). ChatGPT outperforms crowd workers for text-annotation tasks. Proceedings of the National Academy of Sciences, 120(30):e2305016120. https://doi.org/10.1073/pnas.2305016120. González, M., Luengo, J., Cano, J.-R., and García, S. (2021). Synthetic Sample Generation for Label Distribution Learning. Information Sciences, 544:197–213. https://doi.org/10.1016/j.ins.2020.07.071. González-Bustamante, B. (2019). Brechas, representación y congruencia éliteciudadanía en Chile y Uruguay. Convergencia Revista de Ciencias Sociales, 80:1–27. https://doi.org/10.29101/crcs.v26i80.11097. González-Bustamante, B. (2023). Critical events and ministerial turnover in Latin American presidential democracies. PhD thesis, St Hilda’s College, University of Oxford. https://dx.doi.org/10.5287/ora-qz8kdd44g. González-Bustamante, B. (2024). Benchmarking LLMs in Political Content TextAnnotation: Proof-of-Concept with Toxicity and Incivility Data. Preprint, arXiv, Cornell University. https://doi.org/10.48550/arXiv.2409.09741. González-Bustamante, B. (2025a). Charting Reproducibility and Performance: LLMs in Multilingual Toxic Speech Detection. In LLM Pre-Conference Workshop, European Political Science Association (EPSA), Madrid, Spain. Universidad Carlos III de Madrid. Available at https://talkingtomachines.org/large-language-models-workshop. González-Bustamante, B. (2025b). Machine Learning and Political Events: Application of a Semi-supervised Approach to Produce a Dataset on Presidential Cabinets. Social Science Computer Review. OnlineFirst. https://doi.org/10.1177/08944393251315917. Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014). Generative Adversarial Networks. Preprint, arXiv, Cornell University. https://doi.org/10.48550/arXiv.1406.2661. Groves, R. M. (1989). Survey Errors and Survey Costs. Wiley. https://doi.org/10.1002/0471725277. Groves, R. M. and Lyberg, L. (2010). Total Survey Error: Past, Present, and Future. Public Opinion Quarterly, 74(5):849–879. https://doi.org/10.1093/poq/nfq065. 15 Kim, J. and Lee, B. (2024). AI-Augmented Surveys: Leveraging Large Language Models and Surveys for Opinion Prediction. Preprint, arXiv, Cornell University. https://doi.org/10.48550/arXiv.2305.09620. Little, R. J. and Rubin, D. B. (1987). Statistical analysis with missing data. Wiley. Little, R. J. A. (1993). Statistical Analysis of Masked Data. Journal of Official Statistics, 9(2):407–426. Lupu, N., Selios, L., and Warner, Z. (2017). A New Measure of Congruence: The Earth Mover’s Distance. Political Analysis, 25(1):95–113. https://doi.org/10.1017/pan.2017.2. Ma, B., Yoztyurk, B., Haensch, A.-C., Wang, X., Herklotz, M., Kreuter, F., Plank, B., and Assenmacher, M. (2025). Algorithmic Fidelity of Large Language Models in Generating Synthetic German Public Opinions: A Case Study. Preprint, arXiv, Cornell University. https://doi.org/10.48550/arXiv.2412.13169. Morris, E. (2025). Your Polls on ChatGPT. White Paper, Verasight. Available at https://report.verasight.io/synthetic-sampling. Qu, Y. and Wang, J. (2024). Performance and biases of Large Language Models in public opinion simulation. Humanities and Social Sciences Communications, 11:1095. https://doi.org/10.1057/s41599-024-03609-x. Raghunathan, T. E., Reiter, J. P., and Rubin, D. B. (2003). Multiple Imputation for Statistical Disclosure Limitation. Journal of Official Statistics, 19(1):1–16. Rubin, D. B. (1993). Discussion: Statistical Disclosure Limitation. Journal of Official Statistics, 9(2):461–468. Ruiz-Gándara, A., Casales-García, V., and González-Abril, L. (2025). Artificial generation of survey data on the expected bitterness of beer. Expert Systems with Applications, 275:126950. https://doi.org/10.1016/j.eswa.2025.126950. Santurkar, S., Durmus, E., Ladhak, F., Lee, C., Liang, P., and Hashimoto, T. (2023). Whose Opinions Do Language Models Reflect? Preprint, arXiv, Cornell University. https://doi.org/10.48550/arXiv.2303.17548. Shrestha, P., Krpan, D., Koaik, F., Schnider, R., Sayess, D., and Binbaz, M. S. (2024). Beyond WEIRD: Can synthetic survey participants substitute for humans in global policy research? Behavioral Science & Policy, 10(2):26–45. https://doi.org/10.1177/23794607241311793. 16 Sivakumar, J., Ramamurthy, K., Radhakrishnan, M., and Won, D. (2022). Synthetic sampling from small datasets: A modified mega-trend diffusion approach using k-nearest neighbors. Knowledge-Based Systems, 236:107687. https://doi.org/10.1016/j.knosys.2021.107687. Tchokote, E. L. T. and Tagne, E. F. (2025). Effective multimodal hate speech detection on Facebook hate memes dataset using incremental PCA, SMOTE, and adversarial learning. Machine Learning with Applications, 20:100647. https://doi.org/10.1016/j.mlwa.2025.100647. Tolba, M., Ouadfel, S., and Meshoul, S. (2021). Hybrid ensemble approaches to online harassment detection in highly imbalanced data. Expert Systems with Applications, 175:114751. https://doi.org/10.1016/j.eswa.2021.114751. Walid, M. A. A., Ahmed, S. M. M., Zeyad, M., Galib, S. M. S., and Nesa, M. (2022). Analysis of machine learning strategies for prediction of passing undergraduate admission test. International Journal of Information Management Data Insights, 2(2):100111. https://doi.org/10.1016/j.jjimei.2022.100111. Weisberg, H. F. (2005). The Total Survey Error Approach: A Guide to the New Science of Survey Research. University of Chicago Press. https://doi.org/10.7208/chicago/9780226891293.001.0001. Yahi, A., Vanguri, R., Elhadad, N., and Tatonetti, N. P. (2017). Generative Adversarial Networks for Electronic Health Records: A Framework for Exploring and Evaluating Methods for Predicting Drug-Induced Laboratory Test Trajectories. Preprint, arXiv, Cornell University. https://doi.org/10.48550/arXiv.1712.00164. 17 Appendix Table A.1: Goodness-of-Prediction Indicators of Proof-of-Concept Against Economic Perception Ground-Truth Model Prompt Accuracy Precision Recall F1-Score GPT-4o-mini (2024-07-18) Demographics 0.530 0.458 0.530 0.473 Attitudes 0.553 0.516 0.553 0.517 o1-mini (2024-09-12) Demographics 0.515 0.498 0.515 0.497 Attitudes 0.465 0.472 0.465 0.462 o3-mini (2025-01-31) Demographics 0.534 0.485 0.534 0.486 Attitudes 0.476 0.467 0.476 0.468 GPT-4o (2024-11-20) Demographics 0.492 0.481 0.492 0.470 Attitudes 0.439 0.475 0.439 0.404 GPT-4.5-preview (2025-02-27) Demographics 0.543 0.462 0.543 0.449 Attitudes 0.540 0.471 0.540 0.467 GPT-4.1 (2025-04-14) Demographics 0.539 0.468 0.539 0.416 Attitudes 0.542 0.465 0.542 0.449 GPT-4.1-mini (2025-04-14) Demographics 0.540 0.468 0.540 0.431 Attitudes 0.540 0.473 0.540 0.435 GPT-4.1-nano (2025-04-14) Demographics 0.471 0.491 0.471 0.477 Attitudes 0.509 0.519 0.509 0.506 Llama 3.3 (70B) Demographics 0.533 0.462 0.533 0.476 Attitudes 0.510 0.476 0.510 0.489 Llama 4 Scout (107B) Demographics 0.527 0.478 0.527 0.444 Attitudes 0.512 0.495 0.512 0.491 Llama 4 Maverick (400B) Demographics 0.530 0.445 0.530 0.443 Attitudes 0.542 0.479 0.542 0.499 Qwen 2.5 (32B) Demographics 0.538 0.470 0.538 0.489 Attitudes 0.527 0.486 0.527 0.505 Gemma 3 (12B) Demographics 0.540 0.495 0.540 0.394 Attitudes 0.544 0.472 0.544 0.457 Note. Accuracy reports the proportion of synthetic samples aligned with the probabilistic reference sample. Precision represents the fraction of predicted positives that are correct, and recall indicates the proportion of actual positives identified. The F1-score is the harmonic measure that combines both precision and recall. 18 Table A.2: Goodness-of-Prediction Indicators of Expanded Ground-Truth Questions for Benchmark – Top 15 Model ×Question Pairs Model Ground-Truth Accuracy Precision Recall F1-Score GPT-4o (2024-11-20) Trust in political parties 0.940 0.883 0.940 0.911 GPT-4o-mini (2024-07-18) Trust in political parties 0.940 0.883 0.940 0.911 Llama 4 Maverick (400B) Trust in political parties 0.940 0.883 0.940 0.911 GPT-4o-mini (2024-07-18) Trust in Congress 0.893 0.797 0.893 0.842 GPT-4o (2024-11-20) Trust in Congress 0.893 0.797 0.893 0.842 Llama 4 Maverick (400B) Trust in Congress 0.822 0.798 0.822 0.807 Qwen 2.5 (32B) Trust in Congress 0.777 0.797 0.777 0.779 GPT-4o (2024-11-20) Trust in justice 0.820 0.707 0.820 0.748 Llama 4 Maverick (400B) Trust in television 0.807 0.716 0.807 0.744 GPT-4o (2024-11-20) Trust in television 0.821 0.680 0.822 0.744 GPT-4o (2024-11-20) Trust in government 0.807 0.711 0.807 0.723 Llama 4 Maverick (400B) Trust in government 0.795 0.695 0.795 0.737 GPT-4o-mini (2024-07-18) Trust in government 0.778 0.715 0.778 0.732 Qwen 2.5 (32B) Trust in government 0.758 0.679 0.758 0.712 GPT-4o-mini (2024-07-18) Trust in television 0.696 0.708 0.696 0.686 Note. Accuracy reports the proportion of synthetic samples aligned with the probabilistic reference sample. Precision represents the fraction of predicted positives that are correct, and recall indicates the proportion of actual positives identified. The F1-score is the harmonic measure that combines both precision and recall. 19 Empiria Lab Method Series https://empirialab.cl/repositorio Bmetho[email protected] Empiria Lab is a team specialised in data science and artificial intelligence that delivers technological solutions and data analysis for public and private sector clients. Empiria Lab Method Series (ISSN pending) presents concise, practice-oriented working papers that document the technical aspects of our projects, including methodological details, design choices, analytical workflows, and implementation notes. The series is published without a fixed schedule and reflects the timelines of projects in which Empiria Lab is involved. Many projects and services produce more client-friendly reports or outcomes, and this series preserves the methodological detail that underpins the work. Each issue undergoes an internal technical review to promote clarity, rigour, and reproducibility. The views expressed and any remaining errors are the authors’ own. 2025 Empiria Lab and the author(s). This work is licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0). License text: https://creativecommons.org/licenses/by/4.0.