Problematizing the 'family' in welfare and social services data systems

Problematizing the ‘family’ in welfare and social services data systems LAURA CARTER 2024-2025 TECH POLICY FELLOW 1 2 Contents Executive Summary Introduction The importance of welfare benefits Welfare benefits and data systems Linking and automation for fraud detection Individual, ‘family’ and ‘household’ benefits Linking individuals into families Assumptions about families and households Case studies The UK: Supporting Families Pakistan: CNIC identity cards The USA: SNAP Recommendations About the author Acknowledgements Suggested citation Endnotes 3 6 6 6 8 13 14 15 17 17 18 19 20 21 21 21 22 Executive Summary Governments have an obligation to provide social security welfare benefits in cash or kind, though the extent and methods through which this is implemented has varied between countries and over time. Modern welfare systems rely on identification systems and government record-keeping to process applications and determine eligibility. Increasingly, governments are using data systems, algorithms, and AI tools as part of service provisions and collecting and linking much more personal information about individuals. One purpose for this linkage is to detect fraud: though there are numerous examples of inaccurate and harmful automated fraud detection, and storing and linking large amounts of data may not comply with the Fair Information Practice of data minimization. Linking data about individuals together is also done when welfare benefits are designed to be received by groups of individuals: families, households, or other ‘benefits units:’ for example, Supplemental Nutrition Assistance Program (SNAP) in the US, or social services assistance through the ‘Supporting Families Programme’ in the UK. Eligibility for welfare services may be demonstrated through evidence provided by other family members, as in Pakistan’s eligibility determination for the Computerized National Identity Card. How individuals are linked together, however, often relies on assumptions about the relationships between individuals within families and households: which may not reflect people’s lived experience. Family relationships may or may not be discernible from outside the family: families may be linked by genetics, legal relationships, or by choices made by individuals. Policy choices that impose a definition of ‘family’ often rely on cultural assumptions: this is particularly the case for whether ‘extended’ family members should be included. Family relationships are not necessary stable over time, nor even necessarily reciprocal. Linking individuals together in welfare data sets, however, often ‘sorts’ 3 individuals into static, mutually exclusive families, even where this does not reflect the reality of their lives. Where eligibility for welfare benefits is determined based on a group of people--whether this is families, households, or other groups—then there may be multiple legitimate ways for people to apply as a group. An experienced benefits navigator can help ensure that applicants provide information enabling them to claim all the benefits to which they are entitled, in order to meet their needs. As more and more welfare benefits systems are datafied, automated, and outsourced to AI tools, however, the opacity of the system increases. This makes it harder for claimants—and navigators—to understand if benefits are being allocated correctly. 4 Policymakers determining welfare benefits policy should: Recognize the complexity of families and households, and avoid enacting policies which privilege certain forms of family or living arrangements. Ensure that welfare benefits applications and eligibility determination helps people who need it the most. Recommendations Data engineers creating and linking datasets about welfare applications and eligibility determination should: Consider the impact of their data structures on different forms of families, and on changing family and household relationships. Recognize that not everyone fits neatly into mutually exclusive families or households. Build data-minimizing systems: only collect the data that is required for a particular application. Software and AI developers working on automation and application tools for welfare benefits should: Recognize that having a lot of data on individuals and families does not necessarily imply having relevant data for training AI models and tools. Consider how automation and AI tools could negatively impact transparency, or even be used to minimize eligibility 5 Introduction 6 The importance of welfare benefits The obligation for governments to provide social security welfare benefits in cash or kind—to protect their people from a lack of income, unaffordable healthcare, or insufficient family support for children (or adult dependents) —is internationally recognized. Family support, in particular, should include “food, clothing, housing, water and sanitation.” 1 2 3 States interpret these obligations in different ways, which have changed over time. After World War II, a Keynesian model of welfare emerged, particularly in the global north, in which the state took a heavily interventionist role. This was replaced from the 1980s onwards with a neoliberal model which prioritized the market as an allocation mechanism, adopted outsourcing and privatization, and aimed to shift responsibility for meeting human needs from the state to individuals and families. The post-World War II shift towards a social investment approach—seeing social services as investments in people, who retain responsibility for meeting their needs—was reversed in the aftermath of the 2008 global financial crisis, as states returned to neoliberal policies and added austerity measures to the challenges faced by state welfare systems. 4 5 6 Welfare benefits and data systems Modern welfare systems, which emerged in the twentieth century, rely on identification systems and government record-keeping. As computer technology has become more accessible, these systems have increasingly been digitized. Government administrations are increasingly using data systems, algorithms and AI tools to determine eligibility for welfare benefits. 7 India’s Aadhaar system, launched in 2009, assigns every Indian a unique 12digit number linked to demographic and biometric data. The system’s coercive nature, and wide and poorly-regulated use of biometrics, have been widely criticized, but the roll out of the system has continued. 8 9 10 7 The increased digitization and use of computerized systems to administer welfare benefits has meant that government authorities collect and link much more personal information about individuals. This enabled invasive surveillance and stigmatization in the form of what Virginia Eubanks has called the “digital poorhouse.” The examples she documented collected and retained large amounts of data for a long time. These included the Allegheny Family Screening Tool in Pennsylvania, which disproportionately targeted Black families for child removal, and Los Angeles’s coordinated entry system for homelessness services, which required unhoused people to share extensive and intimate details of their lives: the data was retained even for those who did not receive any housing support. 11 12 13 Slovenia introduced the e-Sociala program—connecting multiple social service databases and later applying machine learning and AI processes to them, to optimize social workers’ work processes—in 2010. E-Sociala can also access financial information for different members of the same family.14 At the same time, welfare systems shifted to focus less on meeting the needs of recipients, to maintaining an ‘orderly society.’ Neoliberal changes to welfare policy shifted towards individual and familial responsibility, rather than the responsibility of the state to take care of its people. While the idea of the ‘deserving’ and ‘undeserving’ poor had a long history, under neoliberalism, almost no-one was considered ‘deserving’ of state support. Social welfare institutions were also framed not as meeting human needs, but as power grabs by bureaucrats that were run in the interests of their employees. 15 16 17 18 In the UK, the New Labour government (which came to power in 1997) was committed to ‘joined-up working’ including coordination between services, which required those services to share information in the form of data. Underpinning this sharing of personal data was a view that rights (including to privacy) were reciprocal: data was seen as a reasonable contribution for individuals to make in exchange for support from the state. 19 20 8 As new digital, data, algorithmic and AI tools have become available, governments have begun to adopt these into their public services, including in screening for child maltreatment and administering homelessness services.21 The adoption of these systems have however been criticized for their lack of transparency for the recipients of welfare benefits, as well as for the staff administering benefit distribution. In a report on the ‘digital welfare state,’ the UN Special Rapporteur on Extreme Poverty and Human Rights expressed concerns about increasing rigidity in benefits eligibility determination as a result of increasing digitization; he argued that countries need to “alter course significantly and rapidly to avoid stumbling, zombie-like, into a digital welfare dystopia…in which unrestricted data-matching is used to expose and punish the slightest irregularities in the record of welfare beneficiaries.” 22 23 24 A 2025 press release from Prime Minister Keir Starmer promised to “mainline AI into the veins” of the UK including through “revolutionizing public services.”25 Nonetheless, enthusiasm for data-driven welfare benefits systems—and in recent years, AI applications—continues to grow. In 2012, the Danish government established a public agency to centralize the distribution of welfare benefits, together with a Joint Data Unit that linked data from multiple databases in an aim to identify welfare fraud. 27 Linking and automation for fraud detection The internationally-recognized right to social security includes the provision that “qualifying conditions for benefits must be reasonable, proportionate and transparent. The withdrawal, reduction or suspension of benefits should be circumscribed, based on grounds that are reasonable, subject to due process, and provided for in national law.”26 9 Fraudulently claiming (or attempting to claim) benefits is one of the permissible grounds for this to be done. As well as linking individuals into ‘families’ or ‘households,’ data linkage and automation is often done by governments with the intent of addressing fraud. 28 In 2020, the District of Columbia contracted a private company—Pondera Solutions—to deploy data analysis software to detect Supplemental Nutrition Assistance Program (SNAP) benefits fraud.29 A focus on identifying fraud has formed part of a neoliberal shift in the application of welfare policy, which constructs dependents on welfare benefits as ‘undeserving,’ and favours policies which increase insecurity.30 In 2022, the Arkansas Division of Workforce Services (DWS) lost a case against Legal Aid of Arkansas who had sought public records on the use of automated decision-making to detect fraudulent claims for unemployment benefits. The DWS had claimed that it was exempt from disclosing records under freedom of information obligations, because—in applying algorithmic processing to detect fraud—it was acting as a law enforcement agency.31 This focus on fraud has coincided with the increasing use of computerized systems, data, and automated processing to administer welfare benefits, and the more recent turn to machine learning and large language models in social services more broadly. 32 33 In the UK, the Department for Work and Pensions began using an algorithm to identify fraudulent benefits claims in 2021, and the following year deployed four algorithmic models which aimed to prevent fraud in Universal Credit claims in the areas of “people living together, selfemployment, capital, and housing.”34 Family relationships are not necessarily stable over time, nor even necessarily reciprocal. Where welfare benefits eligibility and application systems are digitized and entered into databases, however, data structures often encourage individuals to be sorted into mutually exclusive ‘families:’ even where this does not reflect the reality of their lives. 16 In India, the Federal government is developing a family version of the individual Aadhaar digital ID system, and pilot programmes have been rolled out in states including Uttar Pradesh, Telengana, and Gujarat. The Uttar Pradesh scheme, launched in 2023, uses the slogan Ek Parivar Ek Pehchan: ‘one family, one identity.’ The Telengana pilot, announced in October 2024, aimed to help people access welfare benefits: according to newspaper reports on the launch, it also aimed to ensure that every family was recognized as a single unit and that every individual was part of a single family. The pilot scheme also required that a woman be named head of the family. 78 79 80 81 82 83 Similarly, the composition of a household can change over time. Assuming households—let alone families—are externally discernible and unchanging over time—discriminates against people whose living situations and/or family relationships do not fit neatly into sets of mutually exclusive families or households. Datasets, however, may not be updated regularly enough to reflect changes in people’s lives. And where the data and the reality is out of sync, this may trigger automated fraud detection processing. In Denmark, UDK’s fraud control models identified ‘unusual’ or ‘atypical’ relationships or residency patterns as an indicator of fraud.84 Where eligibility for welfare benefits is determined based on a group of people--whether this is families, households, or other groups—then there may be multiple legitimate ways for people to apply as a group. An experienced benefits navigator can help ensure that applicants provide information enabling them to claim all the benefits to which they are entitled, in order to meet their needs. As more and more welfare benefits systems are datafied, automated, and outsourced to AI tools, however, the opacity of the system increases. This makes it harder for claimants—and navigators—to understand if benefits are being allocated correctly. An austerity-minded operator could also exploit this to minimize the benefits that a claimant is allocated. 17 Case studies The following sections show how attempts to impose a specific datafied definition of ‘family’ fail to recognize the realities of family and interpersonal relationships, and may harm those who are most at need of state support. The UK: Supporting Families85 The Supporting Families Programme in the UK aims to provide targeted social service support to families through a dedicated keyworker. This programming was based on the idea that social service costs were disproportionately being spent on a small number of ‘troubled’ families, who needed to be identified and their problems addressed so that they would be less of a burden on the rest of the community. The Supporting Families Programme is also explicitly a driver of ‘data maturity’ in local authorities (who are responsible for administering the Programme) and use of data-sharing powers under the Digital Economy Act 2017. 86 87 88 89 However, the UK does not have a legal definition of who constitutes a ‘family,’ nor a household registry system, and so the data that is used to assess which families are eligible for the Programme in fact deals with individuals. The choice of data to include and ‘risk factors’ to classify a family as eligible for support promotes a narrow definition of ‘family,’ in which families with two heterosexual parents in a stable cohabiting relationship, where the male partner works while the female partner carries out childcare, are less likely to be labelled ‘troubled.’ 90 91 More concerningly, the Programme relies on the identification of ‘headline problems’ to identify families in need of support: problems under two such ‘headlines’ must be identified. One such problem is domestic abuse in the family. Another is ‘claiming Universal Credit’ - a means-tested social security benefit, which is claimed by a single person or by a couple: payments made to couples are deposited into one bank account, which has been criticized for risking enabling or exacerbating financial abuse, a component of abuse experienced by most domestic abuse survivors. 92 93 94 95 18 A family in which a couple is claiming Universal Credit, and in which one member of the couple is financially abusing the other, is eligible for inclusion in the Supporting Families Programme. A successful outcome for that couple—and an end to support—however, could include “an adult in the family has moved into continuous employment:” even if the abuse has not been addressed. 96 97 This linkage presents problems for those without family support to apply for a CNIC: in a 2013 survey, 17% of women reported that a lack of support from their husband and/or relatives was a reason they had not applied for a CNIC (the equivalent figure for men was 2%).101 Until a high court ruling in 2021, applicants for digital ID cards had to present their father’s ID card: meaning that individuals who were raised by single mothers, or others who did not have contact with their fathers, were unable to access a card.102 Pakistan: CNIC identity cards Pakistan’s modern identity card—termed the Computerized National Identity Card (CNIC)—is required in order to receive support from government programs, as well as to participate in elections, open a bank account, apply for a passport, or purchase a SIM card. Pakistan’s first identity registration system used kinship to verify an applicant’s citizenship, including providing a list of household members and requiring family members to accompany a person applying for an identity card, a legacy that influences the operation of the current National Database and Registration Authority, which issues CNICs. This linkage between family members poses problems for families where different members have different nationalities, as familial ties to a different country—particularly Afghanistan, due to concerns about Afghans acquiring Pakistani identity cards—can throw an individual’s citizenship into question. 98 99 100 The USA: SNAP The Supplemental Nutrition Assistance Program (SNAP) is the USA’s largest food and nutrition benefits program. The first country-wide food stamp program was authorized by the Food Stamp Act of 1964. Food stamp benefits were intended for ‘households,’ defined by the Act as “a group of related or non-related individuals, who are not residents of an institution or boarding house, but are living as one economic unit sharing common cooking facilities and for whom food is customarily purchased in common.” Single individuals who purchased and cooked food at home were also considered a ‘household.’ 103 104 105 106 In subsequent years this definition was amended by successive Acts: explicitly adding adopted and foster children, including elderly ‘meals on wheels’ recipients as ‘households’ and later participants in drug or alcohol addiction rehabilitation programs. In 1993, the Mickey Leland Childhood Hunger Relief Act clarified that children over 21 living with their parents could be considered separate households if they bought and prepared food separately—as could adult siblings who lived together—but children under 21 could not unless they themselves lived with their own spouses and/or children. 107 108 The current definition of ‘household’ is: “(1) An individual living alone; (2) An individual living with others, but customarily purchasing food and preparing meals for home consumption separate and apart from others; or (3) A group of individuals who live together and customarily purchase food and prepare meals together for home consumption.”109 Benefits are granted for an initial period, and applicants are required to recertify at regular intervals. Changes in household composition are not mandatory to report between recertification, but if SNAP is deemed to have been overpaid, then the overpayments will be reclaimed. If the overpayment was due to intentional misrepresentation by the application, it is referred for fraud prosecution. As enthusiasm for automation in welfare benefits grows, the risk of different family arrangements triggering a fraud investigation increases. 110 19 Defining who counts as a ‘household’ is known to be challenging for some applicants. At present, SNAP requires an interview, which gives an applicant the opportunity to discuss their situation with a caseworker. The more complex a household, the more documentary evidence is required to demonstrate income and other requirements. 111 112 Policymakers determining welfare benefits policy should: Recognize the complexity of families and households, and avoid enacting policies which privilege certain forms of family or living arrangements. Ensure that welfare benefits applications and eligibility determination helps people who need it the most. Data engineers creating and linking datasets about welfare applications and eligibility determination should: Consider the impact of their data structures on different forms of families, and on changing family and household relationships. Recognize that not everyone fits neatly into mutually exclusive families or households. Build data-minimizing systems: only collect the data that is required for a particular application. Software and AI developers working on automation and application tools for welfare benefits should: Recognize that having a lot of data on individuals and families does not necessarily imply having relevant data for training AI models and tools. Consider how automation and AI tools could negatively impact transparency, or even be used to minimize eligibility. Recommendations 20 21 About the author Acknowledgements I would like to thank: the UC Berkeley Tech Policy Fellowship leadership team, including Brandie Nonnecke, Sasha Anderson and Sirena Harrop; the 2024-25 fellowship cohort, especially fellow working group members Naureen Rizvi and Gabrielle Hurtubise-Radet, and Christine Galvagna; the people who were kind enough to speak to me about their work, including Elizabeth Bynum Sorell and Max Ghenis who provided feedback on this paper, Stephanie Bell, Greg Bloom, Alex Hanna, Zehra Hashmi, Ariel Kennan, Kevin de Liban, Jen King, Meredith Lee, Betsy Popkin, Morgan Scheuerman, and Ranjit Singh. Parts of this paper draw on my 2023 PhD thesis, supervised by Professors Lorna McGregor and Róisín Ryan-Flood at the University of Essex. Laura Carter is a 2024-25 Tech Policy Fellow at the Goldman School of Public Policy and the CITRIS Policy Lab, University of California Berkeley. She holds a PhD in Human Rights & Research Methods from the University of Essex, and has previously worked for the Ada Lovelace Institute and Amnesty International. She lives on traditional Duwamish lands in Seattle, USA. Suggested citation Carter L, ‘Problematizing the “Family” in Welfare and Social Services Data Systems’ (Berkeley Tech Policy Fellowship 2025) <https://doi.org/10.5281/zenodo.17310314> Endnotes General Comment No. 19: The right to social security (art. 9) 2008 (E/C12/GC/19) para 2. 1 International Covenant on Economic, Social and Cultural Rights 1966 art 9. 2 General Comment No. 19: The right to social security (art. 9) para 6. 3 Flavia Martinelli, ‘Social Services, Welfare States and Places: An Overview’ in Flavia Martinelli, Anneli Anttonen and Margitta Mätzke (eds), Social Services Disrupted: Changes, Challenges and Policy Implications for Europe in Times of Austerity (Edward Elgar Publishing 2017). 4 Martinelli (n 4). 5 Martinelli (n 4). 6 Mike Zajko, ‘Automated Government Benefits and Welfare Surveillance’ (2023) 21 Surveillance & Society 246, 250. 7 UN Special Rapporteur on Extreme Poverty and Human Rights, ‘Report on Digital Technology, Social Protection and Human Rights’ (2019) para 15 <https://www.ohchr.org/EN/Issues/Poverty/Pages/ DigitalTechnology.aspx> accessed 2 March 2021. 8 See for example: Payal Arora, ‘The Bottom of the Data Pyramid: Big Data and the Global South’ (2016) 10 International Journal of Communication 19. 9 Nayantara Ranganathan, ‘The Economy (and Regulatory Practice) That Biometrics Inspires: A Study of the Aadhaar Project’ in Amba Kak (ed), Regulating Biometrics: Global Approaches and Urgent Questions (2020) <https://ainowinstitute.org/regulatingbiometrics-ranganathan.html> accessed 24 July 2021. 10 Valery Gantchev, ‘Data Protection in the Age of Welfare Conditionality: Respect for Basic Rights or a Race to the Bottom?’ (2019) 21 European Journal of Social Security 3, 6. 11 Virginia Eubanks, Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor (St Martin’s Press 2018). 12 Eubanks (n 12). 13 AlgorithmWatch and Bertlesmann Stiftung, ‘Automating Society Report 2020’ (AlgorithmWatch 2020) 214–5. 14 Del Roy Fletcher and Sharon Wright, ‘A Hand up or a Slap down? Criminalising Benefit Claimants in Britain via Strategies of Surveillance, Sanctions and Deterrence’ (2018) 38 Critical Social Policy 323. 15 Melinda Cooper, ‘All in the Family Debt’ (Boston Review, 31 May 2017) <https://www.bostonreview.net/ articles/melinda-cooper-all-family-debt/> accessed 19 June 2023. 16 Kesia Reeve, ‘Welfare Conditionality, Benefit Sanctions and Homelessness in the UK: Ending the “something for Nothing Culture” or Punishing the Poor?’ (2017) 25 Journal of Poverty and Social Justice 65. 17 Howard Glennerster, ‘Crisis, Retrenchment and the Impact of Neo-Liberalism, 1976-97’ in Pete Alcock and others (eds), The Student’s Companion to Social Policy, vol Fifth edition (Wiley-Blackwell 2016). 18 Perri 6, Charles Raab and Christine Bellamy, ‘Joined-up Government and Privacy in the United Kingdom: Managing Tensions between Data Protection and Social Policy. Part I’ (2005) 83 Public Administration 111. 19 22 6, Raab and Bellamy (n 19). 20 Anna Kawakami and others, ‘Studying Up Public Sector AI: How Networks of Power Relations Shape Agency Decisions Around AI Design and Use’ (2024) 8 Proc. ACM Hum.-Comput. Interact. 450:1. 21 Sam Trendall, ‘“A Lack of Transparency and Accountability” – DWP Urged to Shed Light on Fraud Algorithm’ [2022] PublicTechnology.net <https://www.publictechnology.net/articles/features/lacktransparency-and-accountability-%E2%80%93-dwp-urged-shed-light-fraud-algorithm> accessed 31 October 2022. 22 Laura Carter, ‘Critical Analytics? Learning from the Early Adoption of Data Analytics for Local Authority Service Delivery’ (Ada Lovelace Institute 2024) <https://www.adalovelaceinstitute.org/ report/local-authority-data-analytics/> accessed 21 June 2024. 23 UN Special Rapporteur on Extreme Poverty and Human Rights (n 8) para 77. 24 Prime Minister’s Office, 10 Downing Street, ‘Prime Minister Sets out Blueprint to Turbocharge AI’ (GOV.UK, 13 January 2025) <https://www.gov.uk/government/news/prime-minister-sets-outblueprint-to-turbocharge-ai> accessed 25 June 2025. That same month, however, freedom of information requests revealed that multiple welfare AI tools had been dropped by government ministers: see Robert Booth, ‘AI Prototypes for UK Welfare System Dropped as Officials Lament “False Starts”’ The Guardian (27 January 2025) <https://www.theguardian.com/technology/2025/jan/27/aiprototypes-uk-welfare-system-dropped> accessed 26 February 2025. 25 General Comment No. 19: The right to social security (art. 9) para 24. 26 Amnesty International, ‘Denmark: Coded Injustice: Surveillance and Discrimination in Denmark’s Automated Welfare State’ (2024) 18 <https://www.amnesty.org/en/documents/eur18/8709/2024/en/> accessed 13 November 2024. 27 ILO Convention No. 168 (1988) on Employment Promotion and Protection against Unemployment 1988 (C168) art 20(e). 28 Thomas McBrien and others, ‘Screened & Scored in the District of Columbia’ (Electronic Privacy Information Center 2022). 29 Ragnar Lundström, ‘Framing Fraud: Discourse on Benefit Cheating in Sweden and the UK’ (2013) 28 European Journal of Communication 630. 30 Virginia Eubanks, ‘ITEM 10: How a Small Legal Aid Team Took on Algorithmic Black Boxing at Their State’s Employment Agency (And Won)’ (EPIC - Electronic Privacy Information Center, 12 January 2022) <https://epic.org/item-10/> accessed 3 January 2023. 31 Zajko (n 7) 250. 32 Emily M Bender and Alex Hanna, The AI Con: How to Fight Big Tech’s Hype and Create the Future We Want (HarperCollins 2025). 33 ‘DWP Commits £70m to Algorithms and Analytics to Tackle Benefit Fraud’ (PublicTechnology, 12 July 2023) <https://www.publictechnology.net/2023/07/12/society-and-welfare/dwp-commits-70m-toalgorithms-and-analytics-to-tackle-benefit-fraud/> accessed 12 July 2023. 34 See for example Eubanks (n 12); Simon Hansford, ‘Data Must Be Treated as a National Asset’ (PublicTechnology.net, 10 January 2020) <https://www.publictechnology.net/articles/opinion/datamust-be-treated-national-asset> accessed 15 January 2020. 35 23 Woodrow Hartzog, ‘The Inadequate, Invaluable Fair Information Practices’ (2017) 76 Maryland Law Review. 36 Rob Kitchin, Critical Data Studies: An A to Z Guide to Concepts and Methods (Polity 2025). 37 Omer Tene and Jules Polonetsky, ‘Big Data for All: Privacy and User Control in the Age of Analytics’ (2013) 11 Northwestern Journal of Technology and Intellectual Property. 38 Lilian Edwards and Michael Veale, ‘Slave to the Algorithm? Why a “right to an Explanation” Is Probably Not the Remedy You Are Looking For’ <https://osf.io/97upg> accessed 17 November 2021. 39 Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) 2016 art 5(1)(c). ‘Personal data’ in the GDPR means data relating to an identified person, or a person who can be identified directly or indirectly from the data (art 4(1)). 40 Amba Kak and Sarah Myers West, ‘AI Now 2023 Landscape: Confronting Tech Power’ (AI Now Institute 2023) <https://ainowinstitute.org/2023-landscape> accessed 12 April 2023. 41 Jennifer King and others, ‘The Privacy-Bias Tradeoff: Data Minimization and Racial Disparity Assessments in U.S. Government’, Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (Association for Computing Machinery 2023) <https://dl.acm.org/ doi/10.1145/3593013.3594015> accessed 20 February 2025. 42 Zara Rahman and Julia Keseru, ‘Predictive Analytics for Children: An Assessment of Ethical Considerations, Risks, and Benefits’ (UNICEF Office of Research 2021) <https://www.unicef-irc.org/ publications/1275-predictive-analytics-for-children-an-assessment-of-ethical-considerations-risksand-benefits.html>. 43 Gantchev (n 11). 44 McBrien and others (n 29). 45 Redden and others (n 48) 25. 46 ‘Report’ (Royal Commission into the Robodebt Scheme, 7 July 2023) 330 <https:// robodebt.royalcommission.gov.au/publications/report> accessed 10 July 2023. 47 Caroline Gans-Combe, ‘Automated Justice: Issues, Benefits and Risks in the Use of Artificial Intelligence and Its Algorithms in Access to Justice and Law Enforcement’ in Dónal O’Mathúna and Ron Iphofen (eds), Ethics, Integrity and Policymaking: The Value of the Case Study (Springer International Publishing 2022) <https://doi.org/10.1007/978-3-031-15746-2_14> accessed 23 June 2023. 48 Joanna Redden and others, ‘Automating Public Services: Learning from Cancelled Systems’ (Carnegie UK 2022) 34 <https://www.carnegieuktrust.org.uk/publications/automating-publicservices-learning-from-cancelled-systems/>. 49 Kate Crawford and Jason Schultz, ‘AI Systems as State Actors’ (2019) 119 Columbia Law Review; New York 1941, 1955–6. 50 Redden and others (n 48). 51 Robert Booth, ‘Revealed: Bias Found in AI System Used to Detect UK Benefits Fraud’ The Guardian (6 December 2024) <https://www.theguardian.com/society/2024/dec/06/revealed-bias-found-in-aisystem-used-to-detect-uk-benefits> accessed 6 December 2024. 52 24 Jon Henley and Robert Booth, ‘Welfare Surveillance System Violates Human Rights, Dutch Court Rules’ The Guardian (5 February 2020) <https://www.theguardian.com/technology/2020/feb/05/ welfare-surveillance-system-violates-human-rights-dutch-court-rules> accessed 7 February 2020. 53 Robert Booth, ‘DWP Algorithm Wrongly Flags 200,000 People for Possible Fraud and Error’ The Guardian (23 June 2024) <https://www.theguardian.com/society/article/2024/jun/23/dwp-algorithmwrongly-flags-200000-people-possible-fraud-error> accessed 1 October 2024. 54 Redden and others (n 48) 29–30. 55 These benefits included Income Support, Jobseekers Allowance, Housing Benefit, Child Tax Credit, and Working Tax Credit. See Jon Pareliussen, ‘Work Incentives and Universal Credit: Reform of the Benefit System in the United Kingdom’ (OECD Publishing 2013) <http://0search.ebscohost.com.serlib0.essex.ac.uk/login.aspx? direct=true&db=edsrep&AN=edsrep.p.oec.ecoaaa.1033.en&site=eds-live> accessed 3 June 2021. 56 Department for Work and Pensions, ‘Universal Credit Statistics, 29 April 2013 to 8 July 2021’ (GOV.UK, 17 August 2021) <https://www.gov.uk/government/statistics/universal-credit-statistics-29-april-2013to-8-july-2021/universal-credit-statistics-29-april-2013-to-8-july-2021> accessed 26 August 2021. 57 Mia Monkovic and Ben Ward, ‘Characteristics of Supplemental Nutrition Assistance Program Households: Fiscal Year 2023’ (US Department of Agriculture, Food and Nutrition Service, Evidence, Analysis, and Regulatory Affairs Office 2025) SNAP-23-CHAR 4 <https://www.fns.usda.gov/research/ snap/characteristics-fy23>. 58 Office of Family Assistance, ‘About TANF’ (Administration for Children & Families, 27 September 2024) <https://acf.gov/ofa/programs/tanf/about> accessed 8 September 2025. 59 Office of Family Assistance (n 59). 60 Office of Family Assistance (n 59). 61 Reeve (n 17). 62 Del Roy Fletcher, ‘Introduction to the Special Edition’ (2020) 54 Social Policy & Administration 185. 63 Tim Reeskens and Wim Van Oorschot, ‘Equity, Equality, or Need? A Study of Popular Preferences for Welfare Redistribution Principles across 24 European Countries’ (2013) 20 Journal of European Public Policy 1174. 64 ‘Universal Benefits Cost Less Than Means-Tested Benefits’ (People’s Policy Project, 11 November 2022) <https://www.peoplespolicyproject.org/2022/11/11/universal-benefits-cost-less-than-meanstested-benefits/> accessed 24 September 2024. 65 Fletcher and Wright (n 15). 66 Gantchev (n 11). 67 Emilio Sánchez Hidalgo, ‘Ditching Century-Old “Family Book,” Spain Adopts Individual Digital Records’ (EL PAÍS English, 5 May 2021) <https://english.elpais.com/society/2021-05-05/ditching-century-oldfamily-book-spain-adopts-individual-digital-records.html> accessed 8 September 2025. 68 The Nomad Today, ‘Rules for Receiving Unemployment Benefits in Spain and How to Apply’ (The Nomad Today, 4 November 2021) <https://www.thenomadtoday.com/articulo/work-in-spain/ requirements-to-receive-unemployment-benefits-in-spain/20211104110821014693.html> accessed 8 September 2025. 69 25