Whose Bias?
How does bias shape human and machine decisions — and how can we spot, question, and challenge it?
A pair of co-designed projects with the people who use these systems every day — and who are learning to question the systems we all rely on.
One question, two projects
“Whose Bias?” is an umbrella for work that shares a single conviction: the people most affected by biased systems should have authorship over how those systems are examined and described. Underneath it sit two sibling projects with different materials and different contributors, but closely related research questions — both shaped in collaboration with communities, and reaching across the GLAM and heritage sectors and the local communities those collections belong to.
Probably a Cat 🐱
A co-designed project with young people at SoapBox Islington, building a critical understanding of AI and machine learning through hands-on, experiential sessions — and culminating in a public event the young facilitators lead themselves.
Co-designMachine learningYouth voiceWays of Seeing 👁️
A sibling project examining archival bias in the Warburg Institute’s Image of the Black Archive, working with community contributors to evaluate and contextualise historical metadata. We are asking who described these images, and how. More broadly, it explores whether local LLMs can help transcribe and describe heritage collections at scale, without handing that authorship to a black box.
Local LLMsHeritage & GLAMShared authorshipProbably a Cat
Once upon a time, a pot of public engagement funding did what pots of funding occasionally, magically do: it brought together people who might otherwise never have shared a room. A software engineer with a (perfectly healthy, not at all contagious) Raspberry Pi obsession. A sleep-deprived PhD researcher with suspiciously abundant energy. Youth workers who could 3D-print a fidget toy, put together an image classifier, and run a session on machine learning in the same afternoon. And, most importantly, a group of young people who use AI every day but had never been asked what they thought about it.
The question they gathered around had already had a first outing: “Whose Bias?”, a hands-on workshop at the School of Advanced Study, where participants trained image-classification models on toys and snacks using Teachable Machine and Raspberry Pis. But some questions get better answers outside the university.
So they began again — with toys, snacks, a 2L carton of a certain brand of apple watermelon, and the question: can machine learning and AI be biased? Nearly a year on, we’re still asking.
Why young people? Why now?
of UK children aged 8–17 have already used AI tools.
OfcomYoung people are among those most affected by AI over their lifetimes — and the least represented in decisions about how it’s built, used, and regulated.
Alan Turing Institute, 2025We asked one of our young facilitators what Parliament would say if they stood up and argued AI shouldn’t be used in war. The answer: “Probably they wouldn’t have listened to us.” This project is one small answer to that gap.
Our journey
Youth work has been moving its practice toward the Lundy model of participation, which holds that young people’s involvement only counts when four things are present together: space to form a view, voice to express it, an audience that listens, and real influence on what happens next. Watch who’s leading as you scroll — that shift, from researchers to young people, is the whole point. From consultation, to co-design, to leadership.
Can a machine be biased? Same toys, different cameras: a Raspberry Pi and a Mac don’t see the same world.
Co-led with youth workersClose reading the heavyweights — McCarthy, Dartmouth, the UK Government — and finding the same flaw in all three: “intelligence” defined by “intelligence.” They didn’t pass the vibe test, so the young people wrote their own definition.
Researcher-led, youth-authoredNot a session on Jane Austen. No interfaces, no shortcuts: training MobileNetV2 in Python, epoch by epoch.
Youth worker ledA day in your life, with AI as a character — no magic allowed. Poems and prose: which one’s the machine? Gertrude Stein passed the vibe test; Hemingway took some punches.
Co-designed with young peopleWe didn’t even plan this one. An impromptu conversation on war, morality, and environmental cost. Whose “Oppenheimer moment” is this — and who gets a say?
The young people set the agendaSenate House, as collaborators not visitors: reading rooms, and reading the room, alongside a Pro Vice Chancellor.
Young people centre-stageThe stations are born — manifesto, terminal, Teachable Machine — with a three-role model: facilitate, roam, observe. Everyone picks their comfort zone, and their stretch.
Young facilitators decidedStill co-creating — including the website you’re reading right now.
EveryoneSoapBox opens its doors. They lead. We support. You’re invited.
ThemSo they wrote their own
During one of our sessions, we close-read definitions of AI from the 1950s all the way to the UK Government’s National AI Strategy, and noticed a recurring theme of circular reasoning. The main gap we identified was that intelligence is constantly defined by intelligence, which means we never actually get a solid baseline for what the machine is supposedly doing. We also discussed how terms like “learning” and “reasoning” are used as marketing ploys when it comes to LLMs. By using words that simulate human characteristics, developers are able to “big up” the technology, making it seem far more advanced, smarter, bigger, and human-like than it really is.
“Artificial Intelligence is a technological tool that gives you tailored digital assistance, directly or indirectly, via a knowledge base.”
— the young people’s working definition of AI, built by them, kept deliberately provisional.
What makes a partnership sustainable?
The thing we keep learning: sustainability isn’t a funding line. It’s about reciprocity, respect, care, opportunity, and figuring things out together. There are a few habits we’d defend to anyone.
Moving beyond advisory roles, youth workers and young people propose, design, and deliver the work — sessions, materials, and the event itself — that they create and lead on.
Everything is a braindump until the group says otherwise. Nothing arrives finished.
Code, writing, drawing, talking. People join where they’re comfortable.
Different institutional cultures, different calendars, different registers. The work is in creating hybrid models that hold the space between academic research and youth participation, rather than forcing one to fit the other.
The Lundy model of participation ensures young people voluntarily shape decisions, with meaningful opportunities to be heard, influence outcomes, and participate beyond tokenistic consultation.
Building trust and getting to know your collaborators takes time. Collaboration is slow, relational work that can’t be rushed to a deadline. The event takes a day.
Who brings what
SoapBox is an open-access STEAM youth centre for socially excluded young people aged 10–25 in Islington, built on the principle that every young person has “the right to be included, have a voice and shape the services they access.” The School of Advanced Study is the UK’s national centre for the support of humanities research. Neither of us could do this alone.
SoapBox brings
trusted relationships, youth work expertise, and technical ambition.SAS brings
research methods, critical frameworks, and institutional reach.The young people bring
imagination, curiosity, and lived, daily experience of the very systems we’re examining.Ways of Seeing
Probably a Cat has a sibling. Whose Bias? Ways of Seeing takes the same conviction into a different domain: archival bias in the Warburg Institute’s Image of the Black Archive. Working with community contributors, it evaluates and contextualises historical metadata: the words used to describe images, often written decades ago by people unlike those depicted. We are asking who described these images, and how.
Alongside that close, careful work, it asks a bigger question for the heritage and GLAM sectors: can local LLMs — run on our own machines, not someone else’s cloud — help transcribe and describe collections like this at scale, while keeping authorship with the communities those collections belong to? Where Probably a Cat asks who decides what AI sees, Ways of Seeing asks who decides what humans see — for example, what curators choose to make visible in an archive, how those holdings are catalogued, and what language is used to describe them — and who gets to narrate them now. Same principle, new materials. One partnership, growing beyond a single funding cycle.
The two projects share team members, a philosophical commitment to centring non-expert voices, and a conviction that lived experience is expertise — while keeping their own partners, participants, and purposes. More on Ways of Seeing soon.
References
The figures and frameworks we lean on, for anyone who wants to read further.
- Aitken, Mhairi, Morgan Briggs, and Shakir Mahomed. Understanding the Impacts of Generative AI Use on Children. London: The Alan Turing Institute, 2025. turing.ac.uk.
- Lundy, Laura. “‘Voice’ Is Not Enough: Conceptualising Article 12 of the United Nations Convention on the Rights of the Child.” British Educational Research Journal 33, no. 6 (2007): 927–42.
- Ofcom. Children and Parents: Media Use and Attitudes 2025. London: Ofcom, 2025. ofcom.org.uk.
Want to talk to us? 💌
Whether you’re a researcher, a youth organisation, a funder, or just curious about co-designing AI literacy with young people — we’d love to hear from you.