An AI labor map should change the questions you ask a vendor

The people outside the product screenshot

Brown University highlighted Cunyan Ma's work on mapping AI data labor in an October 26 news story. The underlying project, described by Brown's Center for Technological Responsibility, Reimagination and Redesign in August, connects worker locations, data-labeling platforms, and customer companies.

The project is a useful prompt to widen the frame around an AI tool. An interface shows you what a service returns. It usually does not explain all the people and organizational relationships involved in developing, evaluating, or maintaining that service. Looking for those relationships is a different kind of AI literacy from comparing model scores.

That does not mean a map can settle every labor question. A connection between organizations is not, by itself, proof of a particular worker's pay, contract, experience, or current assignment. The project's own description directs readers to its methodology and limitations. A responsible reader should preserve that distinction when sharing a screenshot or discussing a company.

Our practical contribution here is a proposed method for turning curiosity into better questions. It is not a ranking of vendors, an investigation of any named company's current workforce, or a claim that we interviewed workers. You can use it in a classroom, a student organization, or a small team's software review.

Start with a simple change in perspective. Alongside what can this tool do for us, ask what work had to happen for the tool to exist, who can describe that work, and what evidence we have about its conditions. None of those questions requires pretending the answer is already known.

The important move is from an abstract ethical position to a concrete inquiry. If you want to support fairer technology, you need a way to identify missing information, judge the scope of available evidence, and decide what action follows. A map can start that process without being asked to finish it.

Source: Brown University: Brown student maps the hidden global labor powering AI ↗ · Brown CNTR, Cunyan Ma: Where are the data workers behind AI? Project description ↗ · Rothschild and colleagues: The Problems with Proxies—Making Data Work Visible through Requester Practices ↗

Build a relationship sketch, then mark what you do not know

Choose a single AI application your group is considering. On a page, write the service name and the functions you expect to use. Then sketch the relationships you can actually document from public primary material: a model provider, a software vendor, a data supplier, or an evaluation partner, where those are disclosed.

Use different marks for a documented relationship, a reasonable question, and an unknown. Do not draw an inferred subcontractor as though it were a verified fact. If a connection comes from an older source, record the source date and avoid assuming it describes the present arrangement.

For a fictional example, imagine a campus organization choosing a transcription service. It knows the app vendor and finds a public statement about a model provider. It does not know whether humans review selected recordings or how any review work is organized. The honest sketch contains two named nodes and a clearly labeled unknown, not a completed supply chain invented to make the diagram satisfying.

Next, attach one question to that unknown. Could a person access submitted recordings during quality review? If so, under what process, and what public information describes the conditions of that work? Notice that this question combines a concern about the people in the recording with a concern about the people doing the review.

You are not required to solve an entire industry to make the exercise useful. A relationship sketch can be a small evidence index that helps a group see which assumptions have been doing the work in its conversation.

Keep the sketch dated and revisable. If new information contradicts it, correct the relationship and preserve the earlier source in your notes. The purpose is an honest inquiry that can improve, not a permanent accusation expressed through lines and arrows.

Ask about the design of the work, not only the headline rate

A 2024 study by Annabel Rothschild and colleagues examined practices among 52 data-work requesters. The authors argue that assumptions about workers and ad hoc qualification practices can discount workers' expertise. That is a bounded research finding, not a survey of every platform or every person working on AI.

One practical implication we draw is to ask how work is designed. A posted rate is useful information, but it does not by itself explain qualification time, instructions, rework, rejected tasks, or opportunities to challenge a decision. These are questions for investigation, not conditions we are asserting about a particular vendor.

Imagine a fictional labeling assignment asking workers to decide whether a short comment is sarcastic. The instructions provide two examples and no way to flag missing context. If workers disagree, a requester might blame carelessness. Another possibility is that the task fails to capture the context required for a defensible judgment.

A better review would ask who wrote the instructions, whether workers can point out ambiguous cases, and how the requester responds when knowledgeable contributors disagree. The goal is to learn whether the process treats human judgment as expertise or merely as a source of interchangeable clicks.

For a group reviewing a vendor, draft questions about paid preparation, clarity of task expectations, review and appeal processes, and whether workers can raise concerns without losing access to work. Ask the vendor to distinguish its own employees from contractors and subcontracted arrangements where relevant.

Do not convert an incomplete reply into a numerical ethical score with false precision. Record what the reply establishes, what it does not establish, and whether it changes your next step. A carefully described uncertainty is more informative than a green badge based on a vague assurance.

Transparency should not require exposing a worker

There is a tension worth keeping visible: making a system easier to examine should not mean making individual workers easier to identify or contact without permission. A public-interest question about an organization does not automatically justify publishing personal details about the people doing its work.

For a classroom exercise, use fictional names and organizational relationships. If you investigate a real tool, prefer published institutional information and research that explains how its evidence was obtained. Do not compile workers' private profiles to fill gaps in a diagram, and do not treat a person's social account as permission to interrogate them about an employer.

A useful evidence request can focus on process. Ask for an explanation of task review, worker feedback channels, compensation policies, or independent assessment where available. Those requests can be specific without demanding an individual worker's identity or disclosing confidential assignments.

Suppose a vendor says it cannot name a particular contractor for confidentiality reasons. That does not tell you whether the labor conditions are good or bad. A follow-up could ask what non-identifying information it can share about oversight, who is accountable for conditions, and how concerns are addressed. The quality of the response matters more than how complete your network picture looks.

Likewise, avoid sharing a map screenshot as though every node represents a person whose experience you understand. A node can represent an aggregation, a location, or an organization. Explain what the underlying project says it represents before attaching a moral conclusion to it.

This is our proposed standard for responsible curiosity: seek accountability at the level where decisions are made while being careful about the people with the least control over disclosure. Transparency becomes more useful when it improves understanding without turning workers into props in someone else's technology debate.

A 45-minute exercise that ends with something usable

Try this with three people and one publicly documented AI service. Spend the first ten minutes separately finding primary information about the service's data and evaluation practices. Save the exact links and dates. Do not use a chatbot's confident summary as a substitute for reading the material it points to.

Spend the next ten minutes building a shared relationship sketch. Allow blank spaces. One person should ask whether each connection is documented, another whether the information is current, and the third whether the diagram makes an unsupported implication. Rotate those roles if the conversation becomes one-sided.

Use the next fifteen minutes to choose three unanswered questions. Make one about the nature of human work, one about the way concerns are heard, and one about the evidence behind the vendor's own statements. Keep questions narrow enough that a useful answer would be recognizable.

For example, a hypothetical question about human review could ask which categories of submitted material may be reviewed and whether that process is described in current public documentation. A question about conditions could ask what standards apply across contractors and subcontractors. A question about evidence could ask whether an assessment describes its scope and date.

Use the final ten minutes to write a short decision note. It should say what your group learned, what remains unknown, what action is appropriate now, and when the note should be revisited. Perhaps the outcome is simply to ask for information before a pilot. Perhaps it is to choose a lower-data exercise while you investigate.

These timings are suggested workshop boundaries, not research-derived optimal intervals. Their purpose is to prevent the session from becoming an endless discussion. The deliverable is a traceable set of questions and a proportionate decision, which another group can read and improve.

Connect worker knowledge to the quality question

It is tempting to split AI evaluation into technical performance on one side and labor conditions on the other. For a practical inquiry, consider where those questions meet. A person deciding labels may know that a task is ambiguous, a cultural reference is missing, or an instruction conflicts with another instruction.

In our fictional sarcasm example, disagreement could contain information about the task rather than merely noise to remove. A review process that allows explanation might reveal that two comments need context from a longer conversation. That possibility should make you ask how the system captures uncertainty, not assume one label is automatically correct.

The same reasoning applies to a hypothetical document-classification project. If a contributor notices that two categories overlap, the requester can revise the categories, allow multiple labels, or document a judgment rule. Each option makes a different product. The person performing the work may be the first to notice the choice exists.

This is an argument for listening to expertise, not a claim that every disagreement improves a dataset or every feedback mechanism works. The quality of the process still needs evidence. Ask what happens after feedback is submitted, whether instructions change, and whether contributors receive explanations about disputed judgments.

For a small team building its own dataset, start with a paid pilot whose purpose includes improving the instructions, rather than treating all preparation as an invisible prerequisite. Define that as your proposed project practice, with a scope and budget you can actually sustain.

Then document which ambiguities changed the design. That creates a record of human contribution beyond a final count of labels. It can also help the next reviewer understand why a dataset's categories look the way they do, instead of assuming they arrived fully formed from the model.

Use the map to sharpen responsibility

A useful AI labor conversation should leave you with a clearer account of who can answer which question. An app vendor may describe its service practices. A model developer may describe a training or evaluation process. A researcher may explain the scope and limits of a public dataset or map. Those are different forms of knowledge.

Do not expect one source to certify the entire chain. If a university news story introduces a project, read the project's own description before drawing detailed conclusions. If a vendor publishes a policy, distinguish the policy from evidence about its implementation. If a study describes a sample, preserve that sample's boundaries when applying its lessons.

For a real decision, write down the responsibility your own group can accept. You may be able to choose a service, limit what data you submit, ask questions, support an independent inquiry, or improve how you commission work. You may not be able to verify every condition in a complex supply chain. Admitting that limit is compatible with taking a concrete step.

Avoid two easy endings. One is to declare the whole system unknowable and stop asking. The other is to mistake a satisfying diagram for a completed moral judgment. A better ending is a question assigned to an owner, supported by links, and tied to a decision that can be revisited.

The Brown project matters here as an invitation to look beyond the product surface. Our suggested response is practical: make relationships visible where evidence supports them, protect the limits of what you know, and ask about the design and conditions of the work.

The next time a tool promises to save your team an hour, you can still ask how well it performs. Add another question beside it: whose work helped make that hour possible, and what evidence would help us understand their part?