Who is responsible when a decision is automated?

The place Yolanda did not get

Imagine a city with fewer funded training places than eligible applicants. It hires a company to rank people by their predicted chance of finishing the course. Yolanda is refused a place. Her attendance used to be irregular because the evening bus stopped running; the new course provides transport, but the ranking uses her old record.

The clerk says the system made the decision. The company says it supplied a score. The council says its aim is to fund places that people will use. Yolanda wants to know who has the power to reconsider.

Automation means handing work to a system that carries it out with less direct human intervention. The previous lesson asked whose claims a decision must consider. Here the decision has been handed to such a system, and we ask what becomes of the responsibility for it.

A prediction is not a permission

Suppose the ranking predicts who will finish the course more accurately than the city’s former method did. That is evidence about how well it serves a chosen aim. It does not show that maximizing completions is the right way to distribute these places, or that the council may exclude Yolanda without hearing her.

The lesson on institutions distinguished resources from capabilities, what people can actually do. Yolanda’s old record may describe an obstacle that the new transport removes. And even a sound prediction leaves another question open: should people facing greater obstacles receive less support, or more?

Reuben Binns proposes public reason as a standard for algorithmic accountability, that is, answerability for how a system is built and used. On this view, institutions should justify their decisions through principles that the people subject to them could reasonably accept. Providing an explanation is only the beginning, since its adequacy can be contested, and what counts as a shared, acceptable reason remains disputed within his proposal. [3]

The council must therefore defend two things: its allocation rule, and its authority to apply that rule here. Neither a useful score nor a resident’s objection settles either question on its own.

What is hidden, and from whom?

Opacity is difficulty in understanding how a system reaches its output, and the difficulty can differ from one person to another. In one version of our case, the company withholds information. In another, it discloses a calculation that Yolanda and the clerk lack the training to follow. In a third, even tracing the calculation does not yield an intelligible explanation of this ranking.

These are different obstacles, and disclosure might remove the secrecy while leaving the other problems untouched. An explanation of what produced a score also differs from a reason for letting that score govern someone’s opportunities: philosophical discussion distinguishes explaining a process from justifying its use. [1] [3]

Suppose everyone understands a rule that ranks applicants from distant neighborhoods lower. The rule is now visible, but its fairness still requires an argument.

When responsibility seems to have no owner

A responsibility gap would arise if a harmful outcome occurred but neither the people involved nor the machine met the conditions for moral responsibility, perhaps because a lack of knowledge or control stood in the way of blame. Whether automated systems create such gaps, and how far existing responsibilities answer the problem, remains disputed. [1]

A gap in justified blame is not the same as leaving duties unassigned. Someone may still owe an investigation, an explanation or a repair. Suppose nobody could reasonably have anticipated one error: the council can still owe Yolanda a review of her application. Conversely, “I did not predict this particular refusal” is a weak excuse from an official who ignored clear warnings that applicants had no way to correct their records.

Putting a person at the end of a process is not enough. A clerk who cannot inspect the reasons, change the result or obtain help may merely pass the result along, and calling that arrangement human oversight does not give the clerk control.

Many hands, specific responsibilities

Helen Nissenbaum’s 1996 argument about accountability examines the problem of many hands: when many people contribute to a piece of work, it can become unclear who must answer for the outcome. She also warns against making the computer a scapegoat, so that identifying its causal role ends any inquiry into human decisions. Accountability is answerability, which is not the same as blame or as liability to pay compensation. [2]

Return to the city. The designers should answer for what their model measures and for the limits they could reasonably be expected to identify. Those who deploy it should answer for how well it fits the new course. The council must defend the allocation policy and provide a review that can change a decision. And the clerk should carry Yolanda’s objection into that review instead of presenting the score as beyond question.

These are proposed responsibilities for this case, not a verdict that everyone is equally at fault: who knew what, who had which powers and who neglected which duties all matter. Shared work calls for clear duties and for someone to check that they are fulfilled. Yolanda needs an answer, not another office to visit.

Whose autonomy is at stake?

Technical autonomy concerns how far a system operates without direct human control. That alone does not show that the system can understand moral reasons and bear responsibility. Nor does it establish conscious experience, the felt character of seeing, hearing or hurting discussed in the consciousness lesson. Whether a machine could have experience is a separate question from whether the city may use its ranking. [1]

Yolanda’s autonomy is at stake in another sense: her ability to shape her life and act on reasons. Being able to challenge the refusal lets her take part in a decision that shapes her opportunities. Epistemic injustice, a wrong done to someone as a knower, points to a further risk. If officials discount her account because they regard applicants like her as unreliable, the review can repeat the very wrong it was meant to uncover.

Yolanda can be asked for honest information that she is reasonably able to provide. She should not have to work out how hidden software operates before her objection counts. Affected people can help test a system’s assumptions without becoming responsible for failures they did not control.

Try finding the missing responsibility

The council publishes the ranking formula and asks a clerk to approve every refusal, but the clerk has no power to change any result. A councilor says, “Now it is transparent and a human approves it, responsibility is settled.” Identify two remaining problems and propose a remedy for each. Must your answer assume that the system is conscious? This exercise is optional.

Read a possible response

Publishing the formula may reveal how scores are calculated, but it does not justify choosing likely completion as the allocation rule, or show that past attendance predicts what Yolanda can do now that transport is provided. The council should explain its aim and its grounds for using this information, and a review should examine her changed circumstances.

The clerk’s approval adds a human signature without adding any power to correct the decision. A remedy would give a reviewer the information, time and authority to reconsider it, with a clear route from Yolanda’s objection to an answer, and designers and deployers should supply the relevant evidence. That arrangement can make responsibilities clearer, though it does not prove who deserves blame for the original refusal.

Neither remedy depends on machine consciousness, because the questions concern human and institutional choices about using an output. Yolanda’s objection deserves consideration, but it does not automatically establish that she should receive a place: other applicants also have claims, and the allocation still needs justification.

Carry this forward

Follow the decision beyond its output. Ask who chose the purpose, who can explain the grounds, who can change the result, and to whom they must answer. The next lesson asks whose capacities set the terms of moral standing: does being owed consideration depend on being able to argue for it?

Sources and limits

[1] Vincent C. Müller, “Ethics of Artificial Intelligence and Robotics”, Stanford Encyclopedia of Philosophy, revised 27 March 2026, for opacity, technical autonomy and the disputed responsibility gap. The current page was checked on 20 September 2026. Read §§2.3, 2.6.1 and 2.7.4 in full; the section sign (§) identifies a numbered section. Cited studies, legal claims and technical forecasts were not independently assessed or adopted here.

[2] Helen Nissenbaum, “Accountability in a Computerized Society”, first published in Science and Engineering Ethics 2.1 (1996), pp. 25–42, for the problem of many hands and the warning against scapegoating the computer. Read pp. 1–5, 10–11 and 13–14 of the author’s manuscript, including its supplied notes; pp. 4 and 10 were also inspected as images. These page numbers belong to the manuscript, not the journal. The manuscript’s note numbering has mismatches, but no argument here relies on identifying a cited work from those notes. No complete reading of the essay or independent check of its historical cases is claimed.

[3] Reuben Binns, “Algorithmic Accountability and Public Reason”, Philosophy & Technology 31 (2018), pp. 543–556, for public reason as a standard of accountability; first published online 24 May 2017. Read the publisher’s complete main text, §§1–5, and all nine notes; the most relevant passages are §§2.3, 3.1–3.2 and 4.2. Its legal and technical examples belong to their historical setting and are not current findings established by this lesson. Works cited in the article were not independently read.

Sources consulted in English on 20 September 2026. Yolanda, the city, the allocation system and all variants are invented, and the proposed responsibilities and comparisons are this course’s. No real system’s accuracy, effects, consciousness or legal obligations are asserted. Earlier lessons supplied context without fresh reading of their underlying sources. No independent scholarly review or learner testing is claimed.