IVA IDEAS & RESEARCH · THE FIVE-STAGE GOVERNANCE CYCLE · PART 2 OF 5

Enable: A Control No One Can Operate Is Not a Control

Why naming an overseer is not the same as having one — five conditions that make human oversight usable, and a check for each person you are relying on.

Iva Bruni · 2026 · Part 2 of 5 in the series The Five-Stage Governance Cycle

01 Design · 02 Enable · 03 Demonstrate · 04 Learn · 05 Redesign

Assigned oversight becomes real only when the people assigned have the authority, competence, information, capacity and time to act — and keep them as automation changes their work.

The overseer on the org chart

AI governance frameworks often require human oversight. Organizations may respond by assigning someone responsibility for it. That assignment is necessary, but it does not establish that the person can intervene effectively.

In a review of 41 policies requiring human oversight of government algorithms, Ben Green argued that such policies rest on an uninterrogated assumption that people are able to oversee algorithmic decisions effectively, and that the available evidence suggests people are unable to perform the oversight functions the policies expect (Green, 2022). His conclusion is that policymakers should shift from relying on human oversight to institutional oversight: agencies would have to justify both an algorithm and any human oversight with empirical evidence, and submit those justifications to democratic review before adoption.

Green’s remedy is aimed at public agencies. Organizations still need human oversight. But assigning it is only the beginning. They must give people the means to act, test whether intervention works (Part 3) and revisit the arrangement when circumstances change (Part 5).

Five conditions that make human oversight usable

Oversight depends on five conditions working together. If any one is missing, the organization may have an accountable person on paper without a practical means of intervention.

Condition	Question the organization must answer	What would demonstrate it works?
Authority	Can the person intervene, override, stop or escalate without seeking permission they do not have?	An intervention test confirms they can exercise that authority.
Competence	Can they recognize a problem, understand its implications and choose an appropriate response?	They identify and respond correctly to realistic scenarios.
Information	Can they see enough of what the system is doing to make an informed judgment?	They recognize an error or emerging risk using the information actually available during work.
Capacity	Do they have enough protected time and attention to perform oversight alongside their other responsibilities?	Review volumes and competing demands remain manageable under realistic workloads.
Time	Can they act before intervention becomes ineffective?	Tests show that detection, decision and action fit within the available intervention window.

These conditions are interdependent. Authority means little without information. Information is insufficient without capacity to review it. And even a competent, authorized person cannot intervene effectively if the system moves faster than they can respond.

The value of the five conditions is that they turn a vague requirement — “keep a human in the loop” — into something an organization can examine, test and improve.

Authority and competence

When auditors test an internal control, they ask whether it would meet its objectives if operated as prescribed “by persons possessing the necessary authority and competence,” and later whether the person performing it actually has that authority and competence (PCAOB, AS 2201, ¶.42–.44). The EU AI Act applies the same logic: deployers of high-risk systems must assign human oversight to people with the necessary competence, training and authority, and the necessary support (European Union, 2024, Art. 26(2)). For providers, Article 14(4) requires systems to be delivered so that overseers can understand the system’s capacities and limitations, remain aware of the tendency to over-rely on its output, interpret that output, disregard or override it, and intervene or stop the system. These are obligations for high-risk AI systems under the EU AI Act; following the 2026 amendments, they apply from 2 December 2027 for Annex III systems and from 2 August 2028 for Annex I systems. The series uses them as a reference standard for good practice; a system like the hotel’s would not necessarily be classified as high-risk.

Authority means the person can stop, override or change what the system does, within documented limits, without an approval process that makes intervention ineffective. Competence means they can recognize when the system is wrong in this particular work — something general AI training does not guarantee. NIST calls for personnel to receive AI risk-management training to perform their duties (NIST, n.d., GOVERN 2.2).

Information, capacity and time

A person cannot correct what they cannot see. At the moment of decision, the overseer needs the proposed action, the reason for it and the context it used. In one Singapore deployment case, a coding assistant requires approval by default for actions such as editing files, and explains complex commands in plain English before the person approves (IMDA, 2026).

Oversight also takes attention, which is finite. If an agent produces more items for review than a person can genuinely examine alongside other work, approval becomes a formality. McKinsey argues that the scale of agentic adoption will be capped by how much oversight capacity humans can provide (Sukharevsky et al., 2025) — a consultancy argument, not a measured finding, but it names a real design variable. In Singapore’s framework, one government team’s early lessons led to changes that reduced the cognitive load on human approvers (IMDA, 2026).

Finally, a person needs time to act between the system acting and the consequence landing, and the window an arrangement assumes is often longer than the one people actually have. In the Tempe crash described below, the NTSB estimated that an attentive operator would have had 2 to 4 seconds to respond; distracted, she had about 1 second (NTSB, 2019b, p. 43). In the Boeing 737 MAX safety assessment, pilots were assumed to “take immediate action” against uncommanded trim inputs, which were assumed to be “readily recognizable” (NTSB, 2019a, p. 7). After the Lion Air and Ethiopian Airlines accidents, the NTSB found that the crews’ responses “did not match the assumptions of pilot responses to unintended MCAS operation,” and that the assessment had not adequately accounted for the effect of multiple simultaneous alerts and indications (pp. 8–9). Research points the same way: in a study of 26 drivers using an automated driving feature, Eriksson and Stanton found that taking back control from automation took substantially longer than earlier studies had reported, and longer still when drivers were occupied with another task (Eriksson & Stanton, 2017). Bainbridge’s analysis explains part of the gap: the operator expected to take over in abnormal conditions may be out of practice and have little current knowledge of the process (Bainbridge, 1983). Where the realistic window is shorter than the decision needs, oversight has to move earlier (approval before action) or later (review and reversal afterward).

In practice: an operator who was assigned but not supported

In March 2018 a test vehicle operated by Uber’s Advanced Technologies Group (ATG) struck and killed a pedestrian in Tempe, Arizona. The automated driving system detected her 5.6 seconds before impact but never correctly classified her or predicted her path, and its design relied on the operator, not emergency braking, to avoid a collision (NTSB, 2019b, p. v). The National Transportation Safety Board found the probable cause to be the operator’s failure to monitor the road and the system because she was visually distracted by her personal cell phone. Contributing were ATG’s inadequate safety risk assessment procedures, its ineffective oversight of vehicle operators, and its lack of adequate mechanisms for addressing automation complacency (pp. v–vi). The Board also found that the operator’s licensing, experience and knowledge of the system were not factors (p. vi).

Read through the five conditions, the record shows where the arrangement was thin:

Condition	What the NTSB record shows
Authority	The operator could take control at any time; the system was designed to depend on her doing so (p. v)
Competence	Licensing, experience and knowledge of the system were not factors (p. vi)
Information	No alert was given when the system began a one-second suppression of braking in an emergency (p. 13)
Capacity	Months earlier, ATG had consolidated two operators’ roles into one, which increased the sole operator’s task demands and required her to look away from the road to use a tablet (pp. 29, 45)
Time	Attentive, she would have had 2 to 4 seconds to respond; distracted, she had about 1 second (p. 43)
Upkeep	ATG did not develop effective countermeasures to automation complacency, and supervisors rarely reviewed the in-vehicle video that could have shown whether operators were monitoring (p. 44)

The NTSB keeps two explanations in view, and so should the reader: the operator’s distraction is the probable cause, and the organizational conditions contributed. The five conditions help expose these weaknesses before something goes wrong. They show what people are expected to do — and whether the organization has made that possible. Here the gaps were not in the operator’s knowledge or authority but in her capacity and in the upkeep of her attention — the parts the organization controls. The Board is direct about the underlying difficulty: on monitoring an automation system for its failures, “humans are very poor at this task” (p. 44). After the crash, ATG returned to two operators and added real-time monitoring of operator attentiveness — itself a form of employee monitoring, to which the safeguards in Part 1 would apply — which the NTSB said begins to address the oversight deficiencies (p. 45). A person assigned to monitor automation is not, by itself, evidence that the oversight arrangement is adequate.

The case concerns a developmental automated driving system, not a generative AI agent, and is used here as an analogy for organizational oversight.

Illustrative Scenario: the hotel, continued

This scenario is hypothetical. It continues the example from Part 1.

Part 1’s design names two overseers. The front-desk agent oversees the agent’s service-recovery suggestions (F2) before the outcome. The Front Office Manager (FOM) oversees the agent’s billing corrections (F1) after the fact, and reviews the flags F3 raises about people’s compensation decisions — a different relationship, in which the AI helps review staff. The duty manager holds stop authority on every shift — and that ability to stop the agent must itself be usable and demonstrated, not merely assigned (Part 3 tests it).

Condition	Front-desk agent (overseeing F2 suggestions)	FOM (overseeing F1; reviewing F3 flags)
Authority	A written limit they can use without calling anyone, including to reject a suggestion; a route to the duty manager above it	Can reverse an eligible correction or initiate a compensating adjustment where necessary, change correction rules within the caps (cap changes need the GM), pause F1, and follow up flags only through the disclosed review process
Competence	Guidance and practice on when service recovery is warranted, and on when a suggestion looks wrong	Can tell a pattern of misuse from a busy weekend, and a faulty rule from a faulty decision
Information	The suggestion’s reasons, the guest’s stay history and prior compensation, visible at the desk	Flagged decisions with their context, and each correction with the record it was checked against
Capacity	Service recovery fits the shift’s other demands	Flag and sample volume reviewable in about three hours a week
Time	The decision is made while the guest is present	A weekly rhythm that catches patterns before they repeat

The Front Office Manager may be responsible for reviewing the flags. But if there are more than they can realistically examine each week, the responsibility exists only on paper.

Applying the conditions: the FOM’s weekly review

Read through the five conditions before go-live, the FOM’s role is complete on paper but thin in one place. The conditions locate the weakness and point to a change:

Step	In the hotel
Initial arrangement	The FOM is assigned weekly oversight of the agent’s billing corrections (F1) and the flags F3 raises, in about three hours a week.
Diagnosis	Authority, competence, information and time are in place. Capacity is not: nothing protects the three hours, and no one has checked how many flags and sampled corrections a busy week would produce.
Intervention	The review hours are protected in the rota, flag thresholds are set so that routine variation does not generate flags, and the weekly sample is sized to fit the protected time.
Test	Before go-live, the FOM reviews a week of prepared records containing known errors, at realistic volume. Part 3 sets out how this test is designed and judged (T2).
Evidence	A pass would show that the FOM can find and correct the errors at the tested volume. It would not show that the review holds in a peak week or after a vendor change; Part 4 follows what happens when the volume changes.

Enable ends with an arrangement that has been improved on paper. Whether it works is the next question.

Oversight changes jobs

Enable is also role design.

• Role redesign. Oversight is added to existing jobs. The front-desk agent gains authority and exposure — their decisions are now logged and reviewed. The FOM gains about three hours a week of review, which has to come from somewhere.
• Safe escalation and override. A right to reject a suggestion or stop the agent is usable only if people believe they can use it against a manager’s preference without penalty. That belief is shaped by how the first overrides are received.
• Adoption. In an HBR IdeaCast discussion, Tsedal Neeley argued that leaders should attend both to people’s willingness to adopt AI and to their confidence in using it, and should support learning and experimentation at team level (Bernstein, 2023). These are expert recommendations, not tested outcomes.

For this reason the Check asks for the overseer’s own view: does the person agree they can do this, and what would they change? In the hotel, the front-desk team asks for practice cases before go-live, while the FOM reinforces the need to keep review time protected as operations continue.

Enable does not end at launch

The conditions can be in place on day one and erode later. Bainbridge observed that skills deteriorate when not used, so an operator who mostly monitors may be out of practice when asked to take over (Bainbridge, 1983). In a flight-simulation study, Parasuraman, Molloy and Singh found that when monitoring was combined with other tasks, people detected automation failures substantially less well when the automation’s reliability was constant (unchanging) than when it varied (Parasuraman et al., 1993). Manzey, Reichenbach and Onnasch traced automation-bias errors to incomplete cross-checking and to contradictory information being discounted or not processed (Manzey et al., 2012).

These are laboratory findings, measured in sessions rather than years. They show that oversight capability can degrade under automation; they do not establish that AI use inevitably erodes expertise, which depends on the task, design, feedback and experience. The firm conclusion is narrower: do not assume that capability established at launch is still there a year later.

None of the available remedies is proven for AI oversight. Bainbridge recommended regular practice, including hands-on control and simulation of rare failures (Bainbridge, 1983). Japan’s AI Safety Institute suggests evaluating whether safety is maintained through internal training and exercises (Japan AISI, 2026). Rotating people between overseeing and doing the work, and reviewing flag volumes as the system changes, address competence and capacity directly. Which of these works in a given setting is a question for Parts 3 and 4.

The depth of evidence for each condition should scale with the consequences of failure: an overseer of low-risk, reversible actions needs less than one who can stop a system that decides about customers, money or employees.

Use it for every overseer you rely on

It records whether the conditions exist; only testing (Part 3) and operating evidence (Part 4) show whether oversight works.

What comes next

A condition that exists on paper is still an assumption. Part 3, Demonstrate, tests whether the hotel’s overseers can intervene, correct and stop under realistic conditions.