Amershi et al. (2019), Guidelines for Human-AI Interaction
Peer-reviewed interaction guidance for human-AI systems.
Learn more: Amershi et al. (2019), Guidelines for Human-AI InteractionApplied AI · Engineering practice
Human-in-the-loop vs exception-based AI
Human review is a system-design decision. It improves an AI workflow only when the review point is clear, useful and proportionate to the risk.
The real design question
A review-everything workflow can preserve control while recreating the original workload. A useful design identifies which decisions are reversible, which require evidence and which exceptions deserve a person’s attention.
Review patterns
Teams can review every output, review only outputs below a confidence threshold, or route exceptions defined by risk and business rules. The right pattern depends on consequence, reversibility, evidence quality and reviewer capacity.
Avoiding reviewer fatigue
Escalation should include the reason for review, the supporting evidence and the action available to the reviewer. Feedback and corrections should be captured as auditable events rather than hidden edits.
Applied lesson
AutonomousPlan2CAD explores a progression from assisted drafting toward bounded exception-based autonomy. Observations, inferred geometry, assumptions and human corrections remain distinguishable; full autonomy is not treated as a universal goal.
Practical checks
- Measure review volume and time, not only model confidence
- Define blocking conditions before deployment
- Keep a human accountable for irreversible decisions
- Test whether exceptions are genuinely rarer than the baseline workload
Limits and next step
A system that requires a person to recreate every result has not automated the decision. Start with a narrow, observable workflow and expand only when evidence shows that the review boundary is working.