Review · 2004
Trust in Automation: Designing for Appropriate Reliance
Lee JD, See KA
What it found
People often rely on automation too much or too little. When a system is too complex to understand fully, trust is what guides how much people rely on it, so systems should be designed for trust that matches how good they actually are.
Where the evidence stops
A review across many fields, mostly industrial and safety systems.
How it shapes Pursue
Pursue
Pursue aims for trust that fits its real ability: it says when it is unsure instead of sounding certain.
A design input, not evidence that Pursue works.
Cited in
- 01Should I trust an AI's advice about how to spend my day?Trust it as far as it has earned, and no further. Across six experiments, people followed advice more readily when they believed it came from an algorithm rather than a person (Logg et al., 2019), yet watching an automated aid make mistakes made them distrust even a reliable one (Dzindolet et al., 2003).Trust and uncertainty
- 02How should an AI say it isn't sure?Plainly, and without inventing a confidence number. Modern neural networks are often over-confident (Guo et al., 2017), and a system allowed to decline uncertain cases makes far fewer errors on the ones it answers (Geifman and El-Yaniv, 2017); a 2019 review led from the University of Cambridge found that admitting uncertainty does not necessarily cost trust.Trust and uncertainty