Lab experiments · 2003
The role of trust in automation reliance
Dzindolet MT, Peterson SA, Pomranky RA, Pierce LG, Beck HP
What it found
Before seeing it work, people expected an automated aid to do well, better than themselves. After seeing it make mistakes, they distrusted even a reliable aid, unless they were told why it might err. Knowing why kept trust and use up.
Where the evidence stops
Lab tasks with a decision aid; details of the samples are in the full paper, which we did not read.
How it shapes Pursue
Pursue
When Pursue is unsure, it says why, in plain words.
A design input, not evidence that Pursue works.
Cited in
- 01Should I trust an AI's advice about how to spend my day?Trust it as far as it has earned, and no further. Across six experiments, people followed advice more readily when they believed it came from an algorithm rather than a person (Logg et al., 2019), yet watching an automated aid make mistakes made them distrust even a reliable one (Dzindolet et al., 2003).Trust and uncertainty
- 02How should an AI say it isn't sure?Plainly, and without inventing a confidence number. Modern neural networks are often over-confident (Guo et al., 2017), and a system allowed to decline uncertain cases makes far fewer errors on the ones it answers (Geifman and El-Yaniv, 2017); a 2019 review led from the University of Cambridge found that admitting uncertainty does not necessarily cost trust.Trust and uncertainty