# Trust and uncertainty

> How people come to rely on automated advice, and how a system should admit what it doesn't know.

- Page: https://tre-up.com/research/topics/trust-and-uncertainty
- Topic 05 of 7 in [Research Notes](https://tre-up.com/research) · Last reviewed 2026-10-07

People rely on automation too much or too little, and often follow an algorithm's advice more readily than a person's. One visible mistake can then make them distrust even a reliable system, unless they know why it might err.

Machine-learning research shows systems are often over-confident, and that declining uncertain cases makes the remaining answers more reliable. The goal is trust that matches what a system can really do.

## Questions in this topic

- [Should I trust an AI's advice about how to spend my day?](https://tre-up.com/research/should-i-trust-an-ai-with-advice-about-my-day): Trust it as far as it has earned, and no further. Across six experiments, people followed advice more readily when they believed it came from an algorithm rather than a person (Logg et al., 2019), yet watching an automated aid make mistakes made them distrust even a reliable one (Dzindolet et al., 2003). Markdown: https://tre-up.com/research/should-i-trust-an-ai-with-advice-about-my-day.md
- [How should an AI say it isn't sure?](https://tre-up.com/research/how-should-an-ai-say-it-isnt-sure): Plainly, and without inventing a confidence number. Modern neural networks are often over-confident (Guo et al., 2017), and a system allowed to decline uncertain cases makes far fewer errors on the ones it answers (Geifman and El-Yaniv, 2017); a 2019 review led from the University of Cambridge found that admitting uncertainty does not necessarily cost trust. Markdown: https://tre-up.com/research/how-should-an-ai-say-it-isnt-sure.md

## What this topic rests on (10 sources)

- **Principles of mixed-initiative user interfaces**. Horvitz E (1999). Proceedings of CHI 1999, 159–166. https://doi.org/10.1145/302979.303030
  - Institutions: Microsoft Research (United States)
  - Design: Design principles
  - What it found: Sets out principles for combining automated help with the person staying in control, illustrated with an assistant that offers to schedule meetings from email: act when the expected value is high, ask when unsure, and make it cheap to dismiss or correct.
  - Where the evidence stops: Principles from experience with one system, not an experiment.
  - Our summary: https://tre-up.com/research/sources/horvitz-1999-mixed-initiative
- **Trust in Automation: Designing for Appropriate Reliance**. Lee JD, See KA (2004). Human Factors 46(1):50–80. https://doi.org/10.1518/hfes.46.1.50_30392
  - Institutions: University of Iowa (United States)
  - Design: Review
  - What it found: People often rely on automation too much or too little. When a system is too complex to understand fully, trust is what guides how much people rely on it, so systems should be designed for trust that matches how good they actually are.
  - Where the evidence stops: A review across many fields, mostly industrial and safety systems.
  - Our summary: https://tre-up.com/research/sources/lee-see-2004-trust-in-automation
- **Humans and Automation: Use, Misuse, Disuse, Abuse**. Parasuraman R, Riley V (1997). Human Factors 39(2):230–253. https://doi.org/10.1518/001872097778543886
  - Institutions: Catholic University of America (United States); Honeywell Technology Center (United States)
  - Design: Review
  - What it found: Names the ways people get automation wrong: misuse (relying on it too much), disuse (ignoring it, often after false alarms) and abuse (automating without regard for the people who have to work with it).
  - Where the evidence stops: A conceptual review drawn largely from aviation and industrial settings.
  - Our summary: https://tre-up.com/research/sources/parasuraman-riley-1997-use-misuse
- **The role of trust in automation reliance**. Dzindolet MT, Peterson SA, Pomranky RA, Pierce LG, Beck HP (2003). International Journal of Human-Computer Studies 58(6):697–718. https://doi.org/10.1016/S1071-5819(03)00038-7
  - Institutions: Affiliation not yet confirmed
  - Design: Lab experiments
  - What it found: Before seeing it work, people expected an automated aid to do well, better than themselves. After seeing it make mistakes, they distrusted even a reliable aid, unless they were told why it might err. Knowing why kept trust and use up.
  - Where the evidence stops: Lab tasks with a decision aid; details of the samples are in the full paper, which we did not read.
  - Our summary: https://tre-up.com/research/sources/dzindolet-2003-trust-reliance
- **Algorithm appreciation: People prefer algorithmic to human judgment**. Logg JM, Minson JA, Moore DA (2019). Organizational Behavior and Human Decision Processes 151:90–103. https://doi.org/10.1016/j.obhdp.2018.12.005
  - Institutions: Harvard University (United States); University of California, Berkeley (United States)
  - Design: Six experiments
  - What it found: Across six experiments, ordinary people followed advice more when they believed it came from an algorithm than from a person. The effect weakened when they had to choose between the algorithm and their own judgment, and among experienced forecasters.
  - Where the evidence stops: Estimation and forecasting tasks, not personal decisions about one's own day.
  - Our summary: https://tre-up.com/research/sources/logg-2019-algorithm-appreciation
- **To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making**. Buçinca Z, Malaya MB, Gajos KZ (2021). Proceedings of the ACM on Human-Computer Interaction 5(CSCW1). https://doi.org/10.1145/3449287
  - Institutions: Harvard University (United States)
  - Design: Online experiment; sample: 199 participants
  - What it found: Designs that made people think before seeing the AI's answer reduced over-reliance on the AI more than simple explanations did. People liked those designs least.
  - Where the evidence stops: One decision task online. The most effective designs were the least popular, a real trade-off.
  - Our summary: https://tre-up.com/research/sources/bucinca-2021-cognitive-forcing
- **How Much Information? Effects of Transparency on Trust in an Algorithmic Interface**. Kizilcec RF (2016). Proceedings of CHI 2016, 2390–2395. https://doi.org/10.1145/2858036.2858402
  - Institutions: Stanford University (United States)
  - Design: Online field experiment
  - What it found: When an algorithm's result was worse than people expected, explaining how it worked kept their trust, but explaining too much eroded it again. People whose expectations were met were not affected.
  - Where the evidence stops: Peer grading in online courses; one setting.
  - Our summary: https://tre-up.com/research/sources/kizilcec-2016-transparency-trust
- **Communicating uncertainty about facts, numbers and science**. van der Bles AM, van der Linden S, Freeman ALJ, Mitchell J, Galvao AB, Zaval L, Spiegelhalter DJ (2019). Royal Society Open Science 6(5):181870. https://doi.org/10.1098/rsos.181870
  - Institutions: University of Cambridge (United Kingdom); University of Warwick (United Kingdom); Columbia University (United States)
  - Design: Interdisciplinary review
  - What it found: Offers a framework for saying how sure we are about facts, numbers and science. The limited evidence suggests that admitting uncertainty does not necessarily backfire on trust, though the effect varies between people and formats.
  - Where the evidence stops: A review of a thin literature; it calls for more experiments.
  - Our summary: https://tre-up.com/research/sources/van-der-bles-2019-communicating-uncertainty
- **On Calibration of Modern Neural Networks**. Guo C, Pleiss G, Sun Y, Weinberger KQ (2017). Proceedings of ICML 2017, PMLR 70:1321–1330. https://arxiv.org/abs/1706.04599
  - Institutions: Cornell University (United States)
  - Design: Machine-learning experiments
  - What it found: Modern neural networks are often over-confident: the confidence they report does not match how often they are right. A simple adjustment after training, temperature scaling, fixed much of this on most datasets tested.
  - Where the evidence stops: Image and text classification benchmarks, not advice to people.
  - Our summary: https://tre-up.com/research/sources/guo-2017-calibration
- **Selective Classification for Deep Neural Networks**. Geifman Y, El-Yaniv R (2017). Advances in Neural Information Processing Systems 30 (NeurIPS 2017). https://arxiv.org/abs/1705.08500
  - Institutions: Technion – Israel Institute of Technology (Israel)
  - Design: Machine-learning method and experiments
  - What it found: A trained network can be given the option to decline when it is unsure, set so that its error rate stays under a chosen level. Answering fewer cases made the answers it did give far more reliable.
  - Where the evidence stops: Image classification benchmarks; the mapping to personal advice is an analogy.
  - Our summary: https://tre-up.com/research/sources/geifman-el-yaniv-2017-selective-classification

## Other topics

- [Timing and receptivity](https://tre-up.com/research/topics/timing-and-receptivity): When help lands, and when the same help is just noise.
- [Interruptions and attention](https://tre-up.com/research/topics/interruptions-and-attention): What a ping, a switch or an unfinished task costs the mind.
- [Engagement is not benefit](https://tre-up.com/research/topics/engagement-is-not-benefit): Why opening an app more often is not proof that it helps.
- [Autonomy and wording](https://tre-up.com/research/topics/autonomy-and-wording): Why people push back on advice, and the words that make it easier to hear.
- [Learning a person](https://tre-up.com/research/topics/learning-a-person): How a system can help someone it barely knows yet, and why fresh answers beat old patterns.
- [Privacy and deletion](https://tre-up.com/research/topics/privacy-and-deletion): What deleting your data really means, from the law to the backup tape.

---

© 2026 Tre-Up. You may read, link to, quote and summarise Research Notes, and search engines and AI assistants may index, summarise and cite them, as long as the source is credited with a link to the page on tre-up.com. Please do not present our notes as your own, or republish whole pages, without permission. Terms: https://tre-up.com/terms#use · About Tre-Up: https://tre-up.com/about