# Engagement is not benefit

> Why opening an app more often is not proof that it helps.

- Page: https://tre-up.com/research/topics/engagement-is-not-benefit
- Topic 03 of 7 in [Research Notes](https://tre-up.com/research) · Last reviewed 2026-10-07

Notifications reliably make people open an app. Whether they change what people actually do is a different question, and in a large 2026 trial they did not.

Most people stop opening wellbeing apps within weeks. For anyone judging such an app, the useful question is whether it led to something you did, not how often you opened it.

## Questions in this topic

- [Do reminder apps change what people do, or just how often they open the app?](https://tre-up.com/research/do-reminder-apps-change-behavior): Often only the second. In a six-month University of Michigan trial with 287 adults with high blood pressure (Golbus et al., 2026), notifications made people use the app more but did not raise steps in the next hour or improve food choices the next day. Opening an app is engagement, not benefit. Markdown: https://tre-up.com/research/do-reminder-apps-change-behavior.md

## What this topic rests on (8 sources)

- **Just-in-Time Adaptive Interventions (JITAIs) in Mobile Health: Key Components and Design Principles for Ongoing Health Behavior Support**. Nahum-Shani I, Smith SN, Spring BJ, Collins LM, Witkiewitz K, Tewari A, Murphy SA (2018). Annals of Behavioral Medicine 52(6):446–462. https://doi.org/10.1007/s12160-016-9830-8
  - Institutions: University of Michigan (United States); Northwestern University (United States); Pennsylvania State University (United States); University of New Mexico (United States)
  - Design: Design framework
  - What it found: Lays out how to build support that adapts to a person's changing state: decision points, the information that tailors each decision, the options available, and explicit rules that link them. A person's ability to take in help changes quickly, so the right amount of support at one moment can be too much at the next, and giving nothing is one of the designed options.
  - Where the evidence stops: A design framework built from health-behaviour research. It gives a vocabulary and principles, not a tested rule, threshold or schedule for any particular app or population.
  - Our summary: https://tre-up.com/research/sources/nahum-shani-2018-jitai
- **Just-in-Time Adaptive Interventions: Where Are We Now and What Is Next?** Nahum-Shani I, Murphy SA (2026). Annual Review of Psychology 77:679–703. https://doi.org/10.1146/annurev-psych-121024-044244
  - Institutions: University of Michigan (United States); Harvard University (United States)
  - Design: Review
  - What it found: Reviews where in-the-moment support stands and what is missing. Three problems stand out: people often cannot engage with support at the very moment they need it most, engagement with digital interventions is generally low, and almost no work uses the people around a person.
  - Where the evidence stops: A conceptual review. It sets out challenges and directions; it does not give thresholds or effect sizes.
  - Our summary: https://tre-up.com/research/sources/nahum-shani-murphy-2026-jitai-review
- **Real-world large-scale study on adaptive notification scheduling on smartphones**. Okoshi T, Tsubouchi K, Tokuda H (2018). Pervasive and Mobile Computing 50:1–24. https://doi.org/10.1016/j.pmcj.2018.07.005
  - Institutions: Keio University (Japan); Yahoo Japan Corporation (Japan)
  - Design: Large-scale deployment in a commercial app
  - What it found: Tests whether holding notifications until a sensed 'opportune moment' works inside a real product used by very large numbers of people, with changing contexts, personalised content and breaking news. Delivering at those moments made people respond faster and open more.
  - Where the evidence stops: The outcome is opening and clicking, which is engagement, not benefit to the person. Transfer from news notifications to personal guidance is an assumption.
  - Our summary: https://tre-up.com/research/sources/okoshi-2018-adaptive-scheduling
- **Efficacy of Contextually Tailored Suggestions for Physical Activity: A Micro-randomized Optimization Trial of HeartSteps**. Klasnja P, Smith S, Seewald NJ, Lee A, Hall K, Luers B, Hekler EB, Murphy SA (2019). Annals of Behavioral Medicine 53(6):573–582. https://doi.org/10.1093/abm/kay067
  - Institutions: University of Michigan (United States); University of California, San Diego (United States); Harvard University (United States)
  - Design: Micro-randomized trial; sample: 44 adults, 6 weeks
  - What it found: Activity suggestions were randomly sent or withheld several times a day. A suggestion raised steps in the next 30 minutes by about 14% on average, a large effect at first that faded over the six weeks. Suggestions to go for a walk worked; suggestions to break up sitting showed no detectable effect.
  - Where the evidence stops: A small trial about physical activity. Strong as a method for testing whether to step in; weak as a rule for other behaviours.
  - Our summary: https://tre-up.com/research/sources/klasnja-2019-heartsteps
- **To Prompt or Not to Prompt? A Microrandomized Trial of Time-Varying Push Notifications to Increase Proximal Engagement With a Mobile Health App**. Bidargaddi N, Almirall D, Murphy S, Nahum-Shani I, Kovalcik M, Pituch T, Maaieh H, Strecher V (2018). JMIR mHealth and uHealth 6(11):e10123. https://doi.org/10.2196/10123
  - Institutions: Flinders University (Australia); University of Michigan (United States); Harvard University (United States)
  - Design: Micro-randomized trial; sample: 1,255 users of a workplace well-being app, 89 days
  - What it found: At six possible times a day, users were randomly sent or not sent a tailored message. A message made people about 4% more likely to use the app in the next 24 hours. The time mattered: the effect was largest around 12:30 on weekends, at about 12%.
  - Where the evidence stops: The outcome is opening the app, not a change in behaviour or well-being. One commercial app, one population.
  - Our summary: https://tre-up.com/research/sources/bidargaddi-2018-to-prompt-or-not
- **Impact of Push Notifications on Physical Activity and Sodium Intake Among Patients with Hypertension: Microrandomized Trial of a Just-in-Time Adaptive Intervention**. Golbus JR, Dorsch MP, Chen Y, Basu T, Luff E, Klasnja P, Newman MW, Skolarus LE, Dempsey W, Nallamothu BK (2026). Journal of Medical Internet Research 28:e78218. https://doi.org/10.2196/78218
  - Institutions: University of Michigan (United States)
  - Design: Micro-randomized trial within a 6-month randomized trial; sample: 287 adults with self-reported hypertension; 187,517 randomization points
  - What it found: Tailored notifications about activity and salt were randomized at four times a day for six months, about one a day per person. They did not raise steps in the next hour or lower-salt food choices in the next day. They did make people use the app more.
  - Where the evidence stops: A secondary analysis; the engagement results were exploratory. An older clinical group, so it transfers poorly to healthy adults with work goals.
  - Our summary: https://tre-up.com/research/sources/golbus-2026-mybpmylife
- **How Notifications Affect Engagement With a Behavior Change App: Results From a Micro-Randomized Trial**. Bell L, Garnett C, Bao Y, Cheng Z, Qian T, Perski O, Potts HWW, Williamson E (2023). JMIR mHealth and uHealth 11:e38342. https://doi.org/10.2196/38342
  - Institutions: London School of Hygiene & Tropical Medicine (United Kingdom); University College London (United Kingdom); Harvard University (United States)
  - Design: Micro-randomized trial; sample: 350 new users of an alcohol-reduction app, first 30 days
  - What it found: Each evening new users were randomly sent the usual notification, a new message, or nothing. A notification made opening the app in the next hour about three and a half times more likely.
  - Where the evidence stops: One app about drinking. The lab's reading of the full trial is that the short-term lift did not change how soon people stopped using the app; the abstract we checked reports the one-hour effect.
  - Our summary: https://tre-up.com/research/sources/bell-2023-drink-less-notifications
- **Objective User Engagement With Mental Health Apps: Systematic Search and Panel-Based Usage Analysis**. Baumel A, Muench F, Edan S, Kane JM (2019). Journal of Medical Internet Research 21(9):e14567. https://doi.org/10.2196/14567
  - Institutions: University of Haifa (Israel); Partnership for Drug-Free Kids (United States); Northwell Health (United States)
  - Design: Usage analysis of real-world panel data; sample: 93 apps for anxiety, depression or well-being
  - What it found: Across popular mental-health apps, the median share of installed users who opened the app on a given day was 4%. Peer-support apps did best (17%), breathing-exercise apps worst (1.6%).
  - Where the evidence stops: Usage data only; says nothing about whether the apps helped the people who used them.
  - Our summary: https://tre-up.com/research/sources/baumel-2019-mental-health-app-engagement

## Other topics

- [Timing and receptivity](https://tre-up.com/research/topics/timing-and-receptivity): When help lands, and when the same help is just noise.
- [Interruptions and attention](https://tre-up.com/research/topics/interruptions-and-attention): What a ping, a switch or an unfinished task costs the mind.
- [Autonomy and wording](https://tre-up.com/research/topics/autonomy-and-wording): Why people push back on advice, and the words that make it easier to hear.
- [Trust and uncertainty](https://tre-up.com/research/topics/trust-and-uncertainty): How people come to rely on automated advice, and how a system should admit what it doesn't know.
- [Learning a person](https://tre-up.com/research/topics/learning-a-person): How a system can help someone it barely knows yet, and why fresh answers beat old patterns.
- [Privacy and deletion](https://tre-up.com/research/topics/privacy-and-deletion): What deleting your data really means, from the law to the backup tape.

---

© 2026 Tre-Up. You may read, link to, quote and summarise Research Notes, and search engines and AI assistants may index, summarise and cite them, as long as the source is credited with a link to the page on tre-up.com. Please do not present our notes as your own, or republish whole pages, without permission. Terms: https://tre-up.com/terms#use · About Tre-Up: https://tre-up.com/about