Published on in Vol 5, No 3 (2022): Jul-Sep

Preprints (earlier versions) of this paper are available at, first published .
A Gender Lens on User Quality Ratings From Young Teenagers Assessing the Sun Safe App: Comparing Responses From Co-researchers and Participants of Pilot Intervention Studies

A Gender Lens on User Quality Ratings From Young Teenagers Assessing the Sun Safe App: Comparing Responses From Co-researchers and Participants of Pilot Intervention Studies

A Gender Lens on User Quality Ratings From Young Teenagers Assessing the Sun Safe App: Comparing Responses From Co-researchers and Participants of Pilot Intervention Studies

Research Letter

Corresponding Author:

Shelley Gorman, BSc, PhD

Telethon Kids Institute

University of Western Australia

Level 7, Perth Children's Hospital, 15 Hospital Drive, Nedlands

Perth, 6052


Phone: 61 8 6319 1747

Fax:61 8 6319 1777


We developed the iOS smartphone app Sun Safe to support healthy sun practices in young teenagers (aged 12-13 years) [1]. The production involved co-design with young co-researchers (ie, aged 12-13 years) with a health message of using sun protection when the UV index is ≥3 [1]. Important features include real-time and location-specific weather data on the UV index and gamified educational content [1,2].

We were concerned that indifferent attitudes expressed by male co-researchers during the development of Sun Safe [3] would translate into gendered differences in user quality ratings. Furthermore, we wondered whether involvement in the co-design process could bias quality assessments. The results presented in this letter compare the responses of co-researchers [1] with those of participants of the pilot intervention studies [4].

All methods underpinning the development of the app and pilot intervention studies are described elsewhere [1,4]. Data were collected from co-researchers (n=15, 9 female and 6 male co-researchers) involved in the co-design of Sun Safe across a 10-month period (2018-2019) via telephone interviews or 2-hour in-person workshops (3 were run) [1]. Data were collected from participants (n=24, 17 female and 7 male participants) of placebo-controlled pilot intervention studies, which tested Sun Safe for 6 weeks (2020) [4]. Co-researchers downloaded and used the beta version of Sun Safe (via TestFlight) for 20 minutes during the final workshop (June 18, 2019) [1]. Pilot study participants accessed the fully developed app (v1.0.1, 2020) for 6 weeks in 2020 [4]; they also identified their gender (male, female, other, prefer not to say), age, and postcode of residence during recruitment. User quality ratings data were collected using the User Version of the Mobile Application Rating Scale (uMARS) [4].

There were twice as many recruited female participants (n=26) as male participants (n=13). Co-researchers were older (mean 13.8, SD 0.4 years) than pilot study participants (mean 12.7, SD 0.4 years). Most co-researchers used the app for 5-10 minutes (8/15, 53%); most pilot study participants used it every day or on most days (13/24, 55%).

Female co-researchers responded to more questions than male co-researchers (Table 1). Within subjective quality and perceived impact, male pilot study participants rated the Sun Safe app higher for overall star rating and help-seeking behaviors (Table 1).

Female pilot participants scored Sun Safe lower for engagement than female co-researchers (Figure 1).

Table 1. User quality ratings (User Version of the Mobile Application Rating Scale survey results) of the Sun Safe app for the subjective quality and perceived impact areas of assessment.
 Co-researchersPilot study participants
 MaleaFemaleP valueMaleFemaleP value
Participants, n69N/Ab717N/A
Questions completedc,d, n/N (%)89/156 (57.1)226/234 (96.6)N/A181/182 (99.5)442/442 (100.0)N/A
Subjective qualitye,f
 Recommendedg, mean (SD)3.3 (2.1)3.6 (0.5).633.7 (1.1)3.1 (1.1).26
 App useh, mean (SD)3.0 (1.7)4.1 (0.8).243.9 (0.7)3.3 (1.2).37
 Pay for app?i, n
 Overall star ratingj, mean (SD)4.2 (0.5)3.6 (0.8).154.7 (0.7)3.2 (0.9)<.001
Perceived impacte,k, mean (SD)
 Awarenessl3.7 (0.6)3.7 (0.8)>.994.0 (0.5)3.4 (1.1).17
 Knowledgem4.0 (0.0)4.0 (0.7)>.993.9 (0.7)3.4 (1.2).54
 Attitudesn3.0 (0.0)3.4 (1.1).753.6 (0.8)3.4 (0.9).79
 Intention to changeo3.3 (0.6)3.9 (0.8).414.1 (0.9)3.4 (1.1).17
 Help-seekingp3.7 (1.2)3.7 (0.7)>.993.9 (1.1)2.7 (1.0).04
 Behavior changeq4.0 (1.0)3.7 (1.0).754.0 (0.8)3.1 (1.1).09

aTwo male participants did not complete any questions.

bN/A: not applicable.

cTotal number of questions completed; 26 questions could be completed within the User Version of the Mobile Application Rating Scale (uMARS) survey by each participant.

dPercentage of questions completed of total possible (= total number completed by all participants / (n × 26) × 100), with statistical comparisons of the total number of uMARS survey questions completed (of 26), using Fisher Exact test, between male and female co-researchers (relative risk [RR] 0.60, 95% CI 0.50-0.70; P<.001) and pilot study participants (RR 0.99, 95% CI 0.97-1.00; P=.29).

eThe P values are the results of Mann-Whitney tests comparing data by gender (except for Pay for app?).

fAcross 4 questions, participants rated the subjective quality of the app, using 5-point scales (see below) or yes/no for Pay for app?

gWould you recommend this app to people who might benefit from it? (from 1, not at all, to 5, definitely).

hHow many times do you think you would use this app in the next 12 months? (from 1, none, to 5, >50 times).

iWould you pay for this app? Yes is the number of participants answering yes; no is the number of participants answering no.

jWhat is your overall star rating of the app? (from * to *****; One of the worst apps I’ve used to One of the best apps I’ve used).

kAcross 6 questions, participants rated the app based upon perceived capacity to modify awareness, knowledge, attitudes, intention to change, likelihood to seek help, and behaviors related to their sun health, using a 5-point scale of strongly disagree (1) to strongly agree (5).

lThis app has increased my awareness of the importance of addressing sun health behaviors.

mThis app has increased/changed my knowledge of sun health behaviors.

nThis app has changed my attitudes toward improving my sun health behaviors.

oThis app has increased my intentions/motivation to address my sun health behaviors.

pThis app would encourage me to seek further help to address my sun health behaviors (if needed).

qUse of this app will change my sun health behaviors.

Figure 1. Female pilot study participants rated Sun Safe lower in the engagement area of assessment. Mean scores for questions asked across the engagement area of assessment are shown individually for each co-researcher (3 male and 8 female) and pilot study (7 male and 17 female) participant. Data are shown as mean (SD). Two-way ANOVA was used to compare differences (participant type x gender), with Tukey post hoc tests identifying a statistically significant difference in predicted means of 0.92 (95% CI 0.24-1.60; P=.004) between female co-researchers and female pilot study participants. The five questions were posed, and 5-point Likert scales within this area of assessment were as previously published.
View this figure

Overall, few differences in app quality ratings were observed by gender, suggesting that Sun Safe was equally acceptable for use by young men and women even though fewer male participants were recruited to develop and test Sun Safe [1]. Pilot study participants rated Sun Safe lower for engagement, highlighting the importance of an independent review. Limitations included the relatively small sample size, differences in review time, and ongoing challenges in defining the influences of biological sex and gender on health outcomes [5]. Additional consumer engagement will help determine how games and gamification could be further built into Sun Safe.


We thank all recruited consumers, co-researchers, participants, and their parents/guardians for their input in this project. We also thank administrators and teachers from the local school for their assistance in participant recruitment and participation as well as officials from the Department of Education of Western Australia for their review of documents for governance approval. We also acknowledge Telethon Kids Institute Administrative team members (Leisa Hudson, Marina Stubbs) for their support in conducting the in-house workshops. The Sun Safe app was developed by Dr James White (Reach Health Promotion Innovation, Perth, WA). This research was supported by Healthway (Health Department of Western Australia) via a Health Promotion Exploratory Research Grant (31971). The sponsor had no role in the study design; the collection, analysis, and interpretation of data; the writing of the report; nor the decision to submit the manuscript for publication. JF was supported by a Healthway Early Career Research Fellowship (33020). SG was funded by an Al and Val Rosenstrauss Research Fellowship from the Rebecca L Cooper Foundation.

Conflicts of Interest

None declared.

  1. Nguyen R, Clare IM, Gamage N, Alvares GA, Black LJ, Hart PH, et al. Developing an online tool to promote safe sun behaviors with young teenagers as co-researchers. Front Digit Health 2021;3:626606 [FREE Full text] [CrossRef] [Medline]
  2. Telethon Kids Institute. Sun Safe. App Store.   URL: [accessed 2022-06-15]
  3. Gamage N, Nguyen R, Clare IM, Lucas RM, Strickland M, Granich J, et al. Sun-health behaviours and attitudes towards sun safety amongst Australian teenagers: a qualitative update. BMC Res Notes 2021 Sep 08;14(1):349 [FREE Full text] [CrossRef] [Medline]
  4. Clare IM, Gamage N, Alvares GA, Black LJ, Francis J, Jaimangal M, et al. The effects of using the Sun Safe App on sun health knowledge and behaviors of young teenagers: results of pilot intervention studies. JMIR Dermatol 2022 Mar 16;5(1):e35137 [FREE Full text] [CrossRef]
  5. Colineaux H, Soulier A, Lepage B, Kelly-Irving M. Considering sex and gender in Epidemiology: a challenge beyond terminology. From conceptual analysis to methodological strategies. Biol Sex Differ 2022 May 12;13(1):23 [FREE Full text] [CrossRef] [Medline]

Edited by R Dellavalle, T Sivesind; submitted 25.11.21; peer-reviewed by B Bock, D Buller; comments to author 10.06.22; revised version received 26.06.22; accepted 10.08.22; published 18.08.22


©Isabelle M Clare, Jacinta Francis, Nisali Gamage, Rebecca Nguyen, Shelley Gorman. Originally published in JMIR Dermatology (, 18.08.2022.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Dermatology, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.