JDERM JMIR Dermatol JMIR Dermatology 2562-0959 JMIR Publications Toronto, Canada 39475809 v5i3e35203 10.2196/35203 Research Letter Research Letter A Gender Lens on User Quality Ratings From Young Teenagers Assessing the Sun Safe App: Comparing Responses From Co-researchers and Participants of Pilot Intervention Studies Dellavalle Robert Sivesind Torunn Bock Beth Buller David Clare Isabelle M BBioMedSc 1 https://orcid.org/0000-0001-6341-601X Francis Jacinta BSc, MPH, PhD 1 https://orcid.org/0000-0002-8701-8195 Gamage Nisali BSc 1 https://orcid.org/0000-0001-7517-6498 Nguyen Rebecca BCom, BA 1 https://orcid.org/0000-0001-5069-1346 Gorman Shelley BSc, PhD 1
Telethon Kids Institute University of Western Australia Level 7, Perth Children's Hospital, 15 Hospital Drive, Nedlands Perth, 6052 Australia 61 8 6319 1777 61 8 6319 1747 Shelley.Gorman@telethonkids.org.au
https://orcid.org/0000-0002-7111-6735 Telethon Kids Institute University of Western Australia Perth Australia Corresponding Author: Shelley Gorman Shelley.Gorman@telethonkids.org.au Jul-Sep 2022 18 8 2022 5 3 e35203 25 11 2021 10 6 2022 26 6 2022 10 8 2022 ©Isabelle M Clare, Jacinta Francis, Nisali Gamage, Rebecca Nguyen, Shelley Gorman. Originally published in JMIR Dermatology (http://derma.jmir.org), 18.08.2022. 2022

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Dermatology, is properly cited. The complete bibliographic information, a link to the original publication on http://derma.jmir.org, as well as this copyright and license information must be included.

 app development co-design sun exposure sun protection teenager uMARS UV index vitamin D young adolescents sunburn adolescent smartphone gender sun protection app engagement sunburn risk melanoma decision-making Introduction

We developed the iOS smartphone app Sun Safe to support healthy sun practices in young teenagers (aged 12-13 years) [1]. The production involved co-design with young co-researchers (ie, aged 12-13 years) with a health message of using sun protection when the UV index is ≥3 [1]. Important features include real-time and location-specific weather data on the UV index and gamified educational content [1,2].

We were concerned that indifferent attitudes expressed by male co-researchers during the development of Sun Safe [3] would translate into gendered differences in user quality ratings. Furthermore, we wondered whether involvement in the co-design process could bias quality assessments. The results presented in this letter compare the responses of co-researchers [1] with those of participants of the pilot intervention studies [4].

 Methods

All methods underpinning the development of the app and pilot intervention studies are described elsewhere [1,4]. Data were collected from co-researchers (n=15, 9 female and 6 male co-researchers) involved in the co-design of Sun Safe across a 10-month period (2018-2019) via telephone interviews or 2-hour in-person workshops (3 were run) [1]. Data were collected from participants (n=24, 17 female and 7 male participants) of placebo-controlled pilot intervention studies, which tested Sun Safe for 6 weeks (2020) [4]. Co-researchers downloaded and used the beta version of Sun Safe (via TestFlight) for 20 minutes during the final workshop (June 18, 2019) [1]. Pilot study participants accessed the fully developed app (v1.0.1, 2020) for 6 weeks in 2020 [4]; they also identified their gender (male, female, other, prefer not to say), age, and postcode of residence during recruitment. User quality ratings data were collected using the User Version of the Mobile Application Rating Scale (uMARS) [4].

 Results

There were twice as many recruited female participants (n=26) as male participants (n=13). Co-researchers were older (mean 13.8, SD 0.4 years) than pilot study participants (mean 12.7, SD 0.4 years). Most co-researchers used the app for 5-10 minutes (8/15, 53%); most pilot study participants used it every day or on most days (13/24, 55%).

Female co-researchers responded to more questions than male co-researchers (Table 1). Within subjective quality and perceived impact, male pilot study participants rated the Sun Safe app higher for overall star rating and help-seeking behaviors (Table 1).

Female pilot participants scored Sun Safe lower for engagement than female co-researchers (Figure 1).

User quality ratings (User Version of the Mobile Application Rating Scale survey results) of the Sun Safe app for the subjective quality and perceived impact areas of assessment.

  Co-researchers Pilot study participants
  Malea Female P value Male Female P value
Participants, n 6 9 N/Ab 7 17 N/A
Questions completedc,d, n/N (%) 89/156 (57.1) 226/234 (96.6) N/A 181/182 (99.5) 442/442 (100.0) N/A
Subjective qualitye,f
  Recommendedg, mean (SD) 3.3 (2.1) 3.6 (0.5) .63 3.7 (1.1) 3.1 (1.1) .26
  App useh, mean (SD) 3.0 (1.7) 4.1 (0.8) .24 3.9 (0.7) 3.3 (1.2) .37
  Pay for app?i, n
    Yes 0 3 N/A 0 3 N/A
    No 3 6 N/A 7 14 N/A
  Overall star ratingj, mean (SD) 4.2 (0.5) 3.6 (0.8) .15 4.7 (0.7) 3.2 (0.9) <.001
Perceived impacte,k, mean (SD)
  Awarenessl 3.7 (0.6) 3.7 (0.8) >.99 4.0 (0.5) 3.4 (1.1) .17
  Knowledgem 4.0 (0.0) 4.0 (0.7) >.99 3.9 (0.7) 3.4 (1.2) .54
  Attitudesn 3.0 (0.0) 3.4 (1.1) .75 3.6 (0.8) 3.4 (0.9) .79
  Intention to changeo 3.3 (0.6) 3.9 (0.8) .41 4.1 (0.9) 3.4 (1.1) .17
  Help-seekingp 3.7 (1.2) 3.7 (0.7) >.99 3.9 (1.1) 2.7 (1.0) .04
  Behavior changeq 4.0 (1.0) 3.7 (1.0) .75 4.0 (0.8) 3.1 (1.1) .09

aTwo male participants did not complete any questions.

bN/A: not applicable.

cTotal number of questions completed; 26 questions could be completed within the User Version of the Mobile Application Rating Scale (uMARS) survey by each participant.

dPercentage of questions completed of total possible (= total number completed by all participants / (n × 26) × 100), with statistical comparisons of the total number of uMARS survey questions completed (of 26), using Fisher Exact test, between male and female co-researchers (relative risk [RR] 0.60, 95% CI 0.50-0.70; P<.001) and pilot study participants (RR 0.99, 95% CI 0.97-1.00; P=.29).

eThe P values are the results of Mann-Whitney tests comparing data by gender (except for Pay for app?).

fAcross 4 questions, participants rated the subjective quality of the app, using 5-point scales (see below) or yes/no for Pay for app?

gWould you recommend this app to people who might benefit from it? (from 1, not at all, to 5, definitely).

hHow many times do you think you would use this app in the next 12 months? (from 1, none, to 5, >50 times).

iWould you pay for this app? Yes is the number of participants answering yes; no is the number of participants answering no.

jWhat is your overall star rating of the app? (from * to *****; One of the worst apps I’ve used to One of the best apps I’ve used).

kAcross 6 questions, participants rated the app based upon perceived capacity to modify awareness, knowledge, attitudes, intention to change, likelihood to seek help, and behaviors related to their sun health, using a 5-point scale of strongly disagree (1) to strongly agree (5).

lThis app has increased my awareness of the importance of addressing sun health behaviors.

mThis app has increased/changed my knowledge of sun health behaviors.

nThis app has changed my attitudes toward improving my sun health behaviors.

oThis app has increased my intentions/motivation to address my sun health behaviors.

pThis app would encourage me to seek further help to address my sun health behaviors (if needed).

qUse of this app will change my sun health behaviors.

Female pilot study participants rated Sun Safe lower in the engagement area of assessment. Mean scores for questions asked across the engagement area of assessment are shown individually for each co-researcher (3 male and 8 female) and pilot study (7 male and 17 female) participant. Data are shown as mean (SD). Two-way ANOVA was used to compare differences (participant type x gender), with Tukey post hoc tests identifying a statistically significant difference in predicted means of 0.92 (95% CI 0.24-1.60; P=.004) between female co-researchers and female pilot study participants. The five questions were posed, and 5-point Likert scales within this area of assessment were as previously published.

 Discussion

Overall, few differences in app quality ratings were observed by gender, suggesting that Sun Safe was equally acceptable for use by young men and women even though fewer male participants were recruited to develop and test Sun Safe [1]. Pilot study participants rated Sun Safe lower for engagement, highlighting the importance of an independent review. Limitations included the relatively small sample size, differences in review time, and ongoing challenges in defining the influences of biological sex and gender on health outcomes [5]. Additional consumer engagement will help determine how games and gamification could be further built into Sun Safe.

We thank all recruited consumers, co-researchers, participants, and their parents/guardians for their input in this project. We also thank administrators and teachers from the local school for their assistance in participant recruitment and participation as well as officials from the Department of Education of Western Australia for their review of documents for governance approval. We also acknowledge Telethon Kids Institute Administrative team members (Leisa Hudson, Marina Stubbs) for their support in conducting the in-house workshops. The Sun Safe app was developed by Dr James White (Reach Health Promotion Innovation, Perth, WA). This research was supported by Healthway (Health Department of Western Australia) via a Health Promotion Exploratory Research Grant (31971). The sponsor had no role in the study design; the collection, analysis, and interpretation of data; the writing of the report; nor the decision to submit the manuscript for publication. JF was supported by a Healthway Early Career Research Fellowship (33020). SG was funded by an Al and Val Rosenstrauss Research Fellowship from the Rebecca L Cooper Foundation.

None declared.

 Nguyen R Clare IM Gamage N Alvares GA Black LJ Hart PH Lucas RM Strickland M Jaimangal M White J Gorman S Developing an online tool to promote safe sun behaviors with young teenagers as co-researchers Front Digit Health 2021 3 626606 10.3389/fdgth.2021.626606 34713099 PMC8521794 Telethon Kids Institute Sun Safe App Store 2022-06-15 https://apps.apple.com/au/app/sun-safe/id1479811784 Gamage N Nguyen R Clare IM Lucas RM Strickland M Granich J Gorman S Sun-health behaviours and attitudes towards sun safety amongst Australian teenagers: a qualitative update BMC Res Notes 2021 09 08 14 1 349 10.1186/s13104-021-05764-9 34496962 10.1186/s13104-021-05764-9 PMC8425089 Clare IM Gamage N Alvares GA Black LJ Francis J Jaimangal M Lucas RM Strickland M White J Nguyen R Gorman S The effects of using the Sun Safe App on sun health knowledge and behaviors of young teenagers: results of pilot intervention studies JMIR Dermatol 2022 3 16 5 1 e35137 10.2196/35137 Colineaux H Soulier A Lepage B Kelly-Irving M Considering sex and gender in Epidemiology: a challenge beyond terminology. From conceptual analysis to methodological strategies Biol Sex Differ 2022 05 12 13 1 23 10.1186/s13293-022-00430-6 35550193 10.1186/s13293-022-00430-6 PMC9103114