The study was initially designed as a single center assessor-blinded parallel group, individually randomized controlled trial (RCT), using a 2 (mISkin app intervention vs no intervention) x 2 (sunscreen provision: SPF 15 vs SPF 30) factorial design. This study sought to explore the independent main effects of both sunscreen SPF and mobile app that would result in consistent differences between the levels (SPF 15 vs SPF 30; app vs no app), rather than an interaction effect.

The intended number of participants to be recruited for the definitive trial was 200, with the first 30 comprising the internal pilot phase. The internal pilot was conducted from September 2012 to October 2013. A schedule of events is presented in Multimedia Appendix 1 (schedule of events for the trial, including enrollment, interventions, and outcome assessments).

Holidaymakers from the North East of England traveling abroad, older than 18 years, and owning an Android smartphone were recruited. The recruitment strategy involved placing posters in local urban community areas such as universities, travel agency; large companies; and using social media (ie, Twitter and Facebook). Interested individuals contacted the research team by email. Participants were assessed for inclusion criteria by a researcher (AR) and provided informed consent before participation.

Participant feedback over the course of the trial indicated that random allocation to the distinct SPF strengths was not acceptable. Although the random allocation to sunscreen SPF continued, the protocol was amended to give participants three options: (1) SPF 15, (2) SPF 30, or (3) both. Records were kept about participant allocation acceptance and any request to change allocation.

This study was originally intended to be a full-scale definitive trial. The period until the first 30 participants to enter the study was defined as the internal pilot study.

A simple randomization was used with a 1:1:1:1 allocation ratio to assign participants to the experimental conditions. This was performed using a telephone-based randomization service provided by a staff member independent and blinded to the identity of individuals. An independent researcher generated and administered the randomization list. Only after baseline assessment were participants assigned to experimental groups.

At baseline assessment, the outcome assessor was blinded to allocation. At follow-up, assessors were aware of the allocation, and it was hypothesized that outcome assessors could not influence outcome measurements as these were mainly objective. Researchers conducting the lab analyses for skin damage were blinded to allocation, and samples were given a code unrelated to the trial ID. This coding was performed by an independent lab researcher.

The analyses focused on descriptive data regarding recruitment rates and attrition, as well as acceptability and participants’ satisfaction with the intervention. The main aim was to test whether the proposed protocol was viable for a definitive trial.

For the secondary outcomes, the focus of the analysis was on data yield and quality. Descriptive statistics are provided for participants’ characteristics and trial outcomes at baseline and follow-up.

A total of 42 participants were recruited from December 2012 to October 2013. Figure 2 shows the flow of participants through the feasibility study. As seen on the flow diagram, there were six protocol deviations.

Participants’ characteristics and demographics can be found in Table 1. The mean age of the participants was 35.5 years (SD 9.7; N=42), with more women participating (76% [32/42]). The majority of participants reported that they usually burn and tan minimally (35% [15/42]). The most frequent holiday destination was Spain (N=12) and lasting more than 14 days (N=15).

Modification and potential improvements have been recorded in Table 3, following guidelines to produce an informed decision on the internal pilot trial [31]. Evidence gathered during the internal pilot study identified significant problems influencing acceptability and feasibility of the internal pilot that led to stop the trial early, namely the SPF allocation and improvements to the app. The trial was stopped in October 2013, and a definitive trial on the mISkin app has not been conducted. The study was part of a PhD thesis [32], and funding ran out to relaunch the new protocol.

This study described an internal pilot study aimed at testing acceptability and feasibility of the trial procedures and the mISkin app. The internal pilot study revealed potential issues on participant recruitment, acceptability of the randomization to participants, and features of the intervention. A list of possible solutions to the problems identified within pilot and feasibility RCT was mapped (Table 3) following the algorithm for  decision making after pilot and feasibility trials guidelines [33].

Despite most trial procedures being considered acceptable and feasible, the random allocation to SPF 15 versus SPF 30 was found not to be feasible in a definitive trial protocol. During the internal pilot, the SPF allocation procedures were modified to improve acceptability of participants’ allocation. A preference design trial based on a participant’s preference regarding sunscreen SPF could be a possible solution for a definitive trial.

Regarding the feasibility of recruitment, this was limited by the inclusion requirement to own an Android smartphone. Several participants were excluded because they owned other smartphones, showing the need to improve the interoperability of the mISkin app in a definitive trial. The relatively low rate of recruitment could also be improved by diversifying the pathways of recruitment (eg, pharmacies).

The key issue reported about the mISkin app was the Sun alert service, and future revisions of the app should consider improving this feature. The need for more intelligent and interactive systems has been reported previously [34]. The study by Buller and colleagues [35] also shows that participants were interested in a system that would (1) display how long they could be exposed to the sun without burning (including vitamin D synthesis), (2) show daily UV levels, (3) advice on recommended SPF, and (4) send prompts to reapply sunscreen. Nevertheless, such a system would need to tackle some of the uncertainties related to vitamin D synthesis and sunscreen use. The estimation of time needed to synthesize vitamin D is a complex and challenging calculation influenced by various factors, including skin type, age, body fat, genetic factors, lifestyle, sun protection, solar zenith angle, and atmospheric conditions [36,37]. For sunscreen, most calculations rely on the assumption that people apply the recommended amount (2 mg of sunscreen per cm²of skin surface) to achieve the labeled protection [38,39]. However, evidence suggests that individuals tend to apply less than the recommended amount [14,40], dramatically reducing its protective features. In addition, NICE [41] does not recommend any specific amount of sunlight exposure to stimulate vitamin D production, only stating that longer periods of exposure may be needed for those with darker skin [21].

Sun protection information delivered via a purpose-designed mobile phone app was feasible, attractive, and convenient, and usage was high with 86% using it at least once during holidays. Users were particularly interested in the Sun alert service, highlighting the value of receiving prompts and advice in convenient place (eg, holiday setting). These findings, although preliminary, are very promising and align with those found in other investigations of mobile apps to promote sun protection, in which usage and acceptability have been similarly demonstrated [11,12,35]. The systematic and iterative development of the app [15] and the theory-driven nature of the app [42], including active components shown to be key in interventions promoting sun protection [13], may have led to greater usage. Further research is required to assess whether the mISkin app can successfully change sun protection behaviors and to identify factors that can contribute to the uptake of the app. As it is easy and convenient to receive advice, delivering sun protection information via a mobile phone app is a promising alternative or addition to existing skin cancer prevention interventions.

Overall, sunscreen use was low in this study, with an average daily use of 14.46 g. This is a special concern if the average exposure time of 5.36 hours per day is taken into consideration. The guideline for sunscreen application thickness is 2 mg/cm² [43]. According to Diffey [44], a full body application will consist of 35 g of the sunscreen (ie, one-third of a bottle). In line with our findings, a study conducted by Nicol and colleagues [40], with 364 beachgoers, shows that the daily amount of sunscreen used was 7.67 g/day and 9.33 g/day for the intervention groups. Rodrigues and colleagues [14] also showed that participants used less than the recommended sunscreen amount (1.34 mg/cm²) in an experimental setting. Future preventive strategies should provide more explicit instructions of sunscreen application thickness, possibly relating to practical examples (eg, “as much as a golf ball or a full shot glass” for whole body coverage).

Even though the use of gamification within the mISkin app is original, more efforts could be made to make this feature more engaging. Gamification can be defined as the use of gaming elements in a nongaming context to foster motivation [45]. Two core ingredients of gamification (linked to behavior change techniques) were used in the mISkin app: reinforcement and progress comparison. A key aspect of gamification is the concept of rewards that promote continuous participation, promoting not only engagement, but also behavior change [46]. Gamification applied to digital health has the potential to make the interventions more fun and engaging [45]. Recent findings have shown the potential of health apps to change behavior though gamification [47-49]. The current quiz has a set of questions that, despite allowing participants to go through it several times, does not change. A possible way to make this more attractive and further “gamify” would be to use gaming principles more systematically, such as allowing users to gain points every time they reapply sunscreen; or engage with app features; or by gradually increase the difficulty of the quiz (ie, different levels to increase challenge) [50].

This paper describes the efforts in addressing the main challenges identified in a previous systematic review on sun protection interventions: (1) poor reporting of intervention development, design, and contents; (2) poor outcome measurement; and (3) poor study methodology [13]. The tested intervention was developed based on the most recent evidence base available [13,14], and the development process followed a systematic approach, with a thorough report of the process and description of intervention [15]. In addition, the use of digital technologies followed the most recent advances in the area of behavioral science, with a close involvement of users in the design and development of the app [51]. The use of a combination of biologic, technological, and self-report outcome measures to understand and assess sun protection also provided a step forward in the field. Finally, the mISkin intervention was subject to intensive pilot testing, following the preregistered methods of a definitive trial. The methodology implemented aims to reduce the risk of bias by using robust procedures of blinding, allocation concealment, and intention to treat.

The process evaluation alongside the internal pilot trial is also a strength, as it provides relevant information on the mISkin intervention acceptability and how the trial procedures could be enhanced. However, the findings should be interpreted in the context of the study limitations given the nature of the study (ie, feasibility study), and some caution should be taken in generalizing these results.

The feedback on the app and trial procedures was collected through face-to-face interviews and could have introduced bias to the self-reported acceptability of the intervention and trial procedures. The possibility of measurement reactivity was also a concern, especially considering the comprehensive list of self-reported outcomes used. A recent systematic review on the topic [52] did not find any study assessing question-behavior effect on sun protection behavior and concluded that the “question-behavior effect” on health-related behavior is small and was therefore not considered when designing the protocol.

The amount of sunscreen provided might have not been enough, as it is possible that participants inferred (incorrectly) that by giving them 2 bottles (2 x 200 mL) this amount would be sufficient for their whole holiday, and therefore they tried to eke it out. The app highlighted that sunscreen should not be the first line of defense and targeted other forms of sun protection, such as wearing protective clothes and avoiding sun exposure at midday [4].

For practical reasons and resources available at the time of this study, the UV photos shown in the mISkin app were not personalized. Recent evidence suggests the importance of appearance-based beliefs and how interventions should aim at tackling those by showing personalized UV photos that depict damage [13,53,54]. Future studies should explore whether the effects of visualizing nonpersonalized UV photos are equivalent to personalized UV photos.

Unfortunately, this study faced considerable challenges in recruitment. Despite several attempts to involve high-street travel agencies and the local airport, the involvement of a gatekeeper to facilitate the access to holidaymakers was unsuccessful. It took several contacts to reach the relevant decision makers, and many highlighted that the aim of the study to promote sun protection would conflict with their products. This is consistent with other studies that have encountered similar difficulties when involving tourism industries [55,56].

This paper summarizes an internal pilot and feasibility RCT, testing the acceptability and feasibility of the trial procedures and the newly developed mISkin app. The evidence-based intervention was highly acceptable to participants, but the recruitment strategy and allocation to SPF 15 versus 30 were not feasible. This pilot study offers potential solutions to inform the trial procedures of a future trial and to improve the mISkin app, namely the possibility of participants choosing their sunscreen SPF, using alternative pathways to recruit holidaymakers (ie, pharmacies), and upgrading the interoperability of the mISkin app.