Development, Feasibility, and Acceptability of the Electronic Patient Benefit Index for Psoriasis in Clinical Practice: Mixed Methods Study

Introduction

Studies already showed the relevance of patient-reported outcomes (PROs) in clinical practice [1]. PROs can provide information on the effectiveness and benefits of treatments and on changes in patient functional status over time [2] and can support patient referrals [3]. PROs can also be used to evaluate and compare health care providers and systems and can inform quality improvement efforts [2].

The National Health Service Outcomes Framework cites enhancing quality of life (QoL) for people with long-term conditions as a key domain for improvements [4]. The European Academy of Dermatology and Venereology Task Force on QoL considers that there are several ways in which the measurement of QoL in clinical practice may benefit patients, support clinicians’ decision-making, and contribute to the delivery of high standards of care [5]. PROs in dermatology include, for example, QoL, patient benefit and goal measures, treatment harm assessment, and treatment response adequacy [4,6,7]. They all enhance the management of psoriasis and, thereby, improve patients’ lives [8].

Many PROs are already included in data sets for documenting psoriasis in clinical practice [9], for example, the Patient Benefit Index (PBI) by assessing patients’ treatment goals and benefits [10]. Analyzing patient needs on an individual level facilitates shared decisions by patients and physicians and optimizes personalized treatment [11]. Furthermore, the World Health Organization [6] and the Techniker health insurance [12] emphasize the importance of patient-defined treatment goals.

The PBI (Multimedia Appendix 1) is a valid instrument suitable for the assessment of patient-reported goals and benefits in dermatological studies and practice. In the first part of the questionnaire (Patient Needs Questionnaire; PNQ), patients state how relevant 25 goals are for them on a 5-point Likert scale (0=not important at all to 4=very important). In the second part of the questionnaire (Patient Benefit Questionnaire; PBQ), they state how the identical items detect the extent to which the current therapy has contributed to attaining the therapy goals (scaled from 0=treatment did not help at all to 4=treatment helped a lot). Patients can also tick the option “does not apply to me.” The PBI total score is derived from the ratings on both questionnaires, as the PBI score is the arithmetic mean of all rated benefits (PBQ items) weighted by the relative importance of each corresponding need item (PNQ) for each patient [10,13]. The PBI has already widely been used in research.

With improved access to the internet and increased use of electronic devices, the digital health care sector gained importance. This makes the collection of electronic PROs more feasible [14,15]. Many benefits were detected by the digitalization of PROs such as error reductions, automatic scoring calculations, management of data security measures, and better access to data [16]. Furthermore, electronic PROs turned out to be feasible and acceptable to dermatologists and patient groups with different indications [3,17,18], but burdens remain [18,19]. In dermatology, digital monitoring is still rarely used [6,20] and is not evaluated in practice and for shared decision-making. The aim of this study was (1) to develop the electronic PBI (ePBI), (2) to examine its feasibility, and (3) acceptability for patients with psoriasis.

Methods

This paper follows STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) guidelines where applicable. To develop and test the ePBI, we followed 3 steps.

Step 1: Conceptualization Phase

We conducted 3 focus groups with 14 patients with psoriasis (Figure 1). The goal was to develop a basis for the conceptualization of an electronic documentation system as well as for the development of the ePBI and the questionnaire. The focus groups were recorded, fully transcribed, and subjected to content analysis [21].

‎

Step 2: Development of ePBI

We developed an app and a web version of the PBI [22]. For better feasibility and use in practice, adaptations were necessary:

• App: We included a first screen showing a link to PNQ and PBQ.
• App: We developed 1 page for the introduction text and each item.
• App and web: We included the calculation and presentation of resulting scores.
• App and web: We changed some words within the introduction of the PBQ and deleted or adapted the last sentence (“Please check once more if you have exactly marked each statement ...”).

Due to the PBI being a complex outcome measure and to the adaptations described earlier, we conducted further steps in developing and testing the electronic version as recommended by the Professional Society for Health Economics and Outcome Research (ISPOR) [15,23]. Cognitive debriefing allowed adaptations and iterative development of both electronic versions. Altogether, we conducted 3 rounds with 11 patients in total and used different techniques testing different parts of the ePBI until no further adjustments were necessary (Table 1). Thereby, the items of PNQ and PBQ themselves were not open for discussion.

Step 3: Feasibility and Acceptability Testing of ePBI

An observational study was conducted using a standardized questionnaire on patients’ demography and health status as well as on the feasibility and acceptability for both ePBI versions. We used nominal, ordinal, interval, and ratio scales. Mainly, participants answered items on 5-point Likert scales.

Patients and Conduction

Patients were recruited at an outpatient dermatology care unit of the University Medical Center Hamburg-Eppendorf and were eligible to participate if they were German-speaking and diagnosed with psoriasis. One-half of patients filled in the app version on a smartphone. The other half filled in the web version on a laptop. After that, both groups filled in the questionnaire described earlier [15,23]. The interviewer stopped the time when each patient needed to fill out the ePBI.

Ethical Considerations

According to the ethics commission of the Medical Association of Hamburg, there was no ethics vote required because of anonymous and noninterventional data collection for both the qualitative and quantitative parts of the study (WF-053/19). All participants gave written informed consent before participation. They were informed that they could cancel participation at any time. No compensation was provided for participation.

Analysis

The quantitative data were analyzed using descriptive statistics. For bivariate analysis, we classified metric variables and variables with 5-point Likert scales into 3 categories. Comparison of response behaviors was conducted using cross tables, chi-square test, and Fisher exact test if cross tables had more than 20% of cells with expected counts below 5. A significance level of α=.05 was applied. For investigating the direction of the association, adjusted standardized residuals were calculated with values below –1.96 and above 1.96 revealing significance. Subgroup comparison was conducted for sex, age, education, and ePBI version (app vs web). We used SPSS (version 25.0; IBM Corp).

Results

In total, 139 patients participated, and all filled in the questionnaire on demography, feasibility, and acceptability: 52.5% (n=73) completed the app, and 47.5% (n=66) completed the web version.

Participants

The sample encompassed 67.6% (n=94) of male participants and had a mean age of 47 (SD 13.95; median 49; range 18-84) years. On average, the first psoriasis symptoms appeared 22 (SD 14.66; median 19; range 0-65) years ago, and the first diagnoses had been made 21 (SD 15.15; median 19; range 0-65) years ago. The mean Dermatology Life Quality Index score was 4.57 (SD 5.50; median 3; range 0-22), the mean EQ-5D-5L score was 83.05 (SD 21.92; median 88; range 8.83-100), the mean European Quality Visual Analogue Scale score was 73.93 (SD 20.43; median 80; range 1-100), and the mean PBI score 2.77 (SD 1.13; median 3.11; range 0.10-4.0). Most participants were of higher education (intermediate and high: n=107, 76.9%), and had several comorbidities (eg, n=58, 41.7% had psoriasis arthritis; on average, a patient had 0.98 comorbidities; Tables 2 and 3).

The majority was experienced in using internet-ready devices (eg, n=124, 89.9% used smartphones frequently; Table 4).

Feasibility

Most participants (“rather” or “totally”) agreed on the ePBI being usable (Table 5), stating that the questions and texts were easy to read (n=135, 97.1%), the questions (n=133, 95.7%) or instructions (n=137, 98.6%) or information (n=129, 92.8%) understandable, the handling simple (n=137, 98.5%), the questionnaire easy to complete (n=136, 97.8%), and the information on individual results satisfying (n=122, 87.8%). A small number of participants declared that they needed help filling in the ePBI (n=24, 17.3%) and found it too long (n=30, 21.6%) but visually appealing (n=117, 84.2%; Table 3). On average, they needed 6.92 (SD 2.49; median 7; range 2.75-14) minutes to fill in the electronic PNQ and PBQ, read the results, and the last page including the PBI score (Table 5).

Acceptability

Acceptability of the ePBI was also very high (Tables 6-9) but with more participants being indecisive (between n=7, 5% and n=41, 29.5%). Most participants (“rather” or “totally agreed”) could imagine filling in the ePBI on a regular basis (n=88, 63.3%), but less so if the documentation would be only for their own use (n=55, 39.5% and n=34, 24.5% being indecisive). They could imagine to discuss the content with their physician (n=91, 65.5%). They thought the ePBI could form a good basis for a patient-physician consultation (n=98, 70.5%), support treatment decisions (n=118, 84.9%), improve communication with the physician (n=113, 81.3%), and improve the relationship with the physician (n=78, 56.1%). In addition, it would help them to remember their symptoms and well-being better during patient-physician consultation (n=94, 67.7%). It would also help them to manage their condition (n=90, 64.7%; over periods of time: n=129, 92.8%). In general, patients could imagine filling in electronic questionnaires (n=118, 84.9%). They did not agree that regular completion might make them feel sad (n=120, 86.3%) and they would not prefer filling in the questionnaires on paper (n=113, 81.2%; Table 6). The majority of participants could imagine filling in the ePBI at every patient-physician consultation (n=85, 61.2%; Table 7). However, they accepted much greater effort for documenting their data regarding frequency, length, and device (Tables 7-9).

Subgroup Comparison

Subgroup comparison showed some significant differences regarding sex, age, education, and ePBI version (web or app; Multimedia Appendix 2). Female participants disagreed more often than male participants that data collection of ePBI can improve communication between physician and patient. They were more often of the opinion that they could gain more control over their disease and showed significantly more missing values on the question if they could imagine filling in the ePBI via their own laptop or PC (see adjusted residuals in Multimedia Appendix 2).

Patients older than 60 years were less likely to think that filling in e-surveys daily was too frequent. They were willing to spend more time on completing (long) questionnaires although it took them more time. The younger group even needed less assistance. Anyway, the older groups still preferred completion on paper, respectively, less often to fill in electronic questionnaires in comparison to the younger groups.

Patients with low or no school degrees expected the relationship with their physician to improve by filling in the ePBI. They were less willing to spend a lot of time on questionnaire completion than the higher education groups, as it also took them longer to complete an ePBI.

Only one significant difference was found between the opinions of patients filling in the app in comparison to the web version: patients who filled in the web version more often thought that the questionnaire would form a good basis for a patient-physician consultation than the patients who used the app (see adjusted residuals in Multimedia Appendix 2).

Discussion

Summary

We developed an app and a web version of the existing and valid PBI using the results of focus groups for conceptualization and cognitive debriefing to adapt the preliminary versions as recommended by the ISPOR. The results of the subsequent survey showed a high understanding and feasibility of the ePBI. Moreover, the acceptability was also very high: patients can imagine using and discussing the ePBI in practice and documenting it regularly, expecting improvements, for example, better treatment decisions and communication with their physician, as well as gaining more control over their disease. Importantly, they can imagine filling in the ePBI and other electronic questionnaires regularly, even preferring the electronic over the paper versions. Other studies also showed high feasibility and acceptability of electronic data collection [19,24,25]. These results show that the basis for an implementation is in place. However, the acceptability of patients with dermatological diseases is higher compared to dermatologists’ acceptability [19,24], indicating barriers for implementation as physicians play a key role in disseminating [26].

Principal Findings

The frequent values of the “partly agree” answers (Likert scale) to the acceptability questions show that some patients may have difficulties in imagining the use of an electronic PRO in clinical practice. We assume that the more experiences people gain with electronic devices, the more usable and acceptable it will become.

We did not find fundamental differences in sex. Merely, female participants in our study would use the ePBI more often than male participants to gain more control over their disease, which goes in line with previous findings showing that female participants are more health-conscious than male participants [27]. According to male participants, the ePBI could help to improve communication with their physician, which may be more important for them to improve than for female participants.

We detected several differences in age, which are reinforced by other studies, after which older adults use technologies at lower rates than other age groups [28]. In this study, the older participants had more problems filling in the ePBI and would more often prefer paper versions over digital ones. This may indicate a digital divide. However, they were willing to spend more time documenting data digitally, probably reflecting their higher needs. Existing data also reveal that internet use by older people is increasing, expecting a less digital divide in the future [28].

Participants with lower school degrees had more difficulties filling in the ePBI and were less willing to spend time documenting their data than those with higher education. We assume that it takes them longer to understand the concept of and get used to such digital applications. Again, this may indicate a digital divide. Nevertheless, our data reveal that it may be especially promising for this group to use the ePBI, as participants with lower school degrees more often expected an improved relationship with their physicians.

We only found one difference between the participants’ opinions using the different modes of data collection. They thought the web version would be more suitable to form a good basis for a patient-physician consultation than the app. In the web version, the questionnaire looks more familiar, as the structure is similar to a paper version. It is also easier to discuss data on a big screen than on a mobile phone. However, as soon as data are digitalized, there are several opportunities to display and store them on different electronic devices. Again, we assume that gaining more experience in data collection and use will increase usability and acceptability.

Due to its many years of proven use in research and practice, we have conducted various qualitative studies on the development of the ePBI. We followed guidelines on research, implementation, and validation processes. The sampling size was chosen based on statistical necessity and therefore appropriate to answer the present research question. As recruitment was conducted in a specified outpatient ward, the sample mainly includes patients who are moderately and severely affected. Digital applications especially aim to support these patient groups, as they have more issues dealing with their disease. Our sample consists of many patients who are well-educated and experienced regarding internet-ready devices, possibly resulting in more positive attitudes on feasibility and acceptability as our results show with respect to the subgroup analysis. The digitalization rate is only slightly higher than within the general population, for example, in Germany [29]. As nearly everybody within our sample had great experiences with digital devices, we did not expect significant differences between low or high users within our sample. Unfortunately, we did not capture the number of patients who were asked to participate but declined. Therefore, we have no information about that group.

Conclusions

The app and web version of the ePBI are usable and acceptable for patients. The results show that the basis for an implementation is in place. This offers many advantages not only for research but also for documenting and more comprehensively using patient data in practice, such as in eHealth records or other documentation applications, for shared decision-making and patient participation. Thereby, a focus should be placed on the digital divide with special attention on age and education. People with higher age and lower education seem to have more difficulties using and understanding digital applications. In contrast, they are willing to put more effort into documentation and see other important advantages, which may increase successful use. Information on and experiences with digital applications, such as the ePBI or other digital monitoring solutions, will increase usability and acceptability.

Our results can support other researchers, developers, and decision makers. Physicians need to be better informed and included in processes, as their low acceptability and use may form a barrier for future implementations. In Germany, some political barriers also hinder digitalization in medicine, for example, no unified telematics infrastructure, reimbursement, and data security issues.