A literature search was conducted to identify studies that describe the codes and logic used to identify HS patients in observational databases. This literature search identified 30 articles, which provided a set of diagnosis codes for the identification of HS across vocabularies, including the ICD-9, the International Classification of Diseases, Tenth Revision (ICD-10), and Read codes. Five of the 30 articles included validation metrics. Our study utilized the Systemized Nomenclature of Medicine (SNOMED) vocabulary to develop the codes. The vocabulary and diagnostic codes used in the published studies and the SNOMED terms are presented in Multimedia Appendix 1. The Observational Health Data Sciences and Informatics (OHDSI) open-source Atlas tool [33] was used to create the HS PAs.

The observational databases used in this study were not created specifically to study HS. The observational data were obtained in the delivery of health care or for administrative or billing purposes in electronic format. A network of 9 observational databases (4 administrative claims databases from the United States, 1 from Japan, 1 from France, 1 from Germany, and 1 from Australia; and 1 US electronic health record [EHR] database; Table 1) were used to develop the PAs. The 9 databases were a mix of administrative insurance claims, EHRs, and general practitioner databases. Descriptions and details of each database are shown in Table 2. The databases were transformed to the Observational Medical Outcomes Partnership (OMOP) Common Data Model (version 5.3.1) [34] so the PAs could be consistently applied across databases.

Four HS PAs were developed and evaluated in subjects of all ages [35] (Figure 1). The PA “incident 1x” used the first diagnosis code for HS in a subject’s history and required 365 days of prior continuous enrollment (CE) time to qualify for entry into the HS cohort. The date a subject met both criteria was the subject’s index date. The PA “incident 2x” used the first diagnosis code for HS in a subject’s history and required both a second HS diagnosis code within 31 to 365 days and 365 days of prior CE time. The date a subject met all 3 criteria became the subject’s index date. The prevalent PAs (“prevalent 1x” and “prevalent 2x”) were identical to the corresponding incident versions, except that the first HS diagnosis code was not required to be the first time an HS code occurred in a subject’s history, nor was there a requirement for 365 days prior CE.

The OHDSI CohortDiagnostics tool [36] allowed for evaluation and comparison of PAs at a cohort level, providing overall counts, incidence over time, the diagnosis code that allowed the subject into the cohort, cohort overlap, and temporal characterization.

Use of the PheValuator [37] method provided performance metrics, including the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) associated with each PA. PheValuator is a machine learning–based method of assessing PAs. It constructs a predictive model for the disease and calculates the predictive value of having the disease for each subject using the model. Using PheValuator, performance indices of an algorithm are calculated without reviewing medical charts. While algorithm validation results from chart review are considered the “gold standard,” we have compared the results from PheValuator with prior studies using chart review and found excellent agreement between the 2 methods [38]. Four additional PAs from Kim et al [16] were evaluated for comparison.

Computer code for PheValuator and CohortDiagnostics and the JSON files for the PAs are available on the authors’ website [39].

The use of the IBM and Clinformatics databases was reviewed by the New England Institutional Review Board and was determined to be exempt from broad approval, as this project did not involve human subject research. Patient consent for publication was not required. All patients in the databases were deidentified, and the identities of data contributors were removed.

This study sought to develop and determine the accuracy of 4 HS PAs. The 4 PAs included 2 for incidence and 2 for prevalence, with one in each group having high sensitivity and specificity. Use of the PheValuator method allowed for estimation of sensitivity, specificity, PPV, and NPV without manual chart review. While both the incident and prevalent PAs were useful for the exploration of HS in observational databases, the PAs with definitions requiring just a single HS diagnosis code had lower specificity and higher sensitivity than the definitions requiring 2 codes, which had higher specificity and lower sensitivity. Thus, the choice of which algorithm to use is dependent on the research question being explored. For example, the use of a more sensitive algorithm would be applicable for safety studies, in which the PA is used to determine HS outcomes and missed identification of possible cases is problematic, whereas the use of a PA with higher specificity would be useful for treatment comparison studies, in which the goal is to ensure that all subjects exposed to a treatment have a high probability of having HS.

A few studies have included validation metrics for HS algorithms for observational databases [9,10,16,29,30]. Kim et al [16] used data available from the Massachusetts General Hospital and reported an increase in PPV with an increasing number of HS diagnosis codes (81% for 2 codes vs 97% for 5 codes). Our study replicated the Kim et al cohorts and found an increase in PPV with the use of 5 or more diagnosis codes compared to the use of at least two HS diagnosis codes (mean 84% for >5 codes vs mean 59% for 2 codes) that was similar to, albeit lower than, the published results. In general, our study found higher PPVs compared to studies that used a single HS diagnosis code [9,10,29,30]. The majority of subjects identified in our study were female, which is similar to findings from other studies [5,6,9,16,31]. A US study that used a cross-sectional design and a large electronic medical records database found an overall prevalence of 24.8% for type 2 diabetes, 71.6% for obesity, and 39.9% for hyperlipidemia among HS subjects [8]. Our study, when restricted to US data and examining covariates 365 days prior to and including the index date, identified type 2 diabetes in 26.5%, obesity in 19.6%, and hyperlipidemia in 26.5% of incident 1x HS subjects. The cross-sectional study was restricted to subjects aged 18 years or older, while our study included all ages, which may help in interpreting the decreased proportion of hyperlipidemia observed in our results. It has been reported that administrative databases underreport obesity as a diagnosis and are not an optimal data source for obesity prevalence [42]. This may support our finding of a lower prevalence of obesity compared to the findings of the cross-sectional study.

Strengths of our study include the use of a rigorous, data-driven approach for generating and evaluating the HS phenotypes across a data network that included 9 databases covering US and non-US countries. Network-based phenotype evaluations greatly strengthen the knowledge base for a given algorithm, because they allow the assessment of the consistency of findings across data types, geographic locations, and time periods. When concordant trends emerge, it increases confidence that the observations are the effect of the PA itself rather than an artifact of a particular data source. The PAs were analyzed using multiple approaches, providing ancillary verification of decisions made in determining the cohort logic. Our study includes several study artifacts, including JSON files for the PAs, computer code, and results for all the analyzed PAs, providing transparency in our interpretation of the results.

There were also several limitations to our study. We used administrative data sets primarily maintained for insurance billing, which are well-known to have significant deficits, including coding inaccuracies [43]. In addition, the estimation of performance characteristics using the PheValuator methodology was dependent on the quality of the data in the data set, which can vary substantially [37]. The algorithm validation was performed using a method involving predictive modeling of HS rather than case reviews. Results from PheValuator have been compared to results from previously published validation studies and have demonstrated excellent agreement [38]. This method does have the advantage of using multiple databases to provide a full set of performance metrics, including sensitivity and specificity, which are rarely provided in validation studies using case reviews [37]. The generalizability of our findings to uninsured populations is uncertain, given the insured population that was observed in this study. In the incident PA that defined HS with only a single diagnosis code, it was not possible to determine if any of these were “rule-out” diagnoses. The algorithms presented in this study use codes specific to HS; therefore, jurisdictions and practices that do not use these specific codes and instead use codes for “abscess” or “cyst” would be unable to operationalize these PAs. The study period used for evaluation of the HS algorithms includes the year (2015) when the drug Humira was introduced to treat HS [44]. Education on HS increased, and physicians became more likely to use diagnosis codes specifically indicating HS in observational data. Therefore, to avoid temporal bias, researchers should avoid use of these algorithms in data from prior to 2015.

This study developed and evaluated 4 HS PAs using a rigorous, data-driven approach and generated phenotype performance metrics including sensitivity, specificity, PPV, and NPV. Based on the analyses, we recommend that PAs requiring a single HS diagnosis code be used in studies requiring high sensitivity, while studies requiring high specificity should use PAs requiring 2 HS diagnosis codes. These algorithms will enable researchers to use large observational databases to research HS, which has a high burden of disease. There is a need for better evidence, as currently there are clinical knowledge gaps for HS that observational data is well suited to address.