Comparison of Traditional Citation Metrics and Altmetrics Among Dermatology Journals: Content and Correlational Analysis Study

Introduction

Research impact has traditionally been measured using citation count and impact factor (IF). Academics have long relied on this form of metric system to measure a publication’s impact [1]. A higher number of citations is viewed as an indicator of the importance of the research and a marker for the impact of the publishing journal [2,3]. More recently, social media and online news sources have become important avenues for dissemination of research, resulting in the emergence of an alternative metric system known as altmetrics [4,5]. Altmetrics is data that can explain both the nature and volume of attention that research receives. It measures how many people have engaged with and shared research and allows the researchers or publishers to see in what manner it was shared.

Numerous altmetrics harvesting tools have recently been developed, including the Altmetric Explorer. Altmetric Explorer is a Web-based platform that enables users to browse and report on all attention data for every piece of scholarly content for which the platform has found attention [5]. The system tracks the online attention research receives by aggregating data from numerous avenues of online sharing sources, such as public policy documents, Mendeley, PubMed, mainstream media and social media outputs like Facebook and Twitter, and many more [5]. An altmetric attention score (AAS), derived by an automated algorithm, indicates the amount of attention and, in some cases, public engagement that research has received [5]. Therefore, articles that generate more attention are likely to have higher scores [6]. The AAS is based on three main principles: Volume, sources, and authors [5]. In addition, each form of mention contributes a different amount to the final score; for example, if an article is mentioned and shared via an online newspaper, it receives a higher score than if it is mentioned in a single tweet [5].

The aim of this study was to investigate the relationship between altmetrics and traditional citation metrics among dermatology journals and articles. Does sharing research online via social media and other news sources correlate with article citation counts and journal IFs? We assessed the correlation between AAS and traditional scientific impact markers, namely journal IF and article citation count, for all the dermatology journals and published articles of 2017. 

Methods

We identified dermatology journals available in 2017 and their associated IFs using InCites Journal Citation Reports [7]. We entered all 64 dermatology journals into the Altmetric Explorer search platform, and the search year was limited to 2017. This generated the AAS for each journal and its published articles. Using Dimensions, an online research database that provides the current citation count for published research, we gathered the citation counts for each published article [8]. We applied the Spearman rank correlation coefficient to assess the correlation between dermatology journal IF and AAS as well as dermatology article citation count and AAS. Statistical analysis was performed using SPSS V21 (IBM Corporation, Somers).

Results

From the Journal Citation Report 2017, there were 64 dermatology journals with IFs ranging from 8.1 to 0.08. Of these journals, there was a moderate positive correlation between journal IF and journal AAS (r_s=.513, P<.001; Figure 1). From these journals, 6323 articles were published in 2017. Our data show a weak positive correlation between traditional citation count and AAS (r_s=.257, P<.001; Figure 2).

‎View this figure

‎View this figure

Discussion

Altmetrics complements, rather than replaces, the traditional citation metric system. Researchers and publishers can see not only how their research is distributed online but also in what form it is distributed, either through social media platforms such as Twitter and Facebook or via news articles. Our data demonstrate a moderate correlation between journal IF and AAS among dermatology journals, suggesting that the higher IF dermatology journals have a larger impact across social media and mainstream media. Our data also show a weak correlation between article citation count and AAS. Temporal factors may explain this weak association: Newer articles may receive increased online attention after publication, while it may take longer for scientific citation counts to accumulate. The weak correlation between the number of article citations and AAS in our study is consistent with recently published work of weak correlations between AAS and both cardiology and pediatric surgery citation counts [4,9]. These findings suggest that articles achieving high AAS, and therefore online attention, are not of equivalent interest to academics.

It is also important to note the occasional disconnect between what are deemed newsworthy publications and comprehensive academic research. Altmetrics reflects online crowd attention but does not reflect the quality, validity, and originality of research. The obverse could be said about traditional citation metrics, which focus on quality and validity but not dissemination of new research. For these metrics, the journal reach is limited by subscription fees or firewalls.

The near instantaneous nature of the altmetric score, with the ability for an article to go viral and a corresponding rapid boost in the AAS, enables the gaming of altmetrics. With enough time, coupled with the speed of the internet and freedom to post, research AAS could be artificially inflated by repeated tweets and widespread dissemination of research online. However, altmetrics addresses this issue by weighing author contribution with each mention to the overall attention score.

A limitation of this study was the exclusion of non-dermatological journals such as Nature, New England Journal of Medicine, and the Lancet, which also publish highly cited dermatological articles.

To our knowledge, this is the first study to assess the relationship between dermatology journal citation metrics and AAS.