Evaluating Bias in Social Media Research Using #Sunscreen Content on Instagram Reels

Introduction

Social media has become a source of medical information for the general public, with Instagram and TikTok being among the most influential apps [1-4]. In response to the increasing presence of dermatology-related content, dermatologists have been encouraged to engage in social media to decrease misinformation [1-4]. As dermatology-related content grows on social media, more research is being done to assess its quality [1]. Concerns have been raised about the reliability and reproducibility of social media research, particularly due to the personalized nature of platform algorithms [1]. A 2024 study by Druskovich and Landriscina [1] highlighted how TikTok’s content curation can introduce bias and inconsistency into dermatology research.

This study aimed to assess whether account bias affects the type of dermatologic information presented in social media videos.

Methods

Overview

We used Instagram reels rather than TikTok amid increasing scrutiny of TikTok’s data privacy practices and the potential for a national ban in the United States. We compared Instagram reels videos between two user accounts: one with a biased profile based on prior dermatology engagement, and a new account with no activity or history. The biased profile was an existing Instagram account that had engaged extensively with dermatology-related content. This account followed multiple dermatologists and dermatology organizations and had previously liked, commented on, and shared dermatology posts, thereby establishing a history of dermatology-related activity.

For each account, the first 100 reels displayed under the hashtag #sunscreen were saved to prevent future content influence, resulting in a total of 200 videos. Each video was evaluated and categorized by creator type (dermatologist, non-dermatologist physician, esthetician, beauty blogger, etc) and content type (educational, advertisement, entertainment, etc). Creators were classified as dermatologists if they were verified to be board-certified through online professional profiles, institutional websites, or board certification databases. Engagement metrics, including views, likes, and comments, were recorded. By analyzing both content and engagement metrics, we sought to determine whether Instagram’s hashtag-based video results are influenced by user history or whether they offer an objective method for dermatology social media research. Data analysis was performed using R (version 4.3.1; R Foundation for Statistical Computing) with a significance value set to P<.05.

Ethical Considerations

This study was deemed exempt from ethics approval as it did not involve human participants or identifiable private information. The research was conducted in accordance with the ethical standards outlined in the U.S. Department of Health and Human Services, Code of Federal Regulations, Title 45, Part 46 [5]. All data analyzed were publicly available and fully deidentified prior to analysis, ensuring no individual could be identified directly or indirectly.

Results

Our study found there were no statistically significant differences in the video results between the two accounts. A χ² test assessing differences in content categories and creator types showed no significant variation between accounts (χ²_₆ = 7.6, P = 0.3; Table 1). A Monte-Carlo chi-squared test evaluating video types similarly found no significant differences (χ²_₆=19.9, P=0.8; Table 2). Engagement metrics for views, likes, and comments across all creator types were compared using two-sample t tests, and all P values exceeded 0.05. A paired t test comparing video engagement, calculated by adding the likes and comments and dividing by the view number, showed no difference between accounts (P=0.42; Table 1). These findings suggest that the Instagram reels hashtag system displays dermatologic content consistently and reproducibly, regardless of account history or engagement.

Discussion

These results challenge assumptions that social media research is inherently flawed due to account engagement history [1]. While Instagram curates personalized content based on user interaction, the hashtag system appears to present the same type of information regardless of account history. This has meaningful implications for research methodology, where Instagram hashtags may provide an objective tool for studying dermatologic content online, supporting more reproducible and rigorous investigations.

Our study is limited as it reflects a single time point and one hashtag on one platform. Future work should evaluate the longitudinal stability of Instagram’s algorithm, explore other hashtags, and compare results across broader time frames. Additionally, while the hashtag system offers a consistent sampling method for research, it may not fully reflect the way users typically consume content. Many users do not actively search via hashtags, and some creators, especially those with large followings, may not rely on hashtags for engagement, meaning their content may be underrepresented in hashtag-based analyses. Despite this, the hashtag system provides a practical and reproducible approach for evaluating dermatologic content for research.

As patients continue to turn to social media for skin health information, understanding how content is delivered empowers dermatologists to better counsel patients on how to navigate these platforms critically. These findings may reassure clinicians that accurate, evidence-based content has the potential to compete with less credible sources, especially when paired with effective use of tools like hashtags. Our data show that dermatologists did not receive the highest engagement rate (Table 1), highlighting the need for more dermatologists to create engaging social media content. Rather than dismissing social media as a space dominated by misinformation, dermatologists can leverage these insights to engage with patients and encourage more informed digital consumption.