| Home | E-Submission | Sitemap | Contact Us |  
J Mov Disord > Volume 14(2); 2021 > Article
Kim, Kim, Jun, Ahn, Jung, Minn, and Hwang: Content Analysis of Korean Videos Regarding Restless Legs Syndrome on YouTube

Abstract

Objective

To evaluate the accuracy and quality of Korean videos associated with restless legs syndrome (RLS) on YouTube.

Methods

A YouTube search was performed on April 1, 2020 using the term “restless legs syndrome” in the Korean language. Two reviewers coded the source, content, and demographics of the included videos. Video quality was assessed using the modified DISCERN (mDISCERN) instrument.

Results

Among the 80 videos analyzed, 44 (55.0%) were reliable, and 36 (45.0%) were misleading. There was a trend toward a higher number of mean daily views in the misleading videos than in the reliable videos. Most of the misleading videos (72.2%) advocated complementary and alternative medicine as a primary treatment for RLS. Although the reliable videos had higher mDISCERN scores than the misleading videos, the overall quality of the reliable videos was low.

Conclusion

Many Korean videos regarding RLS on YouTube involve a risk of exposure to misinformation and are of unsatisfactory quality.

Restless legs syndrome (RLS) refers to an urge to move the legs accompanied by uncomfortable and unpleasant sensations that typically occur at night and are improved by movement [1]. Although RLS has been reported to negatively affect sleep and quality of life, dopaminergic agents, including levodopa and short-acting dopamine agonists, generally relieve RLS-related symptoms and subsequently improve quality of life [2,3]. Accordingly, the correct diagnosis and appropriate management of RLS are crucial for every patient. However, RLS is often underrecognized, misdiagnosed, and poorly managed in actual medical settings [4].
With widespread internet use, patients are increasingly searching for online medical information [5]. One of the most popular and visited websites for health-related information is YouTube, which is a video-sharing platform [6]. Given that a video is worth a thousand words, there is no doubt that YouTube videos can be a good source of medical information and an effective educational tool for patients. However, the spread of misinformation online is a critical social issue that has the potential to provoke serious problems [7].
In the current study, we evaluated for the first time the accuracy and quality of Korean videos related to RLS on YouTube. We hypothesized that there would be considerable misinformation on RLS and that videos with misinformative content would have greater viewer engagement.

MATERIALS & METHODS

Search strategy and data collection

A YouTube (http://www.youtube.com) search was performed using the keyword “restless legs syndrome” in the Korean language on April 1, 2020, using the default filter of “relevance”. Since computer history and cookies can influence search results, these data were deleted before the search was conducted. We screened the first 100 videos, considering that internet users who seek medical help seldom go beyond the first few pages of any search result [8]. We excluded videos that met the following criteria: 1) unrelated to RLS (n = 11), 2) contained patient experiences (n = 2), and 3) were unavailable or soundless (n = 7). Finally, 80 videos were included in the analysis.

Video assessments

The included videos were evaluated independently by two neurologists (R. K. and J. S. J.) who were blinded to each other’s ratings before finishing their assessments. The videos were classified as either “reliable” or “misleading.” Reliable videos were defined as videos containing correct scientific information about RLS and not containing any inaccurate information. On the other hand, misleading videos were defined as videos containing inaccurate or scientifically unproven information. Any disagreement between the reviewers was resolved by discussion. Data on the total number of views, number of days since upload, video length, number of “likes” and “dislikes,” and upload source were obtained for each video on April 1, 2020. Upload sources were categorized as university hospitals, commercial entities, news agencies, or individual users. We classified videos uploaded by nonuniversity hospitals or medical websites as commercial entities. Each video was also evaluated for the presence of information about the following four content domains: epidemiology, etiology, symptoms and signs, and treatment.
We measured the quality of the videos using the modified DISCERN (mDISCERN) instrument [9], which is a 5 item scale adapted from the original 16 item DISCERN scale [10]. The total scores of the mDISCERN range from 0 to 5, with higher scores indicating a better quality of information. For this analysis, the mean value of the mDISCERN scores of the two reviewers was used. The video quality was regarded as low if the mDISCERN score was less than 3 [9].

Statistical analysis

The interrater agreement was measured using Cohen’s kappa statistics for the accuracy of the videos and the intraclass correlation coefficient (ICC) for the quality. The kappa and ICC values were considered to be in excellent agreement if they were over 0.75. The data are reported as the means, standard deviations, and frequencies. The normality of the data was tested using the Shapiro-Wilk test. To identify differences between the extracted variables, Student’s t-test, Mann-Whitney U test, a chi-square test, or Fisher’s exact test were performed, as appropriate. All p values were two-sided, and p < 0.05 was considered statistically significant. Calculations were performed using SPSS 25.0 (IBM Corp., Armonk, NY, USA).

RESULTS

Among 80 videos, 44 (55.0%) were categorized as “reliable”, and 36 (45.0%) were categorized as “misleading”. The raters had excellent interrater agreement for the accuracy of the videos (κ = 0.90). Table 1 summarizes the descriptive statistics of the included videos. The cumulative views were 101,139 views for reliable videos and 77,720 views for misleading videos. There was a trend toward a higher number of mean daily views for the misleading videos compared to the reliable videos, although this trend was not statistically significant (4.0 ± 5.7 vs. 2.2 ± 2.8; p = 0.061). Additionally, the misleading videos were significantly longer than the reliable videos (8.0 ± 7.9 min vs. 3.8 ± 3.9 min; p = 0.005). No significant group differences in the mean views, mean days since upload, or number of likes and dislikes were observed. There was a significant difference in upload source between the reliable and misleading videos (p = 0.013). Reliable videos were mainly uploaded by commercial entities (56.8%), followed by individual users (22.7%) and news agencies (15.9%). On the other hand, the misleading videos were mainly uploaded by commercial entities (58.3%) and individual users (41.7%).
Regarding video content, symptoms and signs were mostly covered in both reliable and misleading videos. However, the reliable videos provided more information about the epidemiology domain (p = 0.039), while the misleading videos provided more information about the treatment domain (p = 0.005). Misleading videos contained misinformation that was mostly related to the etiology and treatment of RLS. As a primary etiology of RLS, ten (27.8%) misleading videos advocated “poor blood circulation”, four (11.1%) advocated “constitution type”, and one (2.8%) advocated “oral breathing”. With respect to treatment, 26 (72.2%) misleading videos recommended complementary and alternative medicine (CAM). Misleading information regarding treatment consisted of “oriental medicine” (n = 11, 30.6%), followed by “acupuncture” (n = 9, 25.0%), “nasal breathing” (n = 4, 11.1%), “venesection” (n = 2, 5.6%), and “aroma therapy” (n = 1, 2.8%).
The reliable videos had significantly higher mDISCERN scores than those of the misleading videos (2.9 ± 1.0 vs. 1.3 ± 1.0; p < 0.001) (Table 2). However, 21 (47.7%) reliable videos had low quality (mDISCERN score < 3). The interrater agreement was excellent for the mDISCERN scores (ICC = 0.80). When we looked at the mDISCERN scales individually, the scores on the reliable sources of information (p < 0.001) and the balanced and unbiased information presented (p < 0.001) were significantly higher for the reliable videos than for the misleading videos. Among the reliable videos, high-quality videos were significantly longer (4.6 ± 4.4 min vs. 2.8 ± 3.2 min; p = 0.009) and provided more information about the etiology domain (95.7% vs. 47.6%; p < 0.001) than low-quality videos (Supplementary Table 1 in the online-only Data Supplement).

DISCUSSION

The main finding of the current study was that 45% of the examined Korean videos regarding RLS on YouTube provided incorrect or scientifically unproven information. This proportion is grossly similar to previous studies that have appraised the provided information on other medical fields on YouTube [11-16]. However, a recent study evaluating English-speaking YouTube videos on RLS found that 23% of the videos provided misleading information [17], which is relatively low compared to our outcomes. This discrepancy may be partly explained by the popularity of CAM in Korea, as described below.
As expected, there was a tendency for viewers to prefer watching videos containing misleading information on RLS. This trend has also been observed in several studies that have evaluated YouTube medical videos [12,13,15]. Although the number of views can be influenced by a variety of factors, video content is thought to be one of the important factors related to video popularity. In many cases, the video content can be inferred from the title before the video is viewed. The present study showed that a high proportion of the misleading videos included information on CAM. Despite the lack of evidence-based information on CAM, its use is prevalent in Korea [18,19]. One cross-sectional study reported that more than 70% of Korean adults used CAM in the last 12 months [18]. In this context, YouTube users are more likely to view misleading videos that include information related to CAM. Similarly, a previous study assessing Korean YouTube videos on Parkinson’s disease showed that videos with misleading information were more popular, and most of these videos advocated CAM [12]. Alternatively, it is possible that the video length affects the video popularity. Our results showed that misleading videos were significantly longer than reliable videos. However, considering that engagement decreases with video length [20], this hypothesis cannot explain our findings.
Although reliable videos had better quality than the misleading videos in this study, the overall quality of the reliable videos was insufficient. It is also necessary to support and ensure the quality of online health information. Even if online information is correct, the value of the information can differ according to its quality. To address this issue, we recommend that professional organizations make high-quality videos and increase the visibility of these videos among patients. Videos provided by such organizations would be valuable because their members are familiar with the consensus guidelines published in their respective fields. It would also be helpful if healthcare professionals provided diverse sources of high-quality online information to patients.
The current study has several limitations. First, the cross-sectional study design captured only YouTube videos at one time point. However, the videos on YouTube change over time because a considerable number of videos are uploaded or deleted daily. Second, since video searching was performed using the YouTube default setting, the results may vary depending on the type of setting. Third, some treatments may be classified as misleading because there is a lack of clinical trials and not because of their ineffectiveness. Accordingly, our results should be interpreted in conjunction with such a clinical situation. Fourth, we used a mDISCERN cutoff score of 3 to define low video quality, but this cutoff has not yet been validated. Further research is needed to identify the optimal mDISCERN cutoff score for video categorization according to quality. Finally, we only included Korean-language videos, which may limit the generalizability of the results. Despite these limitations, our study found that approximately half of the examined Korean RLS videos on YouTube provided misinformation and that many of the videos with misleading content advocated CAM. In addition, the reliable videos on RLS were less attractive and of unsatisfactory quality. Healthcare professionals should be aware of the limitations of YouTube and strive to increase the dissemination of accurate and qualified information about RLS.

Supplementary Materials

The online-only Data Supplement is available with this article at https://doi.org/10.14802/jmd.20137.

Supplementary Table 1.

Viewership and demographics according to quality in reliable videos
jmd-20137-suppl1.pdf

Notes

Conflicts of Interest

The authors have no financial conflicts of interest.

Author Contributions

Conceptualization: Ryul Kim, Jin-Sun Jun. Data curation: Ryul Kim, Jin-Sun Jun. Formal analysis: Joohwan Kim, Ryul Kim, Jin-Sun Jun. Funding acquisition: Ryul Kim. Investigation: Joohwan Kim, Ryul Kim, Jin-Sun Jun. Methodology: Ryul Kim, Jin-Sun Jun. Writing—original draft: Joohwan Kim, Ryul Kim. Writing—review & editing: Jin-Sun Jun, So-Hyun Ahn, San Jung, YangKi Minn, Sung Hee Hwang.

Acknowledgments

This work was supported by Inha University Hospital Research Grant.

Table 1.
Viewership and demographics of videos according to accuracy of information
Variables Reliable videos (n = 44) Misleading videos (n = 36) p value
Cumulative views 101,139 77,720 -
Mean views 2,299 ± 4,325 2,159 ± 3,754 0.879
Mean days since upload 958 ± 745 745 ± 633 0.238
Mean daily views 2.2 ± 2.8 4.0 ± 5.7 0.061
Mean length (min) 3.8 ± 3.9 8.0 ± 7.9 0.005
Mean likes 9.6 ± 15.3 13.4 ± 27.6 0.438
Mean dislikes 1.0 ± 1.8 1.5 ± 4.0 0.467
Upload source 0.013
 University hospital 2 (4.5) 0 (0.0) -
 Commercial entity 25 (56.8) 21 (58.3) -
 News agency 7 (15.9) 0 (0.0) -
 Individual user 10 (22.7) 15 (41.7) -
Content domains covered
 Epidemiology 18 (40.9) 7 (19.4) 0.039
  Misleading content - 1 (2.8) -
 Etiology 32 (72.7) 24 (66.7) 0.556
  Misleading content - 13 (36.1) -
 Symptoms and signs 38 (86.4) 34 (94.4) 0.231
  Misleading content - 2 (5.6) -
 Treatment 22 (50.0) 29 (80.6) 0.005
  Misleading content - 26 (72.2) -

Data are reported as n (%) or mean ± standard deviation.

Table 2.
The mDISCERN total scores and subscores according to accuracy of information
Variables Reliable videos (n = 44) Misleading videos (n = 36) p value
mDISCERN score 2.9 ± 1.0 1.3 ± 1.0 <0.001
mDISCERN questions
 Are the aims clear and achieved? 0.7 ± 0.5 0.5 ± 0.5 0.089
 Are reliable sources of information used? 0.8 ± 0.4 0.2 ± 0.4 <0.001
 Is the information presented balanced and unbiased? 0.9 ± 0.3 0.2 ± 0.4 <0.001
 Are additional sources of information listed for patient reference? 0.1 ± 0.3 0.1 ± 0.3 0.985
 Are areas of uncertainty mentioned? 0.3 ± 0.5 0.3 ± 0.4 0.890

Data are reported as mean ± standard deviation. mDISCERN: modified DISCERN.

REFERENCES

1. Trenkwalder C, Allen R, Högl B, Clemens S, Patton S, Schormair B, et al. Comorbidities, treatment, and pathophysiology in restless legs syndrome. Lancet Neurol 2018;17:994–1005.
crossref pmid
2. Winkelmann J, Allen RP, Högl B, Inoue Y, Oertel W, Salminen AV, et al. Treatment of restless legs syndrome: evidence-based review and implications for clinical practice (revised 2017). Mov Disord 2018;33:1077–1091.
crossref pmid
3. Cho YW, Hong SB, Kim DH, Lee HW, Joo EY, Kim JH, et al. The effect of ropinirole on the quality of life in patients with restless legs syndrome in Korea: an 8-week, multicenter, prospective study. J Clin Neurol 2013;9:51–56.
crossref pmid pmc
4. Cotter PE, O’Keeffe ST. Restless leg syndrome: is it a real problem? Ther Clin Risk Manag 2006;2:465–475.
crossref
5. Fox S, Duggan M. Health online 2013. Washington DC: Pew Internet & American Life Project; 2013.

6. Kim R, Kim HJ, Jeon B. The good, the bad, and the ugly of medical information on the Internet. Mov Disord 2018;33:754–757.
crossref pmid
7. Robledo I, Jankovic J. Media hype: patient and scientific perspectives on misleading medical news. Mov Disord 2017;32:1319–1323.
crossref pmid
8. Morahan-Martin JM. How Internet users find, evaluate, and use online health information: a cross-cultural review. Cyberpsychol Behav 2004;7:497–510.
crossref pmid
9. Langford B, Hooten WM, D’Souza S, Moeschler S, D’Souza RS. YouTube as a source of medical information about spinal cord stimulation. Neuromodulation 2021;24:156–161.
crossref pmid
10. Charnock D, Shepperd S, Needham G, Gann R. DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health 1999;53:105–111.
crossref pmid pmc
11. Ortiz-Martinez Y, Ali-Salloum W, González-Ferreira F, Molinas-Argüello J. HIV videos on YouTube: helpful or harmful? Sex Transm Infect 2017;93:481.
crossref pmid
12. Kim R, Park HY, Kim HJ, Kim A, Jang MH, Jeon B. Dry facts are not always inviting: a content analysis of Korean videos regarding Parkinson’s disease on YouTube. J Clin Neurosci 2017;46:167–170.
crossref pmid
13. Kumar N, Pandey A, Venkatraman A, Garg N. Are video sharing web sites a useful source of information on hypertension? J Am Soc Hypertens 2014;8:481–490.
crossref pmid
14. Pandey A, Patni N, Singh M, Sood A, Singh G. YouTube as a source of information on the H1N1 influenza pandemic. Am J Prev Med 2010;38:e1–e3.
crossref pmid
15. Garg N, Venkatraman A, Pandey A, Kumar N. YouTube as a source of information on dialysis: a content analysis. Nephrology (Carlton) 2015;20:315–320.
crossref pmid
16. Moon H, Lee GH. Evaluation of Korean-language COVID-19-related medical information on YouTube: cross-sectional infodemiology study. J Med Internet Res 2020;22:e20775.
crossref pmid pmc
17. Arikanoglu A, Demir M, Aluclu MU. Analysis of YouTube as a source of information for restless leg syndrome. Arq Neuropsiquiatr 2020;78:611–616.
crossref pmid
18. Lee JA, Sasaki Y, Arai I, Go HY, Park S, Yukawa K, et al. An assessment of the use of complementary and alternative medicine by Korean people using an adapted version of the standardized international questionnaire (I-CAM-QK): a cross-sectional study of an Internet survey. BMC Complement Altern Med 2018;18:238.
crossref pmid pmc
19. Choi B, Han D, Na S, Lim B. Factors related to the parallel use of complementary and alternative medicine with conventional medicine among patients with chronic conditions in South Korea. Integr Med Res 2017;6:223–229.
crossref pmid pmc
20. van der Meij H, van der Meij J, Voerman T, Duipmans E. Supporting motivation, task performance and retention in video tutorials for software training. Educ Technol Res Dev 2018;66:597–614.
crossref
Editorial Office
#2011 Mapo Hanwha Obelisk, 555, Dohwa-dong, Mapo-gu, Seoul, 04167, Rep. of KOREA
TEL: +82-70-8285-2913   
E-mail: editor@jmdeditorial.org, manage@jmdeditorial.org
Copyright © Korean Movement Disorder Society.                 Developed in M2PI