Journal of Clinical Neuroscience xxx (2018) xxx–xxx

Contents lists available at ScienceDirect

Journal of Clinical Neuroscience

journal homepage: www.elsevier.com/locate/jocn

Clinical study

The reliability of YouTube videos in patients education for Glioblastoma

Treatment
Karim ReFaey a, Shashwat Tripathi b, Jang W. Yoon a, Jessica Justice c, Panagiotis Kerezoudis d,
Ian F. Parney d, Bernard R. Bendok e, Kaisorn L. Chaichana a, Alfredo Quiñones-Hinojosa a,⇑
a
Department of Neurologic Surgery, Mayo Clinic, Jacksonville, FL, USA
b
University of Texas at Austin, College of Natural Science, Austin, TX, USA
c
University of Central Florida, Burnett School of Biomedical Sciences, Orlando, FL, USA
d
Department of Neurologic Surgery, Mayo Clinic, Rochester, MN, USA
e
Department of Neurologic Surgery, Mayo Clinic, Phoenix, AZ, USA

a r t i c l e i n f o a b s t r a c t

Article history: Background: Glioblastomas (GBMs) are one of the most devastating primary tumors in humans and often
Received 3 June 2018 results in minimal survival rates. Over the past 2 decades, patients have accessed the internet to obtain
Accepted 8 July 2018 information related to their diagnoses. In this study, we aimed to evaluate the accuracy and the reliability
Available online xxxx
of GBM-related YouTube videos.
Methods: In June of 2017, a search was conducted on YouTube using 6 keywords. Videos were sorted
Keywords: using ‘‘Relevance-Based Ranking” option, and the first 3 pages for each search were selected for further
YouTube
analysis. Three independent reviewers evaluated the videos using the validated DISCERN Tool.
Patient education
Glioblastoma
Results: After sorting 23,100 videos, 9 videos were identified and included for analysis. Of the 9 videos
Brain tumor analyzed, 88% (8/9) were from hospitals affiliated with prestigious universities across the country. Of
Quality of life the nine videos included in the analysis, two (22%) scored above a 3. There was an average 55% overlap
in the videos analyzed by key term and the keyword search of ‘‘Malignant Glioma Treatment” had the
highest percentage of videos above a score of 3 (66%).
Conclusion: Many patients with GBM and their families access information on YouTube to familiarize
themselves with the epidemiology, survival, and treatment options for this form of tumor. However,
the information that is currently available online is not monitored or vetted using an official filtering pro-
cess prior to its release. Medical institutions must work to produce more peer-reviewed content in order
to improve the availability of credible health information on internet platforms.
Published by Elsevier Ltd.

1. Introduction
Glioblastoma multiforme (GBM) is one of the most aggressive
primary tumors in adults [1,2]. Despite vigorous efforts and novel
treatment approaches to reduce their impact on mortality, the
median overall survival rate remains approximately 12–16 months
[3–6]. Therefore, as described in previously published articles [7–9],
quality of life (QoL) is an important factor in formulating a treat-
ment plan for patients with GBM [7–9]. In the new millennium,
the internet has become a vital and increasingly convenient source
for information for patients and their families [10]. YouTube is
considered one of the largest internet platforms with over one
⇑ Corresponding author at: Brain Tumor Stem Cell Laboratory, Department of
Neurologic Surgery, Mayo Clinic, Florida, 4500 San Pablo Rd. S, Jacksonville, FL
32224, USA.
E-mail address: quinones@mayo.edu (A. Quiñones-Hinojosa).
billion users [11] and is the second most visited website behind
the ‘‘Google” search engine [12]. Many patients and families access
YouTube to acquire more information and knowledge regarding
their disease and available treatment options [10]. Thus, there is a
continuous need to critically evaluate the quality of the YouTube
healthcare-related videos.
Previously published studies [13–17] have evaluated the
reliability and accuracy of online YouTube videos related to the
medical field such as femoroacetabular impingement [18]; move-
ment disorders [19]; methotrexate self-injection[20]; and pediatric
adenotonsillectomy and ear tube surgery [21]. The consensus from
these studies was that while the internet provides easy and
unlimited access to healthcare-related videos, the majority were
proven to consist of unreliable educational information [18,19].
Furthermore, there is little literature regarding the reliability and
accuracy of the YouTube video content covering GBM and other

https://doi.org/10.1016/j.jocn.2018.07.001
0967-5868/Published by Elsevier Ltd.

Please cite this article in press as: ReFaey K et al. The reliability of YouTube videos in patients education for Glioblastoma Treatment. J Clin Neurosci (2018),
https://doi.org/10.1016/j.jocn.2018.07.001
2 K. ReFaey et al. / Journal of Clinical Neuroscience xxx (2018) xxx–xxx
epidemiologically rare brain tumors [13–21]. In light of this
knowledge gap, the aim of this study is to evaluate the quality
and the reliability of GBM-related YouTube videos.
2. Materials and methods
2.1. Search strategy and data collection
The video-sharing website YouTube was queried in June of 2017
using the separate key terms ‘‘Glioblastoma Treatment, Glioblas-
toma Multiforme Treatment, GBM Treatment, malignant glioma
treatment”. Over 11,600, 4300, 5600, and 1600 results were iden-
tified from these searches, respectively. The YouTube search was
sorted by the ‘‘Relevance-Based Ranking” [22] option of videos
and the first three result pages of each search were assessed. Pre-
vious internet search engine analysis has shown that greater than
90% of users choose from results listed within the first three pages
of the search engine, which represented the top 1.6% of all videos
[23]. This study was exempt from Institutional Review Board
Approval as it involved the use of public access data only.
2.2. Inclusion and exclusion criteria
The inclusion criteria were adapted and modified from prior
studies [19,20]. Based on this modified criteria, 23,100 potential
videos were identified for potential investigation. Only official
videos uploaded by universities or hospitals and in English were
included and considered for further analysis. Videos with multiple
subparts were counted as one video. Duplicate, non-relevant
videos, or videos of magic or non-scientific treatment methods
were excluded from further analysis.
2.3. Variables extracted
Information collected for each video included: video ID, video
name, dates uploaded, viewership, likes, dislikes, and video
duration. Audience interaction with the video was assessed by
examining the total number of views and viewer ‘‘likability”, which
was calculated as the number of likes per day on the video. We also
determined the number of days online; views per day; likes per
day; likes per view; and average duration.
2.4. Scoring system
Videos were assessed independently by three authors made up
of two clinicians (K.R. and P.K.) and one non-clinician (Sh.T.) using
the DISCERN Tool [24]. DISCERN is a grading scale/rubric designed
to judge the quality and reliability of health information. Videos
are scored based on a five-point scale ranging from poor to moder-
ate to good. Evaluators select scores for fifteen points of criteria
that serve as quality indicators along with selecting an overall
quality rating. Each indicator represents a critical feature or stan-
dard that is considered an important characteristic of high quality
information.
Table 1
Assessment of videos.
Variable Keyword
Glioblastoma Multiforme treatment
Videos Found 11,600
Assessed 6 5
Rated as helpful 1 (0.17)
*
Search resulted in several overlaps between key term searches.
Please cite this article in press as: ReFaey K et al. The reliability of YouTube videos in patients education for Glioblastoma Treatment. J Clin Neurosci (2018),
https://doi.org/10.1016/j.jocn.2018.07.001
Videos with a DISCERN score greater than 3 were considered to
be ‘‘good” quality videos. According to the DISCERN Tool, a video
with a grade of 3 is ‘‘moderate” and of fair quality. Videos with a
score of 3 are useful sources, but ultimately the patient would need
additional sources of information. Videos with a score less than 3
were considered to be ‘‘poor” quality; these videos are not consid-
ered useful and should be avoided by patients. Continuous vari-
ables are summarized using medians with interquartile ranges
and categorical variables using frequencies with proportions. Vari-
ables were compared between helpful and unhelpful videos using
the Wilcoxon Rank Sum Test. Statistical analysis was conducted
using R Studio (Version 1.0.143). A biostatistician contributed in
the statistical analysis of this study (Sh.T.).
3. Results
Our search identified 11,600, 4300, 5600, and 1600 videos with
the separate key terms ‘‘Glioblastoma Treatment, Glioblastoma
Multiforme Treatment, GBM Treatment, Malignant Glioma Treat-
ment”, respectively (Table 1). After screening using our inclusion
and exclusion criteria, nine (0.039%) videos were identified and
selected to undergo further analysis. There was an average of
55% overlap between the keyword searches. The oldest video
included was uploaded in 2010 (Table 2). 88% (8/9) of videos ana-
lyzed were uploaded by university hospitals (Table 2). 11% (1/9) of
videos were not published by hospitals or academic institutions,
with one video uploaded by a private medical institution (Table 2).
Additionally, there was no correlation observed between video
duration and the number of views. Of the nine videos included in
the analysis, only two (22%) scored above a 3 out of 5 (Tables 1
and 2). The inter-grader reliability was 0.60 with a maximum of
3 point scoring difference. Of the four key term searches,
‘‘Malignant Glioma Treatment” had the highest percentage of
‘‘good” quality videos (66%) (Table 1). While the average DISCERN
score was 4.06 out of 5 in terms of a clear purpose for the videos,
discussion related to the risks and consequences of not receiving
treatment scored an average of only 1.88 out of 5. Common themes
across the videos included brief discussions of diagnosis followed
by limited information on disease etiology; surgical intervention
options; and potential prognosis. In congruence with other studies,
our results demonstrated that common video-sorting criteria of
likes, duration, views, and age had no effect on the quality of the
video (Table 3).
4. Discussion
The internet has become a key catalyst in the distribution of
readily accessible information to the general public [10,15,25,26].
YouTube has been constantly changing over the 12 years of its
existence. It has become increasingly consumer-friendly, easily
accessible, and free; therefore, many patients and families rely
on YouTube videos to obtain information on disease diagnosis
and management. There are previously published studies [13–17]
evaluating the reliability and accuracy of these online resources
regarding different epidemiological diseases [18–21]. However,
Total
Glioblastoma Treatment GBM treatment Malignant Glioma Treatment
4300 5600 1600 23,100
6 3 9*
1 (0.20) 2 (0.33) 2(0.66) 2 + (0.22)
 K. ReFaey et al. / Journal of Clinical Neuroscience xxx (2018) xxx–xxx 3

Table 2
Video list.

YouTube ID Video Info

URL Uploader Grade [Mean (Range)] Views Likes Duration Upload Date
https://youtu.be/GHRaMDd2wQc North Shore University Health System 3 (3–4) 30,727 80 4.72 Aug 17, 2010
https://youtu.be/_fGXQeS0TkM Mayo Clinic 2 (1–4) 5151 12 2.20 Nov 8, 2012
https://youtu.be/_HpQmqUBd8M SunRise Hospital and Medical Center 2 (2–2) 2789 3 1.35 Apr 17, 2015
https://youtu.be/4WSU1ZuGbds MD Anderson Cancer Center 3 (2–4) 362 6 1.78 Jul 18, 2017
https://youtu.be/Za2hGIiZ2ws UC Irvine Health 4 (4–5) 903 9 11.30 Dec 30, 2016
https://youtu.be/UHmQnGsWiIM Duke Health 3 (2–5) 25,098 94 2.92 Jul 28, 2013
https://youtu.be/sg9K7lLUank Medicine Virginia 2 (1–4) 1206 4 2.22 Nov 23, 2015
https://youtu.be/gem6STxwMKs UNC Eshelman School of Pharmacy 2 (1–3) 1725 7 7.55 Feb 1, 2017
https://youtu.be/q44E0DzLfFI Brigham and Women’s Hospital 4 (4–5) 658 2 2.42 Jan 14, 2015

Table 3
Analysis of variables.
Variable Video Classification
Discern Score >3 Discern Score <3
[Mean (SEM)*] [Mean (SEM)*]
Duration 6.86 (4.44) 3.25 (0.83)
Likes Per Day 0.014 (0.011) 0.022 (0.0078)
Upload Days 560 (202) 1253 (335)
Views 781 (123) 9580 (4807)
*
Standard error of the mean.
there are no studies evaluating the accuracy and the reliability of
the videos related to GBM treatment on YouTube.
Based on the results gathered from our YouTube search, we
found that there is limited amount of high quality and reliable
video content about glioblastoma multiforme treatment available
for patients and families to view (Table 1). Several prior studies
related to the effectiveness of YouTube medical videos recommend
that medical institutions publish videos in order to increase the
amount of accurate medical information available. On the contrary,
our study found that videos published by medical institutions on
GBM were insufficient with only 22% receiving above a 3 on the
DISCERN scoring scale. Criteria such as risk of each treatment; con-
sequence of no treatment; effect of treatment choice on quality of
life; and support for shared-decision making are not often included
in the videos.
When the medical institutions publish videos on different dis-
eases and treatments, they might consider including all the infor-
mation needed for patients to make well-informed decisions.
While academic medical institutions face the challenge of convinc-
ing the public to watch their videos in lieu of others, which are
often flashier and have more views, these institutions might recog-
nize the importance of ensuring a high level of quality and com-
pleteness for their videos. One-way to improve the quality of
videos published by institutions is through the publishing of only
peer-reviewed videos that are in accordance with the DISCERN
scoring scale.
There are a few limitations within our study. This was a cross-
sectional study that highlighted only a limited amount of the infor-
mation available on YouTube. Despite the significant number of
videos (23,100 videos) that resulted from the use of the different
key terms, only a small number of videos (9 videos) were included
in our analysis. The sensitivity of our study would likely increase
with a larger video sample size. Only English videos were selected
for evaluation, which may have negatively impacted our results.
YouTube is a dynamic video platform that continuously changes
based on video popularity and the time of posting. The proprietary
algorithm of the YouTube search may lead to a variation in the
search results according to geographic location. For example,
because this search was performed in the United States of America,
search results would likely have generated a different list of videos
from other countries in which a similar search is conducted using
the same key terms. Additionally, our study exclusively focused on
the YouTube search content and neglected video contents on other
P-Value
websites that might be accessed by the patients.
0.33
0.50
5. Conclusion
0.50
0.22
With the increasing use of social media and the internet,
patients with GBM and their families are turning to online
resources to obtain healthcare information. YouTube is considered
one of the most searched sources of information by both patients
and the general population. However, videos for patients with
GBM may be inaccurate and/or misleading, and neither the
patients nor their families may be able to differentiate between
the useful and misleading information. Therefore, it is important
for clinicians and institutions alike to recognize the potential
impact of incomplete or unreliable YouTube videos on the patients
and their families’ perception and understanding of this disease, in
addition to its potential influence on the patient-physician rela-
tionship. With only 22 percent of videos within our sample size
being rated as helpful, our study recommends that academic and
private medical institutions and hospitals publish more peer-
reviewed informational videos that are in line with the DISCERN
scoring scale prior to uploading publicly accessible content on
mass search engines such as Youtube.
References
[1] Chaichana KL, Garzon-Muvdi T, Parker S, Weingart JD, Olivi A, Bennett R, et al.
Supratentorial glioblastoma multiforme: the role of surgical resection versus
biopsy among older patients. Ann Surg Oncol 2011;18:239–45.
[2] Wrensch M, Minn Y, Chew T, Bondy M, Berger MS. Epidemiology of primary
brain tumors: current concepts and review of the literature. Neuro Oncol
2002;4:278–99.
[3] Chaichana K, Parker S, Olivi A, Quinones-Hinojosa A. A proposed classification
system that projects outcomes based on preoperative variables for adult
patients with glioblastoma multiforme. J Neurosurg 2010;112:997–1004.
[4] Adams H, Adams HH, Jackson C, Rincon-Torroella J, Jallo GI, Quinones-Hinojosa
A. Evaluating extent of resection in pediatric glioblastoma: a multiple
propensity score-adjusted population-based analysis. Childs Nerv Syst
2016;32:493–503.
[5] Almeida JP, Chaichana KL, Rincon-Torroella J, Quinones-Hinojosa A. The value
of extent of resection of glioblastomas: clinical evidence and current approach.
Curr Neurol Neurosci Rep 2015;15:517.
[6] Chaichana KL, Jusue-Torres I, Lemos AM, Gokaslan A, Cabrera-Aldana EE,
Ashary A, et al. The butterfly effect on glioblastoma: is volumetric extent of
resection more effective than biopsy for these tumors? J Neurooncol
2014;120:625–34.
[7] Bottomley A. The cancer patient and quality of life. Oncologist 2002;7:120–5.
[8] Carr AJ, Gibson B, Robinson PG. Measuring quality of life: is quality of life
determined by expectations or experience? BMJ 2001;322:1240–3.
[9] Eseonu CI, Rincon-Torroella J, ReFaey K, Quinones-Hinojosa A. The cost of brain
surgery: awake vs asleep craniotomy for perirolandic region tumors.
Neurosurgery 2017;81:307–14.
[10] Underhill C, McKeown L. Getting a second opinion: health information and the
Internet. Health Rep 2008;19:65–9.

Please cite this article in press as: ReFaey K et al. The reliability of YouTube videos in patients education for Glioblastoma Treatment. J Clin Neurosci (2018),
https://doi.org/10.1016/j.jocn.2018.07.001
4 K. ReFaey et al. / Journal of Clinical Neuroscience xxx (2018) xxx–xxx
[11] YouTube. Statistics. https://www.youtube.com/yt/about/press/ [Accessed
August 2017].
[12] Alexa. Google.com Alexa Traffic Rank. http://www.alexa.com/siteinfo/
youtube.com [Accesed August 2017].
[13] Samuel N, Alotaibi NM, Lozano AM. YouTube as a source of information on
neurosurgery. World Neurosurg 2017;105:394–8.
[14] Elkarim GA, Alotaibi NM, Samuel N, Wang S, Ibrahim GM, Fallah A, et al. Social
media networking in pediatric hydrocephalus: a point-prevalence analysis of
utilization. J Neurosurg Pediatr 2017;20:119–24.
[15] Alotaibi NM, Samuel N, Wang J, Ahuja CS, Guha D, Ibrahim GM, et al. The use of
social media communications in brain aneurysms and subarachnoid
hemorrhage: a mixed-method analysis. World Neurosurg 2017;98:456–62.
[16] Tenyi D, Gyimesi C, Horvath R, Kovacs N, Abraham H, Darnai G, et al.
Concussive convulsions: a YouTube video analysis. Epilepsia 2016;57:
1310–6.
[17] Rossler B, Lahner D, Schebesta K, Chiari A, Plochl W. Medical information on
the internet: quality assessment of lumbar puncture and neuroaxial block
techniques on YouTube. Clin Neurol Neurosurg 2012;114:655–8.
[18] MacLeod MG, Hoppe DJ, Simunovic N, Bhandari M, Philippon MJ, Ayeni OR.
YouTube as an information source for femoroacetabular impingement: a
systematic review of video content. Arthroscopy 2015;31:136–42.
Please cite this article in press as: ReFaey K et al. The reliability of YouTube videos in patients education for Glioblastoma Treatment. J Clin Neurosci (2018),
https://doi.org/10.1016/j.jocn.2018.07.001
[19] Stamelou M, Edwards MJ, Espay AJ, Fung VS, Hallett M, Lang AE, et al.
Movement disorders on YouTube–caveat spectator. N Engl J Med
2011;365:1160–1.
[20] Rittberg R, Dissanayake T, Katz SJ. A qualitative analysis of methotrexate self-
injection education videos on YouTube. Clin Rheumatol 2016;35:1329–33.
[21] Sorensen JA, Pusz MD, Brietzke SE. YouTube as an information source for
pediatric adenotonsillectomy and ear tube surgery. Int J Pediatr
Otorhinolaryngol 2014;78:65–70.
[22] Andrei Serbanoiu TR. Relevance-based ranking of video comments on
YouTube. In: 19th international conference on control systems and
computer science; 2013.
[23] iProspect Search Engine User Behaviour Study. http://district4.extension.
ifas.ufl.edu/Tech/TechPubs/WhitePaper_2006_SearchEngineUserBehavior.pdf
[Accessed August 2017].
[24] Charnock D, Shepperd S, Needham G, Gann R. DISCERN: an instrument for
judging the quality of written consumer health information on treatment
choices. J Epidemiol Community Health 1999;53:105–11.
[25] Heo J, Chun M, Lee HW, Woo JH. Social media use for cancer education at a
community-based cancer center in South Korea. J Cancer Educ 2016.
[26] Murugiah K, Vallakati A, Rajput K, Sood A, Challa NR. YouTube as a source of
information on cardiopulmonary resuscitation. Resuscitation 2011;82:332–4.