Chapter 3

Research Methodology

This chapter presents the different elements of research that inscribes the factual

information in which the researchers considered significant in obtaining desirable result.

Research Design

This meta-analysis adapted a quantitative design to calculate the findings of

recent published articles found in online databases. This review technique provides

tools to determine the learning effect size of utilizing YouTube videos as a means to

improve vocabulary skills, considering a compilation of studies from a set of data

gathered within the criteria (Gijbels, Dochy, Van den Bossche & Segers, 2005). The

measurement of the effect size corresponded to the population of participants and the

given statistical values from previous studies. Such task enabled relativity from various

studies that can vary in population size and also in the conduct of the teaching strategy

(Gijbels, Dochy, Van den Bossche & Segers, 2005). This meta-analysis utilized a

thorough sequence to extract data that explicitly provide to the magnitude and direction

of the effect size of the mean.

In this study, the researchers gathered articles that tested the effectiveness of

Youtube videos to improve student vocabulary, sorted the studies that met the inclusion

criteria, extracted relevant data needed for the synthesis, and analyzed these data in

the Review Manager 5.4 (RevMan) tool.

Data Gathering Procedure

 16

The researchers had undergone a literature search among various databases

online. They browsed through reliable sites such as Google Search, Google Scholar,

ERIC, Academia and ResearchGate to identify relevant published studies with similar

variables from the desired study. Search keywords included “YouTube and vocabulary”,

“YouTube videos and vocabulary”, and “YouTube videos and vocabulary learning”. The

emergence of YouTube had become explicitly popular and effective as digitalization

continues to flourish. This platform is also a known repository of educational videos that

could be used as additional teaching materials. And as expected, several studies had

explored the effectiveness of these videos in the teaching and learning process.

After the articles were gathered, they were categorized based on established

inclusion criteria. It was limited to and only included studies who were closely similar to

each other in terms of respondent’s identity, skills developed, and result presentation to

provide a conclusive overall result. These studies should have explicit findings on the

effects of YouTube videos to the vocabulary learning of students. Specifically, the

criteria included the following:

 prior experimental studies that compared two study groups, with one serving

as control group and another as experimental group exposed to the teaching

intervention in question

 the result of the study presented per study group should include mean,

standard deviation and number of participants

The researchers also considered the year of publication and after thorough

selection, studies published from year 2010 up to present were included in the review.
 17

Only those who were deemed relevant and qualified after categorization were extracted

with the needed data.

A total of twenty eight (28)) studies were found after literature search. However,

after categorization twenty three (23) studies were excluded for not attaining the set

inclusion criteria. Sensibly, five (5) studies prevail with relevance to the given criteria.

Summarized information on the qualified studies were presented in a table.

Quantitative Data Synthesis

A table summarizing all the necessary information of the included studies were

presented in a table (see appendix). The authors, title, year of publication, participants,

intervention, outcomes, and statistical values, particularly the numbers of participants,

mean and standard deviation of the experimental and the control group, were reflected

in the said table. All these data were needed in the data synthesis of the review.

A software downloadable online was used in the analysis. Review Manager 5.4

(Rev.Man v.5.4, 2020) software functions to run and maintain a systematic review of

research or Cochrane Reviews (RevMan User Guide, 2020). This software can be

found on the official website of Cochrane Review organization. RevMan functions to

facilitate all procedures in systematic reviews including text, characteristics of studies,

comparison tables, study data input and management, meta-analysis of the data and

graphical presentation of results (Li, Higgins & Deeks, 2021). It was endorsed by

Cochrane because it ensured quality and efficacy in performing the meta-analysis plus

the inscription of a systematic review progressively (Li, Higgins & Deeks, 2021).
 18

The RevMan software generated a table displaying the statistical values inputted,

assigned weight percentage, and mean difference for each study (Li, Higgins & Deeks,

2021). The magnitude of effect sizes was computed based on the standardized mean

differences which was taken from the given means and standard deviations of prior

included studies (Ray & Shadish, 1996). The effect size is a simple way of quantifying

the difference between the advantages of two groups by using tests of statistical

significance (Coe, 2002). The test for overall effect Z measures the significance of the

overall effect sizes. The p-value provided from the Z test examined whether the pooled

estimate of effect is statistically significant. P-value less than 0.05 is universally deemed

as significant. A forest plot was also provided by the tool to graphically compare the

magnitude of effect size for the individual included studies and the overall effect size.

Other test included in the analysis was the test for heterogeneity level. The test

for heterogeneity level measured the variation in the study outcomes between studies

through I2 – chi square statistics. Random effect analysis model was assumed rather

than fixed effect to enable the variation on outcomes of studies and to explains the

outcome more logically (Ades & Higgins, 2005). Study differences is expected in terms

of sample populations (such as age), interventions received (implementation of

strategy), and follow-up length. Heterogeneity provides an estimate of the percentage of

variability instead of chance. If the I2 – chi square estimated was greater than or equal

to 50%, it indicates the manifestation of high levels of heterogeneity (Higgins & Green,

2011; Higgins, Thompson, Deeks, & Altman, 2003; Higgins & Thompson, 2002).