Professional Documents
Culture Documents
Firstly, the protocol was introduced: Title page, background, objective, methods (search strategy,
study selection, critical appraisal, data extraction, and data analysis), competing interests, timeline,
author’s contributions, acknowledgements, and appendices. Then, the literature search was conducted:
Definition search string(s) and display merged reference list. Once the documentary database was
filtered, the study selection was carried out: Define criteria list, consistency check, apply criteria,
and select studies from papers. The data were extracted using NVivo 12 Plus.
In the first phase, the inclusion criteria were applied to the title and abstract. In the second
phase, the inclusion criteria were applied to the full text. Two independent reviewers acted in the
different phases: (1) Selection based on inclusion criteria and (2) ultimate inclusion in the review.
In the first phase, a Cohen’s kappa coefficient [27] of 0.9 was obtained, considering an «almost perfect»
concordance index between the evaluators [28]. The discrepancies were resolved by later agreement.
As a result of the processing of the 790 articles selected initially, 498 of these were excluded for not
meeting the inclusion criteria, in terms of title and abstract. The full text of the remaining 292 articles
was evaluated, after which a total of 44 articles were excluded for being focused on «blended learning».
Therefore, the final sample consisted of 248 articles (Figure 3).