Journal Pre-proof

Advance Organizers in Flipped Classroom via E-learning Management

System and the Promotion of Integrated Science Process Skills

Abdellah Ibrahim Mohammed Elfeky, Thouqan Saleem Yakoub

Masadeh, Marwa Yasien Helmy Elbyaly

PII: S1871-1871(19)30283-4
DOI: https://doi.org/10.1016/j.tsc.2019.100622
Reference: TSC 100622

To appear in: Thinking Skills and Creativity

Received Date: 18 September 2019

Revised Date: 21 November 2019
Accepted Date: 25 November 2019

Please cite this article as: Elfeky AIM, Masadeh TSY, Elbyaly MYH, Advance Organizers in
Flipped Classroom via E-learning Management System and the Promotion of Integrated
Science Process Skills, Thinking Skills and Creativity (2019),
doi: https://doi.org/10.1016/j.tsc.2019.100622

This is a PDF file of an article that has undergone enhancements after acceptance, such as
the addition of a cover page and metadata, and formatting for readability, but it is not yet the
definitive version of record. This version will undergo additional copyediting, typesetting and
review before it is published in its final form, but we are providing this version to give early
visibility of the article. Please note that, during the production process, errors may be
discovered which could affect the content, and all legal disclaimers that apply to the journal
pertain.

© 2019 Published by Elsevier.

 1

Advance Organizers in Flipped Classroom via E-learning Management System and the

Promotion of Integrated Science Process Skills

Abdellah Ibrahim Mohammed Elfeky, Thouqan Saleem Yakoub Masadeh, and Marwa
Yasien Helmy Elbyaly

Abdellah Ibrahim Mohammed Elfeky is an associate professor at the department of educational

of
technology at Kafrelsheikh University in Egypt. Nowadays he is working at the department of
curricula and instruction at Najran University in Saudi Arabia. His research interests involve
the use of computer and multimedia in teaching, current methods of teaching, and teacher

ro
professional development and preparation. Besides, the use of social media, mobile learning,
augmented reality, CSCL, and virtual reality in teaching.
Thouqan Saleem Yakoub Masadeh is an associate professor of English Language Curriculum

-p
and Instruction at Najran University in Saudi Arabia. His research interests include TEFL/
TESL, teacher preparation and professional development, methods of instruction and technology
adoption at various learning environments.
re
Marwa Yasien Helmy Elbyaly is an associate professor of home economics at Kafrelsheikh
University in Egypt. Nowadays she is working at the department of home economics at Najran
University in Saudi Arabia. Her research interests involve fashion design, handmade
lP

embroidery, pattern-making, online teaching, and learning object repository. In addition, the use
of social media, mobile learning, virtual reality, CSCL, and augmented reality in teaching.
na

Address for correspondence: Dr Abdellah Elfeky, Faculty of Education, Najran University, P.O.
Box: 1988, Najran, Saudi Arabia. Email: abdalah.elfeqi@spe.kfs.edu.eg
ur

Highlights
 This study examined the effectiveness of advance organizers in flipped classroom in
Jo

developing the performance.

 Advance organizer in flipped classroom has helped home economics students to engage

in active learning.

 Advance organizer in flipped classroom has developed students' integrated science

process skills.
 2

 Findings have implications for educational technology development.

Abstract

Flipped Classroom (FC) has been considerably investigated in research and practice contexts

because of the capacity it assumes to have in enhancing learners’ outcomes. However, research

that explores advance organizers' impact, which might result in affecting the expected learning

of
outcomes in the FC, is still insufficient. Hence, the present study aimed to assess the

effectiveness of advance organizers utilization in FC via Learning Management System to

ro
develop learners’ integrated science process skills. It was carried out on all home economics

students enrolled in a course entitled "Research Proposal" who were randomly divided into two

-p
equal groups, experimental and control. Data were collected through a five- domain product
re
evaluation card namely, the operational definition; identifying and controlling variables;

formulating hypothesis; experimenting; and interpreting data. Results showed that achievement
lP

and performance of participants who learnt the course content via FC lectures with the use of an

advance organizer were better than the performance and achievement of their colleagues in the
na

control group that were not provided by any advance organizer. In light of these results, the study

concluded that advance organizers, if used in FC could have interesting benefits for research and
ur

development of learners’ integrated science process skills.

Jo

Keywords: Advance Organizers; Flipped Classroom; Blackboard system; Learning Management

System; Integrated Science Process Skills

Introduction
 3

Delivery of a recorded lecture video with an advance organizer can provide learners with

an overall and comprehensive vision about the lecture topic. It also helps them identify,

understand, and follow the lecture’s main components. Advance organizers are educational tools,

which help connect prior knowledge with new information, which can considered as a critical

step in making learning meaningful (Cutrer, Castro, Roy, & Turner, 2011). Recently, it can be

claimed that educators have promoted the use of flipped classrooms (FCs) by giving their

students more opportunities to them in active learning process because of the role such classes

of
play in encouraging participation in interactive and higher-order activities (Chuang, Weng, &

ro
Chen, 2018).

Studies conducted on this field are really varied. Proponents of FCs, for instance, argue

-p
that such classes have numerous merits, like for example, the enhancement of teacher-student
re
interactions, learning facilitation through active learning activities in classroom (McLean,

Attardi, Faden, & Goldszmidt, 2016). Other studies discus the benefits of FCs such as the
lP

improvement in students’ performance, outcomes, active learning, and higher-order thinking

skills (L. J. Day, 2018; Lewis, Chen, & Relan, 2018; McLaughlin et al., 2014; Tune, Sturek, &
na

Basile, 2013). Another kind of studies looked for the effectiveness of utilizing FC once it is

related to a specific developed learning strategy of flipped model (Chen, Wang, & Chen, 2014;
ur

L. Day, 2017). In addition, "Research Proposal" course is a curricular requirement for students in

home economics department and depends mainly on students' skills in operational definition,
Jo

identifying and controlling variables, formulating hypothesis, experimenting, and interpreting

data. It's main aim was to fulfill students' needs through developing their integrated science

process skills (Elfeky & Elbyaly, 2017).

 4

Therefore, the present study aims to examine how utilization of advance organizers can

contribute to the effectiveness of FC via learning management system outcomes. In other words,

it seeks to identify the extent to which advance organizers in FC can promote learners’ integrated

science process skills mainly in developing their skills in operational definition, identifying and

controlling variables, formulating hypothesis, experimenting, and interpreting data. Then, the

following research question was put forth:

RQ: What is the influence of using Advance Organizers in FC on the promotion of students'

of
Integrated Science Process Skills, i.e. operational definition, identifying and controlling

ro
variables, formulating hypothesis, experimenting, interpreting data, and as a whole?

Theoretical framework

-p
The present study tries to address the impact of integrating three main concepts together
re
(advance organizers, flipped classroom and learning management system) while teaching an

academic course entitled " Research Proposal" on students skills i.e. integrated science process
lP

skills at the end of this course. First, the term advance organizer is defined "as a teaching activity

that helps building or promoting learning" (C.-H. Li, Wu, & Lin, 2017). It is a bridging strategy
na

to connect one unit to another (Malik (2015) and making a linkage between one's newly acquired

information with his existing cognitive structure. An advance organizer is also known as "a
ur

techniques for powerful design used to link prior knowledge to newly learnt material" (S. Li,

2016; Mills, 2017). It is mainly used to activate the learner’s prior knowledge to better
Jo

understand analogy being studied (Zheng, Yang, Garcia, & McCadden, 2008). Moreover, any

advance organizer involves a set of actions like, for example, providing background knowledge,

making explicit links between past learning and new knowledge, pre-teaching new vocabulary in

context, and highlighting key vocabulary (S. Li, 2016). The main aim of utilizing advance
 5

organizers in teaching is to help learners make unfamiliar information more plausible making

their main aim to dominate opinions and information because most concepts and principles are

directly presented to them (Amouzeshi, Mohsenizadeh, & Amouzeshi, 2015). Therefore, it can

be strongly claimed that advance organizers are generally used in flipped classrooms to clarify

and interpret ideas presented to students before the introduction of new knowledge and

information. Via the utilization of an advance organizer, a teacher can provide his students with a

conceptual framework that makes learning and knowledge retention easier. Thus, learners can

of
first organize their old knowledge and familiar ideas and then connect any new idea or

ro
knowledge to these ideas (Jenkins et al., 2017).

Second, a flipped classroom can be considered as "a newly developed teaching method in

-p
and mostly used with undergraduates" (Foster & Stagl, 2018). The main aim of using such
re
teaching strategy is to make acquisition of new ideas and ways of innovation via the use of

university pedagogical mode more easier (Zhu & Xie, 2018). Looking at a flipped classroom as a
lP

pedagogical model, we can say that modern technologies such as e learning management systems

can be used to deliver pre- prepared lessons. Therefore, video and audio lectures, in addition to
na

any other media, can be delivered to students outside classrooms via their smart phones, tablets,

or PCs making learning self-directed. However, self-directed learning in FCs requires learners to
ur

be fully prepared and ready before the start of face-to-face lecture or lessons inside classrooms

(Barrie, Amick, Mitzman, Way, & King, 2018). Henceforth, a flipped classroom strategy can be
Jo

regarded as good alternative not only to traditional classroom or lecture but also to most

traditional teaching methods and so it has been widely used by educators at all levels of

education and across many disciplines (King, Mayer, Barrie, Greenberger, & Way, 2018; Steen-

Utheim & Foldnes, 2018). One interesting reason for the adoption of flipped classroom strategy
 6

in teaching is due to its ability to increase students’ active learning, collaboration and scaffolding

during the learning process, through better allocation of teaching time (Sergis, Sampson, &

Pelliccione, 2018). Hence, face-to-face sessions that follow the utilization of advance organizers

become student-centered devoting time to discussion, group work, and problem-solving

(Pickering & Roberts, 2018). One more interesting that sheds light on the importance of using

flipped classroom us the fact that the role of the teacher can be shifted to be a guide and

facilitator (Lopes & Soares, 2018). Furthermore, information transfer outside classroom can take

of
the form of pre-class assignments that are expected to be completed by students before starting

ro
actual class time (Miller, Lukoff, King, & Mazur, 2018). That is, class time can be invested on

providing learners with unique learning experiences within collaborative activities with their

-p
classmates as well as receiving support by their professor instead of instead of investing it on
re
lecturing (DeLozier & Rhodes, 2017).

Third, Learning Management System (LMS), can involve software applications and web-
lP

based technologies utilized by faculty and learners to either access, plan, implement, supplement,

monitor, and assess learning or to communicate about learning (Elfeky, 2018; Masadeh &
na

Elfeky, 2016). LMS has many types like Moodle, Canvas, Blackboard, and Desire2Learn that all

have become key components in implementing FC teaching (AlJarrah, Thomas, & Shehab,
ur

2018). Henceforth, LMS has become a tool for creating, distributing, tracking, and managing

various types of training and educational material (Mershad & Wakim, 2018), which is widely
Jo

utilized in higher education and offer a gateway to innovative technology- promoted teaching

and learning (Sinclair & Aho, 2018). Digitizing and automating many of the traditional campus

teaching functions and pedagogical activities is actually one of the numerous merits of LMS

(Anderson & Dron, 2018). Thus, students can use LMS to try many learning resources offered by
 7

their instructor and meanwhile interact with their colleagues (Homavazir & Gopal, 2018).

Besides, LMS has some ability to hold course content and e-portfolios, provide chatting and

forum discussions, give online quizzes, log and summarize grades, provide schedule and

calendar reminders, and handle multimedia files such as images, music and voices, flash

presentations, and videos (Byrd, 2018). In short, learners can be engaged in home study through

the integration between flipped classroom strategy and learning management system where a

recorded lecture video can be introduced to help learners develop the power of self-assessment

of
power (Lopes & Soares, 2018). Learning management system in flipped class can involve many

ro
out-of-class and in-class activities like role-play, debates, quizzes, and group presentations

(O'Flaherty, Phillips, Karanicolas, Snelling, & Winning, 2015). In short, unlike traditional

-p
paradigm of teaching, flipped classroom activities and homework are reversed (Lopes & Soares,
re
2018) and the most popular tools of the learning management system are content dissemination,

grade management and assignment (Anderson & Dron, 2018).

lP

Fourth, most of people believe that the main role and function of science is to

systematically explore nature (Prajoko, Amin, Rohman, & Gipayana, 2016). Consequently,
na

science process skills can be either procedural that experiment and investigate habits of one's

mind, or can take the form of scientific inquiry abilities (Zeidan & Jayosi, 2015). Science process
ur

skills are those skills that allow students to gain knowledge and consequently understand it"

(Hadi, 2016) and in the same time they are those basic abilities needed for science use and
Jo

understanding (P. Sari, F. Sudargo, & D. Priyandoko, 2018). They are of two main forms. The

first form refers to the basic skills related to conducting empirical-inductive reasoning and the

second form represents the integrated process skills associated with performing hypothetic-

deductive reasoning (P. M. Sari, F. Sudargo, & D. Priyandoko, 2018). Science process skills may
 8

be inherently more important than basic skills in student’s epistemic development because of the

important role they play in developing his abilities in problem‐solving (Aktamis & Ergin, 2007).

Utilization of such skills is an important indicator whether or not there has been a transfer in

one's knowledge (Faqih, 2018). The use of science process skills model allows variation in

instruction and provides opportunities for organizing rich learning environments. It mainly

comprises of five main aspects namely, operational definition, identifying and controlling

of
variables, formulating hypothesis, experimenting, and interpreting data. It is a quite popular

version of constructivism and has more effect to develop learners’ integrated science processing

ro
skills (Hırça, Calık, & Seven, 2011; Karsli & Ayas, 2014). Therefore, these main aspects are

used in the present study as follows:


-p
Operational Definition: to develop statements that display a concrete description of an
re
object or event by telling one what to do or observe (Koballa, 2015).

 Identifying and Controlling Variables: to identify dependent and independent variables

lP

that are constant or change under different conditions (Karim, Ahmad, & Osman, 2017).

 Formulating Hypothesis: to state an experiment expected outcome in a hypothesis form

na

(Khan, Sultana, & Haque, 2016).

 Experimenting: to test study a hypotheses through manipulation and control of

ur

independent variables and noting the impacts on the dependent variable and then

interpreting and presenting findings in a form of a report that others can follow to
Jo

replicate the experiment (Zeidan & Jayosi, 2015).

 Interpreting data: to interpret and describe experimental data and then make conclusions

(Seetee & Dahsah, 2017).

 9

In brief, pre-recorded lecture video or screencasts are important educational materials for

delivering instruction outside classroom in FC format. Hence, it is reasonable to check how

effective using advance organizers is in students' achievement when using flipped classroom

strategy. Besides, it is crucial to check the ability of a recorded lecture video to deliver enough

and sufficient explanation and consequently develop a group of skills that are difficult to be

delivered through the text. It is also reasonable to do so when there is no logical framework to

orient flipped classrooms' design and implementation.

of
In brief and in light of the above mentioned queries, the present study is based on the

ro
hypothesis that advance organizers in flipped classroom provided through LMS might lead to

developing learners' integrated science process skills.

Methodology
-p
re
Research Design and Participants

The present study adopted two-group experimental design to investigate whether advance
lP

organizers in FC via e-learning management system enhances learners’ integrated science

process skills. The sample consisted of 60 home economics female students registered in
na

curricular requirement course entitled "Research Proposal" provided by Najran University in the

first semester of 2017/2018. Participants were randomly divided into two groups, experimental
ur

and control and so there were 30 students in each group. In the first meeting, all participant

students were informed about the study aims and were requested to submit their written consents
Jo

in case of agreeing to participate. None of them apologized for not taking part or tried to

withdraw the course. The FC strategy was used to teach both groups. Advance organizers were

used to teach students in the experimental group whereas no advance organizers were utilized in

when teaching colleagues in the control group.

 10

To ensure homogeneity and equivalence of both groups regarding their knowledge of the

course content, ANOVA was used to compare participants’ scores in both groups according to

the product evaluation card. Results are shown in Table 1.

Teaching materials

In order to implement the course content through learning management system-based FC,

the teaching material was redesigned into 11-separate lectures. Besides, many instructional e-

learning design models were reviewed to get a picture of how to conduct the present study.

of
Moreover, factors like learners’ characteristics in addition to course goals, content and teaching

ro
activities were taken into consideration when producing the Echo system-based videotaped

lecture. Each lecture was videotaped twice. Advance organizers were used in the first video

-p
lecture to teach participants in the experimental group while no organizes were used in the
re
second taping that targeted participants in the control group. In other words, the content the of

video lectures delivered in the "Research Proposal" course aimed to train students how to write
lP

operational definitions, identify and control variables, formulate hypothesis, experiment, and

interpret data. This content was, on one hand, delivered to students in the experimental group
na

utilizing advance organizers. On the other hand, it was delivered to students in the experimental

group without using any advance organizers. In brief, videotaped lectures with advance
ur

organizers were electronically delivered to students in the experimental group at beginning of the

lecture and before starting in-detail lesson explanation and discussion. That is, students were to
Jo

individually go over and check the content. Expository advance organizers were also used to

provide learners with some basics to help them build their conceptions regarding the newly

presented topic. Some of the most important elements in the new topic were considered and

delivered via a more formal concept map. That is, learners were provided with new concepts in
 11

clearer picture including its main characteristics, origins and definition and accounting for

clarity, comprehensibility and order. Figure 1 illustrates a comparison between the place of

advance organizers in the flipped classroom for teaching students in the experimental and control

groups.

The experimental group The control group

of
Recorded Lecture video
3. Expository Advance Recorded Lecture video

ro
Prior to Organizers 1. In detail lesson explanation
class 4. In detail lesson explanation

1. Teacher asks a set of

questions to ensure that -p 1. Teacher asks a set of
questions to ensure that
re
students have studied the students have studied the
recorded lecture video recorded lecture video
During before class. before class.
lP

class 2. Students in groups 2. Students in groups

participate in learning participate in learning
tasks related to integrated tasks related to integrated
science process skills. science process skills.
na

Fig. 1. Place of advance organizers in experimental and control groups' flipped classroom
ur

Blackboard provided by Deanship of E-learning at Najran University was used where a link
Jo

specific for Echo system-based recorded lecture video was attached to allow students in both

experimental group to access the content of each lecture as shown in figure 2.

 12

of
ro
Fig. 2. Blackboard system with recorded lecture video

-p
In addition, a set of interactive tools such as discussion across forms; groups that

facilitate collaboration among students and allow them to strengthen the virtual relations with
re
colleagues; and blogs as open communication means for students to share their ideas were

provided by Blackboard system and used along with each recorded lecture video for both groups.
lP

FC lectures were implemented in two phases. In the first phase, students were to watch

individually a 25- minute recorded video delivered to them before class via blackboard system.
na

Then, they have to answer their instructor's questions in a classical face-to-face class to make

sure that they have watched the recorded lecture video before coming to class. After that,
ur

students can be, in groups, engaged in learning tasks related to integrated science process skills.

Like for instance, writing key questions, documenting references and resources, identifying
Jo

affecting variables, controlling fixed variables, writing study null and alternative hypotheses,

describing proposed experimental design, identifying data collection techniques and tools, and

processing data or results.

Data source and gathering

 13

Data collection went through a set of processes. First, each participant had to submit a

research proposal at the end of the second-week for in advance assessment by an independent

instructor who was a professor of curricula and instruction at home economics department. Then,

at the beginning of the 12th week, each participant student was to resubmit the research proposal

for final assessment using the developed five-domain product evaluation card where each

domain represented one main skill of integrated science process skills. There were five subskills

in domains one and two while there were six subskills in domains three and four. In the last

of
domain, there were only four subskills. For each item in these domains, Likert five-point scale

ro
ranging from one=strongly disagree to five=strongly agree was used. Cronbach Alpha for the

card's internal reliability was 0.82. In addition, another independent instructor who was also a

-p
professor of curricula and instruction at home economics department was requested to analyze a
re
random sample of approximately 10% of all research proposals to ensure findings' reliability and

to check inter-rater reliability that was 88%. After that, three professors of curricula and
lP

instruction at the department of home economics were requested to asses all participants’ final

research proposals and so each research proposal was evaluated three times and had three scores.
na

Mean scores of the three evaluators' scores were extracted for each student. Differences between

participants’ mean scores in the pre-application and post–application of developed product card
ur

were considered to calculate the modified gain ratio for all participants.

Experimental procedure
Jo

To ensure that all participants were subjected to the same teaching material and

methodology, one instructor who was one of the researchers in the present study, taught all

participant students. Therefore, they all, at the end of the first week, were required to prepare

and then individually submit their research proposals at the end of the second week or beginning
 14

of the third one to be assessed according to the product evaluation card. After that, the nature of

learning through FC on Blackboard system was made clear for participants in both groups during

lectures in the third week and then assigned into control and experimental groups. Students in

section (A) were assigned as experimental and those in in section (B) were known as a control

group and each one was asked to check and activate her username and password assigned on

Blackboard system. At last, the way of submitting assignments was made clear in addition to

stressing the importance of submitting each assignment individually for final evaluation.

of
Data analysis

ro
One-way ANOVA and SPPS (23.0) were used to evaluate students' assignment submitted

for checking Students' homogeneity in the pre-evaluation stage. T. test for independent samples

-p
and means of participants' scores were used to compare assignments of students in both groups
re
when p < 0.05.

Results
lP

Pre-assessment of participants' skills in science integrated process

As said before, each participant student had to submit a research proposal at the end of
na

the second-week for in advance assessment by an independent lecturer, using the product

evaluation card to check her homogeneity before starting the experiment. Table 1 reveals the
ur

differences between participants' scores in the pre-application using the product evaluation card.
Jo

Table 1

ANOVA for the differences between participants' mean scores in the pre-application of product
evaluation card.
Integrated science Sum of Mean F.
Sig.
process skills Squares Square ratio
Operational definition Between Groups 2.017 2.017 1.16 0.286
Within Groups 100.833 1.739
Total 102.85
 15

Identifying and Between Groups 1.35 1.35 0.809 0.372

Controlling Variables
Within Groups 96.833 1.67
Total 98.183
Formulating Between Groups 3.267 3.267 1.638 0.206
Hypothesis
Within Groups 115.667 1.994
Total 118.933
Experimenting Between Groups 1.667 1.667 1.136 0.291
Within Groups 85.067 1.467
Total 86.733
Interpreting data Between Groups 1.067 1.067 1.205 0.277
Within Groups 51.333 0.885

of
Total 52.4
Total Between Groups 2.4 2.4 0.151 0.699
Within Groups 922.6 15.907

ro
Total 925.0

Results in table 1 shows the differences between mean scores of participants’ research

-p
proposals in both groups with regard to their skills in the Operational Definition, Identifying and
re
Controlling Variables, Formulating Hypothesis, Interpreting data, and integrated science process

skills as a whole were all insignificant at (α=0.05). That is, all participants were homogeneous
lP

and had equivalent abilities in skills of integrated science process before being exposed to the

experiment.
na

Post-assessment of participants' skills in science integrated process

To show the differences between participant students' performance in both groups

ur

regarding the skills of writing research proposals, T. test was used for all domains.

Results related to student’ Operational Definition skills

Jo

To check whether there were statistically significant differences between students' skills

in the operational definition when writing a research proposal, T. Test for two independent

samples was used. Results are shown in table 2.

Table 2
 16

Significance of differences between mean scores of students' grades in both groups regarding
their operational definition skills.
Integrated science Mean T.
Group M SD Sig.
process skill Difference Ratio

Operational definition 1 13.3 4.36404 2.34 2.524 0.040

2 10.96 2.56614

Table 2 indicates that t. value for the differences between participants' modified gain ratio

of
on the product evaluation card was (T=2.524). Mean score of participants in the experimental

group was (M=13.3) while it was (M=10.96) for participants in the control group. In other

ro
words, there was a statistically significant difference between both groups regarding their

-p
operational definition skills in favor of the experimental group. This difference (P=0.040<0.05)

indicates that the modified gain ratio of students in the experimental group who studied the
re
course content through the utilization of advance organizers in FC was a higher than the
lP

modified gain ratio of their colleagues who studied through the use of FC without any utilization

of any advance organizers. Therefore, it can be strongly claimed that the use of advance

organizers in FC was more effective in promoting the operational definition skills of home
na

economics students enrolled in "Research Proposal" course.

To confirm this finding, Eta Squared (η2) was used to extract the workable importance of
ur

the findings and locate to what extent operational definition skills of experimental group were
Jo

promoted. Significance of (η2) for the difference between modified gain ratio of participants in

both groups with regard to their operational definition skills was calculated and was (0.353).

Thus, it can be strongly claimed that utilization of advance organizers in FC has an impact in

promoting learners’ operational definition skills of integrated science process.

 17

In addition, qualitative analysis of participant students' research proposals' content proved

that students in the experimental group were more capable in considering the criteria for

formulating good research title, writing key questions, identifying study population, formulating

terms, and documenting references and resources, in comparison with their peers' ability in the

control group.

Results related to students’ skills in Identifying and Controlling variables

To check whether there were statistically significant differences between students' skills

of
in identifying and controlling variables when writing a research proposal, T. Test for two

ro
independent samples was used. Results are shown in table 3.

-p
Table 3

The significance of differences between mean scores of students' grades in both groups regarding
re
the skills of identifying and controlling variables
Integrated science Mean T.
Group M SD Sig.
process skill Difference Ratio
lP

Identifying and 1 14.23 3.48082 2.56 3.455 0.031

Controlling Variables
2 11.67 2.10637
na

Table 3 indicates that t. value for the differences between participants' modified gain ratio
ur

on the product evaluation card was (T=3.455). In other words, there was a statistically significant

difference between both groups regarding their skills in identifying, and controlling variables in
Jo

favor of the experimental group. This difference (P=0.031<0.05) indicates that the modified gain

ratio of students in the experimental group who studied the course content through the utilization

of advance organizers in FC was a higher than the modified gain ratio of their colleagues who

studied through the use of FC without any utilization of any advance organizers. Therefore, it

can be strongly claimed that the use of advance organizers in FC was more effective in
 18

promoting the skills of home economics students enrolled in "Research Proposal" course in

identifying and controlling variables.

To confirm this finding, Eta Squared (η2) was used to extract the workable importance of

the findings and locate to what extent identifying and controlling variables, skills of experimental

group were promoted. Significance of (η2) for the difference between modified gain ratio of

participants in both groups with regard to their operational definition skills was calculated and

was (0.379). Hence, it can be strongly claimed that utilization of advance organizers in FC has an

of
impact in promoting learners’ operational definition skills of integrated science process.

ro
Meanwhile, qualitative analysis on the contents of the learner's proposals showed that

most participants in the experimental group were more worthy in identifying affecting variables,

-p
identifying (independent, dependent and persistent) variables, highlighting conditions for fixing
re
and not fixing variables, testing effect of independent variable on the dependent one, and

controlling fixed variables in comparison with the control group.

lP

Results related to students’ skills in Formulating Hypothesis

To check whether there were statistically significant differences between students' skills
na

in formulating research hypothesis when writing a research proposal, T. Test for two

independent samples was used. Results are shown in table 3.

ur

Table 4
Jo

The significance of differences between mean scores of students' grades in both groups regarding
the skills of formulating hypothesis
Integrated science Mean T.
Group M SD Sig.
process skills Difference Ratio

Formulating Hypothesis 1 18.07 3.41329 3.04 4.072 0.034

2 15.03 2.23581
 19

Table 4 indicates that t. value for the differences between participants' modified gain ratio

on the product evaluation card was (T=4.072). In other words, there was a statistically significant

difference between both groups regarding their skills in formulating hypothesis in favor of the

experimental group. This difference (P=0.034<0.05) indicates that the modified gain ratio of

students in the experimental group who studied the course content through the utilization of

advance organizers in FC was a higher than the modified gain ratio of their colleagues who

studied through the use of FC without any utilization of any advance organizers. Therefore, it

of
can be strongly claimed that the use of advance organizers in FC was more effective in

ro
promoting the skills of home economics students enrolled in "Research Proposal" course in

hypothesis formulating.

-p
To confirm this finding, Eta Squared (η2) was used to extract the workable importance of
re
the findings and locate to what extent formulating hypothesis skills of experimental group were

promoted. Significance of (η2) for the difference between modified gain ratio of participants in
lP

both groups with regard to their formulating hypothesis skills was calculated and was (0.371).

Hence, it can be strongly claimed that utilization of advance organizers in FC has an impact in
na

promoting learners’ formulating hypothesis skills of integrated science process.

In brief, the utilization of advance organizers in FC was more effective in enhancing

ur

participants' abilities in splitting long questions to shorter ones, formulating probable answers

that can be tested for every question, writing study null and alternative hypotheses, and
Jo

distinguishing between hypotheses to be tested descriptively and those to be described

quantitatively.

Results related to students’ skills in Experimenting field

 20

To check whether there were statistically significant differences between students' skills

in experimenting when writing a research proposal, T. Test for two independent samples was

used. Results are shown in table 5.

Table 5

The significance of differences between mean scores of students' grades in both groups regarding
the skills of experimenting
Integrated science Mean T.
Group M SD Sig.

of
process skills Difference Ratio

Experimenting 1 17.03 3.44897 2.43 3.216 0.044

ro
2 14.6 2.29843

-p
Table 5 indicates that t. value for the differences between participants' modified gain ratio
re
on the product evaluation card was (T=3.216). In other words, there was a statistically significant

difference between both groups regarding their skills in experimenting in favor of the
lP

experimental group. This difference (P=0.044<0.05) indicates that the modified gain ratio of

students in the experimental group who studied the course content through the utilization of
na

advance organizers in FC was a higher than the modified gain ratio of their colleagues who

studied through the use of FC without any utilization of any advance organizers. Therefore, it
ur

can be strongly claimed that the use of advance organizers in FC was more effective in

promoting the skills of home economics students enrolled in "Research Proposal" course in
Jo

experimenting.

To confirm this finding, Eta Squared (η2) was used to extract the workable importance of

the findings and locate to what extent experimenting skills of experimental group were

promoted. Significance of (η2) for the difference between modified gain ratio of participants in

both groups with regard to their experimenting skills was calculated and was (0.348). Hence, it
 21

can be strongly claimed that utilization of advance organizers in FC has an impact in promoting

learners’ experimenting skills of integrated science process.

These results prove that most participants in the experimental group were more capable

in describing sampling technique and type, describing proposed experimental design, identifying

data collection techniques and tools, using appropriate equipment, and concluding results were in

short time in comparison with peers' ability in control group.

of
Results related to students’ skills in Interpreting data

To check whether there were statistically significant differences between students' skills in

ro
interpreting data when writing a research proposal, T. Test for two independent samples was

used. Results are shown in table 6.

Table 6
-p
re
The significance of differences between mean scores of students' grades in both groups regarding
the skills of interpreting data
lP

Integrated science Mean T.

Group M SD Sig.
process skills Difference Ratio

Interpreting data 1 11.93 2.34790 2.51 3.527 0.047

na

2 9.42 3.12411
Table 6 indicates that t. value for the differences between participants' modified gain ratio
ur

on the product evaluation card was (T=3.527). In other words, there was a statistically significant

difference between both groups regarding their skills in interpreting data in favor of the
Jo

experimental group. This difference (P=0.047<0.05) indicates that the modified gain ratio of

students in the experimental group who studied the course content through the utilization of

advance organizers in FC was a higher than the modified gain ratio of their colleagues who

studied through the use of FC without any utilization of any advance organizers. Therefore, it

can be strongly claimed that the use of advance organizers in FC was more effective in
 22

promoting the skills of home economics students enrolled in "Research Proposal" course in

interpreting data.

To confirm this finding, Eta Squared (η2) was used to extract the workable importance of

the findings and locate to what extent interpreting data skills of experimental group were

promoted. Significance of (η2) for the difference between modified gain ratio of participants in

both groups with regard to their interpreting data skills was calculated and was (0.337). Hence, it

can be strongly claimed that utilization of advance organizers in FC has an impact in promoting

of
learners’ interpreting data skills of integrated science process.

ro
Besides, qualitative analysis the learners' research proposals indicated that most learners

in the experimental group were better than colleagues in the control group in identifying topic

-p
related data or results, processing data or results, and identifying topic related laws and principles
re
or theories.

Results related to students’ skills in Integrated Science Process Skills as a whole

lP

To check whether there were statistically significant differences between students' skills

in the integrated science process as a whole when writing a research proposal, T. Test for two
na

independent samples was used. Results are shown in table 7.

Table 7
ur

Significance of differences between mean scores of students' grades in both groups regarding the
integrated science process skills as a whole.
Jo

Integrated science Mean T.

Group M SD Sig.
process skills as a whole Difference Ratio

Total 1 74.57 8.56892 12.89 7.193 0.006

2 61.68 4.77509
 23

Table 7 indicates that t. value for the differences between participants' modified gain ratio

on the product evaluation card was (T=7.193). Mean score of participants in the experimental

group was (M=74.57) while it was (M=61.68) for participants in the control group. In other

words, there was a statistically significant difference between both groups regarding their

integrated science process skills as a whole in favor of the experimental group. This difference

(P=0.006<0.05) indicates that the modified gain ratio of students in the experimental group who

studied the course content through the utilization of advance organizers in FC was a higher than

of
the modified gain ratio of their colleagues who studied through the use of FC without any

ro
utilization of any advance organizers. Therefore, it can be strongly claimed that the use of

advance organizers in FC was more effective in promoting the integrated science process skills

-p
as a whole of home economics students enrolled in "Research Proposal" course.
re
To confirm this finding, Eta Squared (η2) was used to extract the workable importance of

the findings and locate to what extent integrated science process as a whole skills of
lP

experimental group were promoted. Significance of (η2) for the difference between modified

gain ratio of participants in both groups with regard to integrated science process as a whole
na

skills was calculated and was (0.398). Hence, it can be strongly claimed that utilization of

advance organizers in FC has an impact in promoting learners’ integrated science process skills
ur

as a whole.

Discussion
Jo

The main aim of the present study was to assess the effectiveness of advance organizers

utilization in FC via Learning Management System to develop learners’ integrated science

process skills. Skills of home economics students enrolled in an academic course entitled

"Research Proposal" were investigated. Findings regarding the development of students’ skills in
 24

the study main domains namely, operational definition; identifying and controlling variables;

formulating hypothesis; experimenting' and interpreting data were very encouraging. Such

Findings can emphasize what Hadi (2016) concluded regarding gaining and understanding new

knowledge. Mean scores of student’s research proposals using the developed product evaluation

card indicate the effect of using advance organizers on promoting learners’ skills in such crucial

areas for research proposal writing. Results of the present study also corroborate the findings of

L. J. Day (2018) and Lewis et al. (2018). In short, it can be strongly claimed that advance

of
organizers utilization in recorded lecture videos in FCs can contribute to learners' knowledge

ro
construction. Mills (2017) argued that advance organizers can help learners link new information

with their cognitive structure. Relations were made clearer between learner's concepts, realities

-p
and issues, on one hand and the introduction to new learning materials via FC, on the other hand.
re
One interesting interpretation for that is the fact that face-to-face sessions can make learning

more student-centered and allow time for discussion, group work, and problem-solving
lP

(Pickering & Roberts, 2018).

Results related to the integrated science process skills as a whole were also positive and
na

reflected the effectiveness of using advance organizers in FCs through the recorded lecture video

on Blackboard system. Aktamis and Ergin (2007) mentioned that integrated processing skills are
ur

more important than basic skills in student’s epistemic development because they play an

important role in developing his abilities in problem‐solving. In short, experimental group

Jo

students' integrated science process skills were promoted better than skills of their colleagues in

the control group who received learning in FC without using advance organizers.

Limitations and future directions

 25

It is difficult to generalize the results of the present study because of a set of reasons or

limitations. First, the focus of this research was to check the role of advance organizer's

utilization in FC via e-learning management system to promote learners' integrated science

process skills. Second, participation in the present study was limited to a group of female

students because of the nature of home economics specialty and because of the fact that co-

education is not allowed or common in Saudi Arabia. Therefore, other researchers are invited to

carry out identical studies in other departments where co-education is permitted to confirm that

of
such a method is successful in another environment. Third, collected data were limited to Najran

ro
University, which is a public university in Saudi Arabia. Therefore, to promote generalizability

of results, researchers are called to conduct similar studies and collect data from multiple

-p
institutions (Choi & Park, 2018). Lastly, participants were university students of 19 and 20 years
re
old that make it unfair to compare results of the current study with results related to students of

different age (Elfeky & Elbyaly, 2018).

lP

Conclusion

In the present research, an advance organizer was utilized to explain the content of
na

"Research Proposal" course via the strategy of FCs. E-learning management system

"Blackboard" with tutorial videos was also utilized to design FCs of this course. Besides, the
ur

study aimed to investigate the impact of advance organizers' utilization in FC via e-learning

management system to promote students’ integrated science process skills. Results indicated that
Jo

learners who were taught via lectures using FC strategy provided with an advance organizer

showed better skills in study domains mainly, operational definition; identifying and controlling

variables; formulating hypothesis; experimenting; interpreting data; and integrated science

 26

process skills as a whole in comparison with their colleagues in the control group who were not

supplied with any advance organizers.

Statements on open data, ethics and conflict of interest

a. Data can be accessed by contacting authors.
b. No agreements of any type were needed as all participant students were enrolled in a
course taught by an instructor who was one of the researchers of the present at Najran
University.
c. The authors declare that they have no conflict of interest.

of
Acknowledgements

ro
The researchers are grateful to all home economics students enrolled in the "Research

Proposal" course. They are thankful for the Deanship of eLearning at Najran University for the

-p
provision of all teaching tools that helped make this work successful.
re
lP
na
ur
Jo
 27

References

Aktamis, H., & Ergin, O. (2007). Investegating the relationship between science process skills and
scientific creativity. HACETTEPE UNIVERSITESI EGITIM FAKULTESI DERGISI-HACETTEPE
UNIVERSITY JOURNAL OF EDUCATION(33), 11-23.
AlJarrah, A., Thomas, M. K., & Shehab, M. (2018). Investigating temporal access in a flipped classroom:
procrastination persists. International Journal of Educational Technology in Higher Education,
15(1), 1.
Amouzeshi, Z., Mohsenizadeh, M., & Amouzeshi, A. (2015). Effect of Teaching Nursing Process Using
Integrated Method (Concept Map and Advance Organizer) on Nursing Students’ Clinical
Learning. Future of Medical Education Journal, 5(1), 68-71.
Anderson, T., & Dron, J. (2018). Integrating learning management and social networking systems. Italian
Journal of Educational Technology, 25(3), 5-19.

of
Barrie, M. G., Amick, C., Mitzman, J., Way, D. P., & King, A. M. (2018). Bringing the Flipped Classroom to
Day 1: A Novel Didactic Curriculum for Emergency Medicine Intern Orientation. Western Journal
of Emergency Medicine, 19(1), 145.

ro
Byrd, R. (2018). Using Appropriate E-learning Systems to Optimize Teaching and Learning. GSTF Journal
on Computing (JoC), 2(3).
Chen, Y., Wang, Y., & Chen, N.-S. (2014). Is FLIP enough? Or should we use the FLIPPED model instead?

-p
Computers & Education, 79, 16-27.
Choi, H. J., & Park, J.-H. (2018). Testing a path-analytic model of adult dropout in online degree
programs. Computers & Education, 116, 130-138.
Chuang, H. H., Weng, C. Y., & Chen, C. H. (2018). Which students benefit most from a flipped classroom
re
approach to language learning? British Journal of Educational Technology.
Cutrer, W. B., Castro, D., Roy, K. M., & Turner, T. L. (2011). Use of an expert concept map as an advance
organizer to improve understanding of respiratory failure. Medical teacher, 33(12), 1018-1026.
lP

Day, L. (2017). Does Motivation or Learning-Strategies Develop in a Flipped-Classroom? The FASEB

Journal, 31(1 Supplement), 392.393-392.393.
Day, L. J. (2018). A gross anatomy flipped classroom effects performance, retention, and higher‐level
thinking in lower performing students. Anatomical sciences education.
DeLozier, S. J., & Rhodes, M. G. (2017). Flipped classrooms: a review of key ideas and recommendations
na

for practice. Educational Psychology Review, 29(1), 141-151.

Elfeky, A. I. M. (2018). The effect of personal learning environments on participants’ higher order
thinking skills and satisfaction. Innovations in Education and Teaching International, 1-12.
Elfeky, A. I. M., & Elbyaly, M. Y. H. (2017). THE USE OF CSCL ENVIRONMENT TO PROMOTE
ur

STUDENTS'ACHIEVEMENT AND SKILLS IN HANDMADE EMBROIDERY. European Journal of

Training and Development Studies, 4(2), 19-32.
Elfeky, A. I. M., & Elbyaly, M. Y. H. (2018). Developing skills of fashion design by augmented reality
Jo

technology in higher education. Interactive Learning Environments, 1-16. doi:

10.1080/10494820.2018.1558259
Faqih, M. I. (2018). MEMETAKAN KETERAMPILAN PROSES SAINS SISWA SMP KELAS VII DI KABUPATEN
GUNUNG KIDUL YOGYAKARTA. Phenomenon: Jurnal Pendidikan MIPA, 7(2), 187-195.
Foster, G., & Stagl, S. (2018). Design, implementation, and evaluation of an inverted (flipped) classroom
model economics for sustainable education course. Journal of Cleaner Production.
Hadi, A. M. A. (2016). Students' Perceptions of Science Laboratory Learning Experience to Acquire
Integrated Science Process Skills Among Undergraduate Students at UPSI. Faculty of Science and
Mathematics, Universiti Pendidikan Sultan Idris.
 28

Hırça, N., Calık, M., & Seven, S. (2011). Effects of guide materials based on 5E model on students’
conceptual change and their attitudes towards physics: A case for ‘work, power and energy’unit.
Journal of Turkish Science Education, 8(1), 139-152.
Homavazir, M. F., & Gopal, R. (2018). A Research Study on the Digitalisation of Higher Education and its
Impact on Teaching and Students Assessment in Commerce, Management and Science Colleges
in Mumbai. AADYA-Journal of Management and Technology (JMT), 8(1), 93-100.
Jenkins, J., Railey, J., Koul, B., Tight, M., Gagne, R. M., & Marland, P. (2017). Unit-6 Some Instructional
Strategies for Improved Learning From Distance Teaching Materials: IGNOU.
Karim, E., Ahmad, J., & Osman, K. (2017). Fuzzy Delphi Method for Content Validation of Integrated
Science Process Skills Instrument. International Journal of Academic Research in Business and
Social Sciences, 7(6), 773-785.
Karsli, F., & Ayas, A. (2014). Developing a laboratory activity by using 5E learning model on student
learning of factors affecting the reaction rate and improving scientific process skills. Procedia-

of
Social and Behavioral Sciences, 143, 663-668.
Khan, M. A. A., Sultana, O., & Haque, M. M. (2016). Problems to implement science process skills in
promoting scientific literacy at primary Science classrooms. PRIMARY EDUCATION JOURNAL, 21.

ro
King, A. M., Mayer, C., Barrie, M., Greenberger, S., & Way, D. P. (2018). Replacing Lectures with Small
Groups: The Impact of Flipping the Residency Conference Day. Western Journal of Emergency
Medicine, 19(1), 11.

-p
Koballa, T. (2015). Science instruction in the middle and secondary schools: Developing fundamental
knowledge and skills: Upper Saddle River, NJ: Pearson.
Lewis, C. E., Chen, D. C., & Relan, A. (2018). Implementation of a flipped classroom approach to promote
active learning in the third-year surgery clerkship. The American Journal of Surgery, 215(2), 298-
re
303.
Li, C.-H., Wu, M.-H., & Lin, W.-L. (2017). The Use of a “Think-Pair-Share” Brainstorming Advance
Organizer to Prepare Learners to Listen in the L2 Classroom. International Journal of Listening, 1-
lP

14.
Li, S. (2016). Does providing an advance organizer in online drill and practice games improve language
learning outcomes? , University of Twente.
Lopes, A. P., & Soares, F. (2018). Perception and performance in a flipped Financial Mathematics
classroom. The International Journal of Management Education, 16(1), 105-113.
na

Malik, R. (2015). A study of the effectiveness of advance organizer model on students’ achievement in
economics. International Journal of Advanced Research in Management and Social Sciences,
4(4), 182-188.
Masadeh, T. S. Y., & Elfeky, A. I. M. (2016). Efficacy of Open-Source Learning Management Systems in
ur

Developing the Teaching Skills of English Language Student Teachers. American Journal of
Educational Research, 4(4), 329-337.
McLaughlin, J. E., Roth, M. T., Glatt, D. M., Gharkholonarehe, N., Davidson, C. A., Griffin, L. M., . . .
Jo

Mumper, R. J. (2014). The flipped classroom: a course redesign to foster learning and
engagement in a health professions school. Academic Medicine, 89(2), 236-243.
McLean, S., Attardi, S. M., Faden, L., & Goldszmidt, M. (2016). Flipped classrooms and student learning:
not just surface gains. Advances in physiology education, 40(1), 47-55.
Mershad, K., & Wakim, P. (2018). A Learning Management System Enhanced with Internet of Things
Applications. Journal of Education and Learning, 7(3), 23.
Miller, K. A., Lukoff, B. M., King, G., & Mazur, E. (2018). Use of a social annotation platform for pre-class
reading assignments in a flipped introductory physics class. Paper presented at the Frontiers in
Education.
 29

Mills, R. J. (2017). Using Analogical Problem Construction As An Advance Organizer To Teach Advanced
Database (SQL) Nomenclature. The Review of Business Information Systems (Online), 21(1), 1.
O'Flaherty, J., Phillips, C., Karanicolas, S., Snelling, C., & Winning, T. (2015). Corrigendum to" The use of
flipped classrooms in higher education: a scoping review"[The Internet and Higher Education 25
(2015) 85-95].
Pickering, J. D., & Roberts, D. J. (2018). Flipped classroom or an active lecture? Clinical Anatomy, 31(1),
118-121.
Prajoko, S., Amin, M., Rohman, F., & Gipayana, M. (2016). The Profile and The Understanding of Science
Process Skills Surakarta Open University Students in Science Lab Courses. Paper presented at the
Proceeding of International Conference on Teacher Training and Education.
Sari, P., Sudargo, F., & Priyandoko, D. (2018). Correlation among science process skill, concept
comprehension, and scientific attitude on regulation system materials. Paper presented at the
Journal of Physics: Conference Series.

of
Sari, P. M., Sudargo, F., & Priyandoko, D. (2018). The Effect of The Practical-Based Learning Model
Toward Science Process Skills and Concept Comprehension of Regulation System. JPI (Jurnal
Pendidikan Indonesia), 6(2), 191-197.

ro
Seetee, N., & Dahsah, C. (2017). Science Process Skills in Kindergarten Projects. Paper presented at the
Conference Proceedings. New Perspectives in Science Education.
Sergis, S., Sampson, D. G., & Pelliccione, L. (2018). Investigating the impact of Flipped Classroom on

-p
students' learning experiences: A Self-Determination Theory approach. Computers in Human
Behavior, 78, 368-378.
Sinclair, J., & Aho, A.-M. (2018). Experts on super innovators: understanding staff adoption of learning
management systems. Higher Education Research & Development, 37(1), 158-172.
re
Steen-Utheim, A. T., & Foldnes, N. (2018). A qualitative investigation of student engagement in a flipped
classroom. Teaching in Higher Education, 23(3), 307-324.
Tune, J. D., Sturek, M., & Basile, D. P. (2013). Flipped classroom model improves graduate student
lP

performance in cardiovascular, respiratory, and renal physiology. Advances in physiology

education, 37(4), 316-320.
Zeidan, A. H., & Jayosi, M. R. (2015). Science Process Skills and Attitudes toward Science among
Palestinian Secondary School Students. World journal of Education, 5(1), 13-24.
Zheng, R. Z., Yang, W., Garcia, D., & McCadden, E. (2008). Effects of multimedia and schema induced
na

analogical reasoning on science learning. Journal of computer assisted learning, 24(6), 474-482.
Zhu, W., & Xie, W. (2018). Evaluating Instructional Effects of Flipped Classroom in University: A Case
Study on Electronic Business Course. International Journal of Distance Education Technologies
(IJDET), 16(1), 45-55.
ur
Jo
 30

Appendix A Product Evaluation Card

Tick the extent that matches your opinion regarding the students' ability in these areas regarding
the research proposal:

Strongly agree
Neither agree
nor disagree
Card domains

Disagree
disagree
Strongly

Agree
Operational Definition
1 Considering criteria for formulating good
research title.

of
2 Writing key question that the study will
answer.
3 Identifying study population.

ro
4 Formulating terms associated with the result
and its causes or phenomena and their
conditions.

-p
5 Documenting references and resources.
Identifying and Controlling Variables
6 Identifying affecting variables.
7
re
Identifying independent, dependent and
persistent variables.
8 Highlighting conditions for fixing and not
fixing variables.
lP

9 Testing effect of independent variable on the

dependent one.
10 Controlling fixed variables.
Formulating Hypothesis
na

11 Identifying study questions to be answered.

12 Isolating logical questions from direct ones.
13 Splitting long questions to shorter ones.
14 Formulating probable answers that can be
ur

tested for every question.

15 Writing study null and alternative
hypotheses.
Jo

16 Distinguishing between hypotheses to be

tested descriptively and those to be described
quantitatively.
Experimenting
17 Describing sampling technique and type.
18 Describing proposed experimental design.
19 Identifying data collection techniques and
tools.
20 Following simple, direct and clear steps.
 31

21 Using appropriate equipment.

22 Concluding results were in short time.
Interpreting data
23 Identifying topic related data or results.
24 Processing data or results.
25 Identifying topic related laws, principles or
theories.
26 Including study answer in a previous study
related to study chosen topic.

of
ro
-p
re
lP
na
ur
Jo