The current issue and full text archive of this journal is available on Emerald Insight at:

https://www.emerald.com/insight/1741-5659.htm

Framework of
A framework of smart pedagogy smart
based on the facilitating of high pedagogy

order thinking skills

Qingquan Meng and Jiyou Jia 251
School of Education, Peking University, Beijing, China, and
Received 15 November 2019
Zhiyong Zhang Revised 6 March 2020
16 March 2020
Huangdu Middle School, Shanghai, China Accepted 25 May 2020

Abstract
Purpose – The purpose of this study is to verify the effect of smart pedagogy to facilitate the high order
thinking skills of students and to provide the design suggestion of curriculum and intelligent tutoring
systems in smart education.
Design/methodology/approach – A smart pedagogy framework was designed. The quasi-experiment
was conducted in a junior high school. The experimental class used the smart pedagogy and smart learning
environment. The control class adopted conventional teaching strategies. The math test scores of these two
classes were compared to verify the effectiveness of smart pedagogy.
Findings – The smart pedagogy framework contains three sections including the situated learning (S),
mastery learning (M), adaptive learning (A), reflective learning (R) and thinking tools (T) (SMART) key
elements model, the curriculum design method and detailed teaching strategy. The SMART key elements
model integrates the situated learning, mastery learning, adaptive learning, reflective learning and thinking
tools to facilitate the high order thinking. The curriculum design method of smart pedagogy combines the
first five principles of instruction and the SMART key elements model to design the curriculum. The detailed
teaching strategies of smart pedagogy contain kinds of innovative learning methods. The results of the quasi-
experiment proved that the learning outcome was significantly promoted by using smart pedagogy.
Originality/value – This research investigates a general framework that can be used to cultivate the high
order thinking skills in different subjects and grades was one of the first to introduce high order thinking
skills into smart education. The framework of smart pedagogy was innovative and effect in practice.
Keywords Smart education, High order thinking, Smart pedagogy, Thinking tools
Paper type Research paper

Introduction
With the continuous development of E-learning systems and intelligent tutoring
systems (ITS) (Jia, 2015), smart education, which can enable students to learn more
efficiently, individually and flexibly has gained more and more attention in recent years
(Spector, 2018). The concepts relevant to the smart education include smart learning,
smart learning environment, smart school, smart classroom, smartboard and smart
pedagogy, etc.

This research work is supported by the “Research on Optimization Design of Intelligent Tutoring Interactive Technology and Smart
System Based on Semantic Graphic and Thinking Visualization Tools Under the Guidance of Key Education
Competence” (ECA190481) from the 13th five-year National Research Program on Educational
Vol. 17 No. 3, 2020
pp. 251-266
Sciences in China (2019). This research is also sponsored by Beijing Lexue 100 Online Education Co. © Emerald Publishing Limited
1741-5659
Ltd. The authors thank all the teachers and students who have participated in the research program. DOI 10.1108/ITSE-11-2019-0076
ITSE The smart education constructs a technology-integrated learning environment that
17,3 enables teachers to implement effective teaching pedagogies so that learners can obtain
appropriate personalized learning services and gain development in activity, thinking and
creativity (Gros, 2016). The smart education includes three sections, namely, a smart
learning environment, smart learners and smart pedagogy. The smart pedagogy plays a
more and more important role in smart education (Zhu et al., 2016).
252 The thinking abilities especially high order thinking skills are more and more important
in the rapidly developing society. It is urgent to improve the thinking ability of students
(Rabadi et al., 2018).
Research showed that high order thinking skills could be facilitated by using smart
education. Julius proved that using the smart board can overcome high order thinking skills
learning difficulties under the assistant of effective learning strategies (Julius et al., 2018).
While smart devices cannot ensure the improvement of thinking ability, technology and
pedagogy need to be combined in smart education. For example, Sung et al. mentioned that
the ebooks were designed to provide information in the multimedia form, it seldom engaged
the high order thinking (Sung et al., 2018). So, they proposed a problem-posing framework
that guides students to observe and pose questions. The experiment results showed that this
framework can significantly improve the students’ learning achievements. Zhou et al. (2018)
proposed a debating teaching strategy on students’ critical thinking development in a smart
classroom. By way of debate, students formed their own understanding and make a
reasonable judgment. Nobuyuki and Akira (2018) used organizational strategies to promote
the learning effect of smart education in computer classrooms using information and
communications technology (ICT) classrooms. The learner factors were also taken into
consideration of the facilitation of the high order thinking in the smart classroom
environment. Structural equation modeling revealed that learning style and internet attitude
were the main factors that affected the learners’ high order thinking (Wu et al., 2019).
The literature mentioned above proposed some strategies that can facilitate high order
thinking skills. While they can only be used in a certain field. In practice, we need a general
method that can be used in different subjects and grades. This research attempt to construct a
general framework of the smart pedagogy around the cultivation of high order thinking skills.
The framework of the smart pedagogy contains three sections including key elements model,
curriculum design and teaching strategies. The SMART key elements model includes situated
learning (S), mastery learning (M), adaptive learning (A), reflective learning (R) and thinking
tools (T). As for the design of the curriculum, the first five principles of instruction (Merrill,
2002) and the SMART key elements model are combined to effectively facilitate the high order
thinking skills. The detailed teaching strategies of smart pedagogy include massive open
online courses (MOOCs), small private online course (SPOC), personal inquiry learning,
collaborative learning, blended learning, mind map and concept map, etc.
With the support of the smart learning environment, teachers can use the smart
pedagogy to cultivate students’ high order thinking ability and realize smart education.
Quasi-experiment is conducted to verify the effectiveness of the smart pedagogy framework.

Literature review
Situated learning, mastery learning, adaptive learning, reflective learning and thinking tools
education
With the exponential advances of technology such as cloud computing, big data, internet of
things and wearable technology, education is greatly reconstructed in the past decades.
Smart education is a new concept that describes learning in an intelligent era. The smart
education is based on smart devices and intelligent technologies (Lee et al., 2014), Learners
can access learning resources by using smart devices. Many researchers have defined the Framework of
smart education, Coccoli et al. (2014) defined smart education as follows: “education in a smart
smart environment supported by smart technologies, making use of smart tools and smart
devices, can be considered smart education.” IBM (1997) describes the smart education as
pedagogy
follows: “a smart, multi-disciplinary student-centric education system – linked across
schools, tertiary institutions and workforce training.” Zhu and He (2012) stated that “the
essence of smart education is to create intelligent environments by using smart
technologies”. 253
The key features of smart education are described as self-directed, motivated, adaptive,
resource-enriched and technology-embedded by MEST (MEST: Ministry of Education,
Science and Technology of the Republic of Korea, 2011). In the opinion of Lee et al. (2014), the
features of smart education are proposed as formal and informal learning, social and
collaborative learning, personalized and situated learning, application and content focus.
The goal of smart education is to realize personalized learning, self-learning and self-
motivated. The learners can study in their own pace and gain better learning outcomes.
They can access to the resources at any time and in any place (Kim et al., 2013).
A research structure of smart education is proposed by Zhu et al. (2016). Three elements
are involved in smart education, namely, smart environment, smart pedagogy and smart
learners (Figure 1). The goal of smart education is to cultivate smart learners. Smart
pedagogies are methodological aspect and the smart environment is a technological issue.
Smart learners are the center of smart pedagogies and smart environments.

Situated learning, mastery learning, adaptive learning, reflective learning and thinking tools
pedagogy
Smart pedagogy is a set of innovative teaching strategies based on smart devices and a
smart learning environment (Uskov et al., 2018). It is an essential section of smart education.
Some researchers have found that the utilization of technologies such as multimedia
equipment and smart devices without significantly improving the teaching and learning
strategies did not promote pupils’ performance (OECD, 2015; Ramsden, 2011). On the other
hand, without the student-centered pedagogies, the use of technologies just strengthens the
didactic teaching (JISC, 2009). So, the design of smart education must give consideration to
both technologies and pedagogies.
In the traditional classroom, the learning goal and process are usually the same for each
student. In smart education, the learning pace should be varied according to learner’s
readiness level, interest and learning profiles (Mironova et al., 2016). The students who learn
in cooperative teams can keep the memory of knowledge longer and engage in discussion at
a high level of active thinking (Stahl et al., 2006). Intrinsic motivation is more important than
external motivation because the interest and passion of students are inspired (Peters and
Araya, 2010). Some other teaching strategies are listed below (Hogue et al., 2011; Cavanaugh

Figure 1.
Research structure of
smart education
proposed by Zhu et al.
(2016)
ITSE et al., 2013; Kapralos et al., 2015 and Villesseche et al., 2019), namely, learning-by-doing,
17,3 game-based learning, adaptive learning, flipped classroom, crowdsourcing-based learning,
badging-based learning, seamless learning, MOOCs, SPOC, crossover learning, personal
inquiry learning, learning through storytelling, learning to learn and learning through
argument, etc.
Zhu et al. (2016) propose a smart pedagogy framework including four instructional
254 strategies. From inside to outside, the framework involves class-based differentiated
instruction, group-based collaborative learning, individual-based personalized learning and
mass-based generative learning.

High order thinking

The high order thinking skills are a set of abilities that need complex thinking and have
several criteria. One of the criteria is defined according to the Bloom taxonomy including
remembering, understanding, applying, analyzing, evaluating and creating (Anderson and
Krathwohl, 2001). The high order thinking contains the last three-part of taxonomy
including analyzing, evaluating and creating (Pegg, 2010). The high order thinking is also
considered as a kind of thinking pattern that requires more mental effort to tolerate doubt,
discover knowledge and solve a problem in innovative ways (Newman, 1991).
Many methods are proposed to facilitate the high order thinking skills. Many
psychologists and educators have confirmed that high order thinking is facilitated in the
procedure of problem-solving situations or complex task resolutions (Raiyn and Tilchin,
2015). Problem-based learning is also proved effective to develop high order thinking skills
(Suprapto et al., 2017). The problem-solving situation is a commonly used method to
facilitate high order thinking (Raiyn and Tilchin, 2015). The problem-solving situation is a
teaching strategy of situated learning (Brown et al., 1989). In another hand, situated learning
can be realized in an electronic device by means of multimedia and virtual reality
technology. Visual learning is proved to be effective in improving students’ high order
thinking skills (Raiyn, 2016). Mastery learning instructional strategy has positive effects on
improving the students’ achievement, especially the effects that are stronger on the weaker
students (Kuilk and Bangertdrowns, 1990). If the teaching content is arranged according to
mastery learning, the students can perform significantly better in the high order thinking
skills such as creating (Khan and Masood, 2014). Mastery learning can be realized in
electronic devices conveniently (Purbohadi et al., 2014). Adaptive learning is one of the most
significant characters of smart education (MEST: Ministry of Education, Science and
Technology of the Republic of Korea, 2011). Adaptive learning allows for personalized
instruction while altering the learning pathways according to the difference of students. The
benefits of adaptive learning have been proved by previous experiments (Liu et al., 2017).
The adaptive learning can be realized by large data sets and machine learning algorithms
(Shelle et al., 2018). In smart education, teachers should be familiar with the merit of adaptive
learning by using smart equipment. Beckwith (2016) has found that reflective learning can
help develop high order thinking skills in medical education. Erdogan (2019) claimed that
cooperative learning supported by reflective thinking activities can be said to have a
positive effect on students’ critical thinking skills. Reflective learning pedagogy is an
important way to facilitate high order thinking. The round table cooperative learning
models are used to construct high order thinking skills (Harry et al., 2017). Thinking tools
such as concept maps have been proved to be an effect on developing high order thinking
skills (Canas et al., 2017). Thinking tools are visible exercises means to help students
improve their thinking ability. The high order thinking skills can be facilitated by using a
concept map as the alternate assessment tool (Ghani et al., 2017).
A situated learning, mastery learning, adaptive learning, reflective learning Framework of
and thinking tools pedagogy framework smart
The framework of the situated learning, mastery learning, adaptive learning, reflective pedagogy
learning and thinking tools pedagogy
Smart pedagogy is an essential section of smart education. In this paper, a general
framework of smart pedagogy with great emphasis on the cultivation of high order thinking
skills is investigated. The framework can be realized in a smart learning environment and 255
used in a blended learning scenario.
The framework of smart pedagogy includes three sections shown in Figure 2. The first
module is the key element of smart pedagogy including situated learning, mastery learning,
adaptive learning, reflective learning and thinking tools, the key elements can be
abbreviated as SMART by connecting the first letter of the five pedagogies. All of the
five pedagogies are proved to be effective in the facilitation of high order thinking and can
be conveniently realized in a smart learning environment. The key elements model is the
criterion of curriculum design in smart education. The second module is the curriculum
design method of smart pedagogy. The first five principles of instruction (Merrill, 2002) and
the SMART key elements are combined to design the curriculum that can effectively
facilitate the high order thinking skills. The third module is detailed teaching strategies of
smart pedagogy including thinking classes, MOOCs, SPOC, personal inquiry learning,
collaborative learning, blended learning, mind map and concept map, etc.

The key elements model of the situated learning, mastery learning, adaptive learning,
reflective learning and thinking tools pedagogy
In the design of the key elements model of smart pedagogy, the facilitation of high order
thinking skills was chiefly considered. Though there are many ways to facilitate high order
thinking skills. To simplify the process of curriculum design, we should choose some key
elements of smart pedagogy that can be realized on smart equipment such as mobile phones
and panel PC. The key elements of pedagogy are not concrete ways to facilitate the high
order thinking skills, but the concept to design the curriculum and ITS. We integrate a
general element model of smart pedagogy including situated learning (S), mastery learning
(M), adaptive learning (A), reflective learning (R) and thinking tools (T), they can be
abbreviated as SMART model. The schematic diagram is shown in Figure 3.
As reviewed in Section 2, the problem-solving situation is a key method to develop high
order thinking skills. The situated learning is an indispensable element of smart pedagogy.
Mastery learning is proved to be an effect on improving the learning outcome of students.
By mastery learning, students can achieve lower-order thinking including remembering,
understanding and applying, which are the basic condition of developing high order

Figure 2.
The framework of
smart pedagogy
ITSE
17,3

256

Figure 3.
The schematic
diagram of the
SMART key
elements model

thinking skills. Adaptiveness is an important characteristic of smart education. The

adaptive learning can be realized by means of artificially intelligent technology. It is also a
key aspect that teachers should take into consideration. Reflective learning can be advocated
in personal learning and cooperative learning to excite the active thinking of students. The
thinking tools are the visible means of exercising and evaluating the thinking ability of
students. Some other methods such as visual learning and round table cooperative learning
are not involved in the key elements. The visual learning is a concrete way to present the
knowledge, which is not an essential method in smart education. Cooperative learning is a
commonly used strategy in smart education, but it is an implementation method to improve
the teaching outcome. So, we regard it as a teaching strategy not a key element of smart
pedagogy.
In Figure 3, the thinking tools lie in the center of the model, they are the exercising and
evaluating tools to realize deep thinking and they can support the realization of the other
four learning methods. They can be seen as a set of scaffoldings and applied to close the gap
between the low order thinking and high order thinking. The situated learning strategy is
used to create a problem situation. Mastery learning is applied to give students adequate
instructions, which can support independent thinking. The adaptive learning is used to
emphasize the differences of students. Reflective learning is applied to excite the active
thinking of students and help students construct knowledge. The role of thinking tools such
as concept maps is to assist students to achieve meaningful learning.
The five elements of SMART pedagogy can be used in the design of the curriculum and
ITS. The detailed description lies as follows:
(1) The situated learning strategy is applied to express knowledge in an intelligible
way and motivate students to think actively. Situated learning links the knowledge
to real-life events, activity and culture (Brown et al., 1989). Students learn by means
of authentic contexts and they are required to participate in authentic activities
that relevant to the real-world (McLellan, 1994). In the curriculum, the teachers
should describe more authentic information and pose more problems. In ITS, the
designers should arrange more questions that are related to the real-life event.
(2) Mastery learning is proposed to close the knowledge gap between low and high
achieving students by an associated set of instructional practices (Bloom, 1968).
The merit of Mastery learning to close the achievement gaps is proved by Guskey
 (2007). In the curriculum, the teachers should give sufficient time and appropriate Framework of
instruction. In ITS, the designer should arrange a set of instructional practices. smart
(3) Adaptive learning can help educators support different students personally. The pedagogy
concept of adaptive learning is not new. In recent years, adaptive learning’s
resurgence was because of the quick development of technology. Adaptive
learning allows different students to learn personalized content in their own pace
according to the readiness level, interest and student profile (Huang and Shiu, 257
2012). In the curriculum, the teachers should allow for personalized instruction
while altering their pathways through course objectives. In ITS, the designer
should continually assess the knowledge level of students and provide appropriate
learning resources.
(4) The reflective learning strategy is introduced to help students construct
knowledge. Reflection is an important procedure in learning and teaching. The
reflective thinking is defined by Dewey as “active, persistent and careful
consideration of any belief or supposed form of knowledge in the light of the
grounds that support it and the further conclusion to which it tends” (re-ed 1997).
Reflective thinking encourages deliberate and conscious thinking before and after
any action (Hatton and Smith, 1994). Teachers should organize collaborative
activities to help construct the knowledge by reflective learning and facilitate high
order thinking skills (Herrington and Oliver, 2000). In ITS, the designers should
advocate the reflection in the learning procedure.
(5) The thinking tools are applied to help students think effectively. There are many
kinds of thinking tools used in education such as concept maps (Novak and Gowin,
1985), thinking map (David and Larry, 2013) and mind map (Buzan, 2005).
Thinking in pictures is faster than thinking in words. Thinking tools can bring
images to mind and lower the cognitive load of thinking. Thinking tools can help
individuals or groups think visually and strengthen the efficiency of thinking.
Mixed with collaborative learning, thinking tools can enhance deeper learning
(Chen et al., 2018). In a computer-based environment, the thinking tools can also
promote the learning outcome by representing the problem-solving process (Wang
et al., 2018). In the curriculum, the teachers can guide students to use the thinking
tools. In ITS, the designer can also use them to display the knowledge visibly.

The curriculum design method of situated learning, mastery learning, adaptive learning,
reflective learning and thinking tools pedagogy framework
The most important application of the SMART key elements model is the design of the
curriculum. In the framework of smart pedagogy, we combine the first five principles of
instruction (Merrill, 2002, 2006) and the SMART key elements to design the curriculum that
can effectively facilitate the high order thinking skills. The first five principles of instruction
include task centeredness, activation, demonstration, application and integration, the
framework is shown in Figure 4. They are important criteria in the design of curriculum and
any high-quality instruction should meet the five principles.
(1) Task centeredness emphasizes that the instructions should be conducted around
the task-relevant to real-life problems.
(2) The activation principle reminds the educator to activate the prior knowledge to
promote the learning procedure.
ITSE (3) The demonstration principle suggests teachers demonstrate more specific
17,3 examples to help the learners remember and apply the knowledge.
(4) The application principle indicates that learning can be promoted when learners
use their knowledge to solve problems.
(5) The integration principle emphasizes that skills are facilitated when learners
integrate their knowledge into their life.
258
Based on the first five principles of instruction, we can optimize the curriculum design by
SMART key elements model to enhance the effectiveness of facilitating the high order
thinking skills. We construct a coordinated system to describe the curriculum design
method of the SMART key elements model shown in Figure 5.
The horizontal axis is the five principles of instruction proposed by Merrill. These are the
design criteria of the curriculum. The vertical axis is the five learning strategies of the
SMART key elements model. These are effective methods to facilitate high order thinking
skills. The junction points are the optimizations by the SMART key elements model in the
corresponding procedure. For example, the junction point of demonstration and thinking
tools means that the teacher should consider the use of thinking tools to demonstrate the
new knowledge.

Figure 4.
The framework of the
first five principles of
instruction

Figure 5.
The curriculum
design method of
smart pedagogy
framework
Methodology Framework of
Quasi-experiment was conducted to verify the effectiveness of the smart pedagogy smart
framework. In the experiment, the experimental class used the smart pedagogy framework
mentioned before and equipped with a smart learning environment. The control class
pedagogy
adopted the conventional teaching strategy. After one school year, we compared the pre-test
scores and post-test scores by covariance analysis method, the results showed that the
teaching effect of the experimental class is better than that of the control class and the
differences were statistically significant. The results of a questionnaire filled by
259
experimental class students showed that the learning procedure was satisfying. Through
the above quasi-experimental research, we can conclude that the use of smart pedagogy can
improve students’ learning outcomes.

Participants
A quasi-experimental design was carried out at a middle school in Shanghai, China. A total
of 99 eighth grade students (47 men and 52 women) participated in this study, of whom 51
(24 men and 27 women) were in the experimental class and 48 (23 men and 25 women) were
in the control class. To avoid the influence of different teachers on the experimental
outcomes, the two classes were instructed by the same math teacher. In the experiment, the
experimental class used the smart pedagogy framework mentioned before and equipped
with a smart learning environment. The control class adopted the conventional teaching
strategy. In the academic year of 2018–2019 (from September 2018 to July 2019), the
students in the experimental class need to learn in the smart learning environment and
use the smart pedagogy framework, while the students in the control class adopted the
conventional teaching strategy. The learning content of the two classes is math in eighth
grade and the textbook and examination are all the same.

Experimental process
Before the experiment, the math teacher of the two classes were trained to use the smart
equipment and software environment. The courses on the smart pedagogy mentioned above
were provided to the teacher. In July 2018, an examination was conducted to verify the level
of the students.
The ITS used in this study is the “Lexue 100” mathematical elearning system (Zhang and
Jia, 2017). The system focuses on mathematics teaching in middle school. Based on the
learning theory, this system emphasizes the concepts of adaptive, interactive and mastery
learning. The system can give instant feedback. After the student completes the answer, the
system will adaptively recommend the appropriate content based on the student’s answers
(Jia and Zhang, 2019). The “Lexue 100” system is treated as a key part of a smart learning
environment. The experimental class uses this system to learn in class and at home and
different students learn in different pathways. Besides the smart learning environments,
smart pedagogy is adopted in the experiment.
The detailed teaching strategies are designed according to the framework of smart
pedagogy.
Situated learning. The teacher of the experimental class uses the problem-driven
strategies to inspire the inner passion and learning interest of students. This pedagogy
meets the requirement of situated learning.
Mastery learning. A blended learning strategy is applied in the experimental class, the
“Lexue 100” ITS is used in the daily curriculums. The setting of “Lexue 100” system is
designed according to the mastery learning, students follow the instructions step by step.
ITSE Adaptive learning. The “Lexue 100” elearning system can achieve adaptive learning, i.e.
17,3 different learners can study in different pathways according to their knowledge level. The
teacher can instantaneously know the problems of each student. The teachers also use the
SPOC strategy to teach the appropriate material to special students.
Reflective learning. Self-organized learning and collaborative learning are integrated in
the experimental class. The simple content in the courses is studied by means of self-
260 organized learning, while the complex and open-ended questions are studied using the
collaborative learning strategy. Throughout the learning procedure, the systematic, rigorous
and disciplined ways of thinking are emphasized to realize the reflective learning.
Thinking tools. The “innovative thinking course,” which lasts about 12 h per semester is
held in the experimental class. The content of course includes thinking games, mind maps,
reading, memory and critical thinking. Through this course, students’ enthusiasm for
learning is inspired and the mind map is introduced in the teaching. The mind map is used
to be scaffolding to construct math knowledge. The mind map drawn by students is shown
in Figure 6.
Curriculum design. The curriculum is designed following the first five principles of
instruction and SMART key elements model. The teaching strategies mentioned above are
flexibly applied in the courses.
The math teacher taught the control class using conventional strategies including
traditional lectures and assigning the homework in the textbook. The teacher adopted the
electronic whiteboard to display the knowledge and the ITS was not used in the class. The
experimental class used the “Lexue 100” system in every math class and the curriculum was
designed under the direction of smart pedagogy. After one school year, the examination is
conducted in July 2019. We compared the pre-test scores and post-test scores by the
covariance analysis method. The results experimental class students were asked to fill in the
questionnaire to evaluate the satisfaction degree.

Results
To verify the teaching effect of the framework of smart pedagogy, the quasi-experimental
research was carried out. After one year, the learning outcome of the two classes was
compared. The experimental results proved that the framework of the smart pedagogy
proposed in this paper can improve the teaching effect.

Figure 6.
The mind map was
drawn by a student
Before the experiment, the pre-test scores of the two classes were recorded in July 2018. Framework of
After the experiment, the examination held in July 2019 was treated as a post-test. The smart
results of the experimental class and the control class were compared by analyzing the test
scores. The pre-test and post-test scores of the experimental class and control class were
pedagogy
listed in Table 1.
Because the difficulty of the post-test was higher than that of the pre-test, the average
score of the post-test was lower than that of the pre-test. In the pre-test, the average score of
the experimental class was lower than that of the control class, but in the post-test, the 261
average score of the experimental class was higher than the control class. We used
the covariance analysis method to evaluate whether there was any significant difference
between the achievement scores of the two classes, the post-test score is the dependent
variable, the pre-test score is covariate variable. After the covariance correction, the average
score of the experimental class was 5.923 higher than the control class and the p-value was
0.004 < 0.05, indicating that the application of the smart pedagogy and smart learning
environment had a significant positive impact on the scores.
We also evaluated the satisfaction degree of smart education by questionnaires. The
students in the experimental class filled in the five-point Likert scales, 5 is strongly agreed, 4
is agreed, 3 is not sure, 2 is disagree, 1 is strongly disagree. There are 15 questions in the
questionnaires, the questions and average choice score are listed in Table 2.
The average choice score of all the questions is 4.08 that means most of the students
agree that the learning procedure is satisfying.

Discussion and conclusion

The study aimed to construct a general framework of smart pedagogy based on the
development of high order thinking skills and to examine the effect of whether the

Class Pre-test average score Post-test average score

Table 1.
Control class 64.96 57.31 The average score of
Experimental class 60.78 59.74 different classes

No. Questions Average choice score

1 I am very interested in mathematics learning 4.17

2 I am very confident in learning mathematics 4.08
3 I like to explore problems related to mathematics 4.08
4 I like to participate in math-related activities 4.00
5 I like to do math exercises 3.92
6 I have a sense of achievement after solving the math problem 4.67
7 I am not worried about mathematics learning 3.92
8 I often preview the mathematics content to be learned tomorrow 3.92
9 I can understand the contents of the teacher’s explanation in class 4.42
10 I often answer teacher’s questions in class 3.92
11 I can accurately remember the formula 4.42
12 I am able to solve the problem using mathematical formulas 4.00 Table 2.
13 I often review what I have learned after class 4.08 The results of
14 I correct my mistakes in math exercises 4.42 learning outcome
15 I try different ways to solve the math problem 4.42 questionnaires
ITSE framework would enhance students’ learning outcomes. The implications of the findings are
17,3 discussed as follows.
The findings provide evidence to support smart pedagogy based on the development of
high order thinking skills. The previous study was focused on a certain application field
(Zhou et al., 2018; Nobuyuki and Akira, 2018; Wu et al., 2019). The smart pedagogy
framework was a general method that can be used in different subjects and grades.
262 In the quasi-experiment, the students of the experimental class used the “Lexue 100”
system in every math class. The “Lexue 100” system could assess the knowledge that the
students did not grasp and then provide the appropriate learning sources to a certain
student. In the “Lexue 100” system, many questions base on the real-life were designed to
realize the situated learning. The sequences of learning were arranged under the direction of
mastery learning. The math teacher used self-organized learning and collaborative learning
to activate the reflection of students. The “innovative thinking course” was held in the
experimental class. The content of course included thinking games, mind maps, reading,
memory and critical thinking. The thinking tools were used to be scaffolding to construct
math knowledge. In addition, the math curriculum was designed following the first five
principles of instruction and SMART key elements model. While the control class was
taught in conventional methods by the electronic whiteboard.
In smart education, smart devices cannot ensure the improvement of thinking ability, the
technology and pedagogy need to be combined (Sung et al., 2018). While the results in
Table 1 revealed that the average math score of the experimental class increased rapidly and
there was a significant improvement above the control class. It proved that smart pedagogy
had a positive influence on the learning outcome. According to the questionnaire, results
showed in Table 2, we can also verify that the learning procedure of the experimental class
was satisfactory.
Related to the smart pedagogy framework showed in Figure 2, the situated learning can
increase the learning interest (Brown et al., 1989), the situated learning realized in the “Lexue
100” system had enhanced the mathematics learning interesting. Mastery learning
instructional strategy has positive effects on improving the students’ achievement,
especially the effects that are stronger on the weaker students (Kuilk and Bangertdrowns,
1990). The “Lexue 100” system is designed according to mastery learning. In the
examination in July 2018, the average score of the experimental class is 4.18 points lower
than that of the control class. While in the examination in July 2019, the average score of the
experimental class is 2.43 points higher than that of the control class. The mastery learning
played an important role in the improvement of math score. The benefit of adaptive learning
has been proved by previous experiments (Liu et al., 2017). Adaptive learning was also
realized in the “Lexue 100” system and help the students facilitate high order thinking skills.
Reflective learning pedagogy is an important way to facilitate high order thinking
(Beckwith, 2016). By combining the self-organized learning and collaborative learning, the
reflection was activated. Thinking tools such as concept maps have been proved to be an
effect on developing high order thinking skills (Canas et al., 2017). The mind map was
regularly used in the math class.
In summary, the results of this study imply that smart pedagogy is effective in the math
learning of the junior high school. The curriculum and ITS can be designed according to the
smart pedagogy framework. The framework can be used in different subjects and grades.
The following suggestions can be derived from the conclusions of this study. In the smart
learning environment, the teachers should consider the facilitation of students’ high order
thinking skills and use the situated learning, mastery learning, adaptive learning, reflective
learning and thinking tools when they are designing the curriculum.
 There are several limitations that need to be considered for future research. First, we only Framework of
examined the effect on math learning of junior high school, the other subjects such as smart
languages and sciences should be considered to verify the effectiveness of smart pedagogy.
Second, the sample size was limited, we may encounter new problems when using the smart
pedagogy
pedagogy on the school or community level. Third, the framework of smart pedagogy needs
to be further optimized in future research.
263
References
Anderson, O.W. and Krathwohl, D.R. (2001), Taxonomy for Learning, Teaching, and Assessing: A
Revision of Bloom’s Taxonomy of Educational Objectives, Addison Wesley Longman,
New York, NY.
Beckwith, P. (2016), “Developing higher-order thinking in medical education through reflective learning
and research”, Journal of Pedagogic Development, Vol. 6 No. 3, pp. 125-134.
Bloom, B.S. (1968), “Learning for mastery”, Evaluation Comment, Vol. 1 No. 2, pp. 1-12.
Brown, J.S., Collins, A. and Duguid, P. (1989), “Situated cognition and the culture of learning”,
Educational Researcher, Vol. 18 No. 1, pp. 32-42.
Buzan, T. (2005), Mind Map Handbook, Thorsons, Great Britain.
Canas, A., Reiska, P. and Mollits, A. (2017), “Developing higher-order thinking skills with concept
mapping: a case of pedagogic frailty”, Knowledge Management and E-Learning, Vol. 9 No. 3,
pp. 348-365.
Cavanaugh, C., Hargis, J. and Kamali, T. (2013), “Substitution to augmentation faculty adoption of iPad
mobile learning in higher education”, Interactive Technology and Smart Education, Vol. 10 No. 4,
p. 270.
Chen, W., Allen, C. and Jonassen, D. (2018), “Deeper learning in collaborative concept mapping: a
mixed-methods study of conflict resolution”, Computers in Human Behavior, Vol. 87,
pp. 424-435.
Coccoli, M., Guercio, A., Maresca, P. and Stanganelli, L. (2014), “Smarter universities: a vision for the
fast-changing digital era”, Journal of Visual Languages and Computing, Vol. 25 No. 6,
pp. 1003-1011.
David, H. and Larry, A. (2013), Student Successes with Thinking Maps, Second Edition, School-Based
Research, Results, and Models for Achievement Using Visual Tools, Corwin Press, Thousand
Oaks.
Erdogan, F. (2019), “Effect of cooperative learning supported by reflective thinking activities on
students’ critical thinking skills”, Eurasian Journal of Educational Research, Vol. 19 No. 80,
pp. 89-112.
Ghani, I., Ibrahim, N., Yahaya, N. and Surif, J. (2017), “Enhancing students’ HOTS in laboratory
educational activity by using concept map as an alternative assessment tool”, Chemistry
Education Research and Practice, Vol. 18 No. 4, pp. 875-892.
Gros, B. (2016), “The design of smart educational environments”, Smart Learning Environments, Vol. 3
No. 1, pp. 15-26.
Guskey, T.R. (2007), “Closing achievement gaps: revisiting Benjamin S. Bloom’s ‘learning for mastery”,
Journal of Advanced Academics, Vol. 19 No. 1, pp. 8-31.
Harry, Y., Budi, S. and Ery, D. (2017), “The application of carousel feedback and round table
cooperative learning models to improve student’s higher-order thinking skills and social studies
learning outcomes”, International Education Studies, Vol. 10 No. 10, pp. 39-49.
Hatton, N. and Smith, D. (1994), “Facilitating reflection: issues and research”, Proceedings from the 24th
Conference of the Australian Teacher Education Association, available at: https://eric.ed.gov/?
id=ED375110
ITSE Herrington, J. and Oliver, R. (2000), “An instructional design framework for authentic learning
environments”, Educational Technology Research and Development, Vol. 48 No. 3, pp. 23-48.
17,3
Hogue, A., Kapralos, B. and Desjardins, F. (2011), “The role of project-based learning in IT a case study
in a game development and entrepreneurship program”, Interactive Technology and Smart
Education, Vol. 8 No. 2, p. 120.
Huang, S. and Shiu, J. (2012), “A User-Centric adaptive learning system for E-Learning 2.0”, Educational
264 Technology Society, Vol. 15 No. 3, pp. 214-225.
IBM (1997), “Smart education”, available at: www.ibm.com/smarterplanet/global/files/
auenukcities_ibm_smarter_education_now.pdf
Jia, J. (2015), “Intelligent tutoring systems”, in Spector, M. (Ed.), Encyclopedia of Educational
Technology, Sage, Thousand Oaks, CA, pp. 411-413.
Jia, J. and Zhang, J. (2019), “The analysis of online learning behavior of the students with poor academic
performance in mathematics and individual help strategies”, Blended Learning: Educational
Innovation for Personalized Learning, ICBL, pp. 205-215. July 02-04, 2019.
JISC (2009), Designing Spaces for Effective Learning: A Guide to 21st Century Learning Space Design,
JISC, Bristol, available at: www.jisc.ac.uk/media/documents/publications/learningspaces.pdf
Julius, E., Soh, H.M., Abdullah, A.H., Mokhtar, M. and Suhairom, N. (2018), “Using a digital smart board
to overcome higher-order thinking skills learning difficulties in data handling among primary
school students”, International Journal of Interactive Mobile Technologies (IJIM), Vol. 12 No. 7,
pp. 43-59.
Kapralos, B., Fisher, S. and Clarkson, J. (2015), “A course on serious game design and development
using an online problem-based learning approach”, Interactive Technology and Smart Education,
Vol. 12 No. 2, pp. 116-136.
Khan, F. and Masood, M. (2014), “The effectiveness of interactive multimedia courseware with
cooperative mastery approach in enhancing higher-order thinking skills in learning cellular
respiration”, International Educational Technology Conference 2014, Vol. 14, pp. 977-984.
Kim, S., Song, S.M. and Yoon, Y.I. (2013), “Smart learning services based on smart cloud computing”,
Sensors, Vol. 11 No. 8, pp. 7835-7850.
Kuilk, C. and Bangertdrowns, R. (1990), “Effectiveness of mastery learning-programs – a meta-
analysis”, Review of Educational Research, Vol. 60 No. 2, pp. 265-299.
Lee, J., Zo, H. and Lee, H. (2014), “Smart learning adoption in employees and HRD managers”, British
Journal of Educational Technology, Vol. 45 No. 6, pp. 1082-1096.
Liu, M., McKelroy, E., Corliss, S. and Carrigan, J. (2017), “Investigating the effect of an adaptive learning
intervention on students’ learning”, Educational Technology Research and Development, Vol. 65
No. 6, pp. 1605-1625.
McLellan, H. (1994), “Situated learning: continuing the conversation”, Educational Technology, Vol. 34
No. 10, pp. 7-8.
Merrill, M.D. (2002), “First-principles of instruction”, Educational Technology Research and
Development, Vol. 50 No. 3, pp. 43-59.
Merrill, M.D. (2006), Handling Complexity in Learning Environments; Theory and Practice, Elsevier,
Amsterdam.
MEST: Ministry of Education, Science and Technology of the Republic of Korea (2011), “Smart
education promotion strategy, president’s council on national ICT strategies”.
Mironova, O., Amitan, I. and Vendelin, J. (2016), “Maximizing and personalizing e-learning support for
students with different backgrounds and preferences”, Interactive Technology and Smart
Education, Vol. 13 No. 1, pp. 19-35.
Newman, F. (1991), “Promoting high order thinking skills in social studies: overview of a study of 16
higher school department”, Theory and Research in Social Education, Vol. 19 No. 4, pp. 324-340.
Nobuyuki, O. and Akira, S. (2018), “Practice of organizational strategies of improving computer rooms Framework of
for promoting smart education using ICT equipment”, 2017 International Association for
Development of the Information Society International Conference on E-Learning (IADIS), Lisbon,
smart
20-22 July. 2017. pedagogy
Novak, J.D. and Gowin, B. (1985), Learning How to Learn, Cambridge University, Oxford.
OECD (2015), Students, Computers and Learning: Making the Connection, OECD Publishing, Paris.
Pegg, J. (2010), “Promoting the acquisition of higher-order skills and understandings in primary and
secondary mathematics”, Research Conference 2010, pp. 35-38.
265
Peters, M. and Araya, D. (2010), “Transforming American education: learning powered by technology”,
E-Learning and Digital Media, Vol. 8 No. 2, pp. 11-24.
Purbohadi, D., Nugroho, L., Santosa, I. and Kumara, A. (2014), “Design of intelligent tutoring system for
mastery learning”, Advanced Science Letters, Vol. 20 No. 1, pp. 213-217.
Rabadi, A., Minwer, W. and Rifqa, S. (2018), “The level of high order thinking and its relation to the
quality of life among students at Ajloun university college”, International Education Studies,
Vol. 11 No. 6, pp. 8-21.
Raiyn, J. (2016), “The role of visual learning in improving students’ high order thinking skills”, Journal
of Education and Practice, Vol. 7 No. 24, pp. 115-121.
Raiyn, J. and Tilchin, O. (2015), “Higher-Order thinking development through adaptive problem-based
learning”, Journal of Education and Training Studies, Vol. 3 No. 4, pp. 93-100.
Ramsden, B. (2011), “Evaluating the impact of learning space”, Reference Services Review, Vol. 39 No. 3,
pp. 451-464.
Shelle, G., Earnesty, D., Pilkenton, A. and Powell, E. (2018), “Adaptive learning: an innovative method
for online teaching and learning”, Journal of Extension, Vol. 56 No. 5, pp. 234-248.
Spector, J.M. (2018), “Smart learning futures: a report from the 3rd US-China smart education
conference”, Smart Learning Environments, Vol. 5 No. 1, pp. 5-15.
Stahl, G., Koschmann, T. and Suthers, D. (2006), “Computer-supported collaborative learning: a
historical perspective”, Cambridge handbook of the learning sciences 2006, pp. 409-426.
Sung, H., Hwang, G. and Chen, S. (2018), “Effects of embedding a problem-posing-based learning
guiding strategy into interactive e-books on students’ learning performance and higher-order
thinking tendency”, Interactive Learning Environments, Vol. 27 No. 3, pp. 389-401.
Suprapto, E., Fahrizal, F., Priyono, P. and Basri, K. (2017), “The application of problem-based learning
strategy to increase high order thinking skills of senior vocational school students”,
International Education Studies, Vol. 10 No. 6, pp. 123-129.
Uskov, V.L., Bakken, J.P., Howlett, R.J. and Jain, J.C. (2018), Smart Universities: Concepts, Systems and
Technologies, Springer, p. 421, ISBN 978-3-319-59453-8.
Villesseche, J., Le Bohec, O. and Quaireau, C. (2019), “Enhancing reading skills through adaptive e-
learning”, Interactive Technology and Smart Education, Vol. 16 No. 1, pp. 2-17.
Wang, M., Wu, B., Kirschner, P.A. and Spector, J.M. (2018), “Using cognitive mapping to foster deeper
learning with complex problems in a computer-based environment”, Computers in Human
Behavior, Vol. 87, pp. 450-458.
Wu, D., Xing, D. and Lu, C. (2019), “The effects of learner factors on higher-order thinking in the smart
classroom environment”, Journal of Computer in Education, Vol. 6 No. 4, pp. 483-498.
Zhang, B. and Jia, J. (2017), “Evaluating an intelligent tutoring system for personalized math teaching”,
Proceedings of International Symposium on Educational Technology 2017, pp. 126-130.
Zhou, P., Wang, Q., Zhang, Y. and Yang, Q. (2018), “The influence of debating teaching on students’
critical thinking development in the smart classroom”, 2018 Seventh International Conference of
Educational Innovation through Technology, 12-14 Dec, IEEE, Auckland, doi: 10.1109/
EITT.2018.00018.
ITSE Zhu, Z. and He, B. (2012), “Smart education: a new frontier of educational informatization”, E-Education
Research, Vol. 12, pp. 1-13.
17,3
Zhu, Z., Yang, M. and Riezebos, P. (2016), “A research framework of smart education”, Smart Learning
Environments, Vol. 3 No. 1, pp. 4-21.

Further readings
266 Dewey, J. (1997), How we Think, Courier Corporation, North Chelmsford.
Rodgers, C. (2002), “Defining reflection: another look at John Dewey and reflective thinking”, Teachers
College Record, Vol. 104 No. 4, pp. 842-866.
Yang, J., Pan, H., Zhou, W. and Huang, R. (2018), “Evaluation of smart classroom from the perspective
of infusing technology into pedagogy”, Smart Learning Environments, Vol. 5 No. 1, pp. 20-31.
Yukselturk, E., Ozekes, S. and Türel, Y.K. (2014), “Predicting dropout student: an application of data
mining methods in an online education program”, European Journal of Open, Distance and E-
Learning, Vol. 17 No. 1, pp. 118-133.

About the authors

Qingquan Meng is the Postdoctoral Fellow in the Department of Educational Technology, Graduate
School of Education of Peking University. His research areas include artificial intelligence in
education, facilitating of creative thinking and thinking tools. Qingquan Meng is the corresponding
author and can be contacted at: qqmeng@gse.pku.edu.cn
Jiyou Jia is Professor and Head of Department of Educational Technology, Graduate School of
Education, Peking University and the Director of International Research Center for Education and
Information in Peking University. His research areas include ITS and artificial intelligence.
Zhiyong Zhang is a Math Teacher in the Huangdu middle school in Shanghai, China.

For instructions on how to order reprints of this article, please visit our website:
www.emeraldgrouppublishing.com/licensing/reprints.htm
Or contact us for further details: permissions@emeraldinsight.com