Problems of Education in The 21st Century, Vol. 77, No. 3, 2019

Problems of Education in the 21st Century, Vol. 77, No.
3, 2019
ISSN 1822-7864 (Print) ISSN 2538-7111 (Online)
PROBLEMS OF EDUCATION
IN THE 21st CENTURY
Scientia Socialis, Ltd., a member of the Lithuanian Publishers Association and a member of
Publishers International Linking Association, Inc. (Crossref) in cooperation with SMC „Scientia
Educologica“, Lithuania, The Associated Member of Lithuanian Scientific Society, European
Society for the History of Science (ESHS) and ICASE (International Council of Associations for
Science Education)
The articles appearing in this journal are indexed, listed, and abstracted in:
Emerging Sources Citation Index (ESCI) (Web of GIGA Information Centre (Electronic Journals
Science | Clarivate Analytics) Library)
SCOPUS CNKI (China National Knowledge Infrastructure)
Index Copernicus Scholar
EBSCO: Education Research Complete Turkish Education Index
Education Research Index Database Coverage List Information Matrix for the Analysis of Journals
“Core” coverage (MIAR)
Central & Eastern European Academic Source Directory of Open Access Scholarly Resources
Database Coverage List (EBSCO-CEEAS) (ROAD)
JournalSeek Eurasian Scientific Journal Index
Ulrich`s Periodicals Directory Crossref
Contemporary Science Association/AAP databases ProQuest
Included into a List of Journals Recognized by Polish Europub: Academic and Scholarly Research
Ministry of Science and Higher Education Publication Center
Open Academic Journals Index (OAJI)
Publisher
Scientia Socialis, Ltd. in cooperation with SMC „Scientia Educologica“, Lithuania, the Associated
Member of Lithuanian Scientific Society, the Association of Lithuanian Serials, European Society
for the History of Science (ESHS) and International Council of Associations for Science Education
(ICASE)
Editors-in-Chief
Dr., Prof. Agnaldo Arroio, University of Sao Paulo, Brazil
Dr., Prof. Vincentas Lamanauskas, Šiauliai University & SMC „Scientia Educologica“, Republic
of Lithuania
Editorial Board
Dr., Prof. Boris Aberšek, University of Maribor, Slovenia
Dr., Prof. Saleh A. Alabdulkareem, King Saud University, Saudi Arabia
Dr. Monica Baptista, University of Lisbon, Portugal
Dr., Prof. Martin Bilek, Charles University, Czech Republic
Dr., Prof. Andris Broks, University of Latvia, Latvia
Dr., Prof. Marco Antonio Bueno Filho, Federal University of ABC, Brazil
Dr. Paolo Bussotti, University of Udine, Italy
Dr. Muammer Calik, Karadeniz Technical University, Turkey
Dr., Prof. Margaret Chan Kit Yok, MARA University of Technology (UiTM), Malaysia
Dr., Harun Yilmaz, Scientific and Technological Research Council of Turkey, Turkey
Dr., Angela James, University of Kwazulu – Natal, South Africa
Dr., Prof. Vladimir S. Karapetyan, Armenian State Pedagogical University named after
Kh. Abovyan, Armenia
Dr., Prof. Kuo-Hung Huang, National Chiayi University, Taiwan
Dr. Milan Kubiatko, University of Zilina, Slovakia
Dr., Prof. Miroslaw Kowalski, University of Zielona Gora, Poland
Dr. Todar T. Lakhvich, Belarusian State Medical University, Republic of Belarus
Dr. Eleonora Melnik, Petrozavodsk State University, Republic of Karelia, Russia
Dr., Prof. Danuše Nezvalova, Palacky University, Czech Republic
Dr. Osman Pekel, Suleyman Demirel University, Turkey
Dr., Prof. Yuriy Pelekh, Rivne State University of Humanities, Ukraine
Dr., Prof. Raffaele Pisano, Lille 3 University, France
Dr. Costin Pribeanu, Academy of Romanian Scientists, Romania
Dr. Agneta Simeonsdotter Svensson, University of Gothenburg, Sweden
Dr. Uladzimir K. Slabin, University of Oregon, USA
Dr. Laima Railienė, Scientific Methodical Centre „Scientia Educologica“, Republic of Lithuania
Dr., Prof. Borislav V. Toshev, Sofia University, Bulgaria
Dr., Prof. Milan Turčani, Constantine the Philosopher University, Slovakia
Dr., Prof. Nicos Valanides, Educational Robotics and Science Organization, Cyprus
Copyright of this issue is the property of Scientia Socialis, Ltd, Lithuania. By virtue of their appearance in this
open access journal, articles are free to use, with proper attribution, in educational and other non-commercial
settings
Index Copernicus (IC™ Value): 87.04 (2015)
ICDS (Secondary Composite Index Broadcasting): 9.5 (2018).
CGIJ OAJI: 0.350 (2018).
H Index: 3 (2018)
Problems of Education in the 21st Century is an international, periodical, peer reviewed scientific journal,
issued by the Scientia Socialis Ltd. in cooperation with SMC „Scientia Educologica“.
Address: Scientia Socialis, Ltd.

Donelaičio Street 29, LT-78115 Siauliai, Lithuania
E-mail: problemsofeducation@gmail.com
Phone: +370 687 95668
WEB page: http://www.scientiasocialis.lt/pec
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
Contents 313
Editorial
EDUCATION AS A HUMAN RIGHT AGAINST HATE SPEECH AND INTOLERANCE

Agnaldo Arroio ....................................................................................................... 314
Articles
CHARACTERISTICS OF INDEPENDENT SCHOOLS DIRECTED AT STUDENTS IN

NEED OF SPECIAL SUPPORT: A STUDY OF SCHOOL WEBSITE PRESENTATION
Lotta Anderson, Gunvie Möllås, Lisbeth Ohlsson ............................................................... 317
DEVELOPING OF FACTOR STRUCTURE FOR LEARNING STRATEGIES OF ESTONIAN

STUDENTS IN MATHEMATICS AT THE UNIVERSITY LEVEL
Indrek Kaldo, Kandela Õun .........................................................................................338
CLASSIFICATION OF ENGINEERING STUDENTS' SELF-EFFICACY TOWARDS

VISUAL-VERBAL PREFERENCES USING DATA MINING METHODS
Citra Kurniawan, Punaji Setyosari, Waras Kamdi, Saida Ulfa .................................................. 349
THE FORMATIVE RESEARCH OF ECUADORIAN UNIVERSITY TEACHING STAFF

Derling Mendoza Velazco, Irma Abrigo Córdova, Janela Romero Chávez,
Fredy Cueva Bravo, Magda Francisca Cejas .......................................................................364
ON PERCEPTION OF COMPUTER ALGEBRA SYSTEMS AND MICROSOFT

EXCEL BY ENGINEERING STUDENTS
Natalia M. Mezhennaya, Oleg V. Pugachev ....................................................................... 379
EFFICIENCY OF PUBLIC SPENDING ON HIGHER EDUCATION:

A DATA ENVELOPMENT ANALYSIS FOR EU-28
Maja Mihaljevic Kosor, Lena Malesevic Perovic, Silva Golem ................................................ 396
EXPLORING THE STUDENT AND SOCIAL ACCOUNTABILITY OF THE LIFE

SCIENCES CURRICULUM: A CASE OF HIV/AIDS
Lindelani Mnguni .................................................................................................... 410
THE OPINIONS OF PRE-PRIMARY EDUCATION TEACHERS ON THE ADVANTAGES

AND DISADVANTAGES OF HETEROGENEOUS AGE GROUPS
Ivana Rochovska, Zlatica Huľová, Vladimír Klein, Miroslava Gašparová ................................... 424
LEARNING EXPERIENCE AS A FACTOR OF MOTIVATION IN LOWER-SECONDARY

SCHOOL IN FOREIGN LANGUAGE LEARNING
Eva Stranovská, Anikó Ficzere, Silvia Hvozdíková, Beáta Hockicková ...................................... 437
Information
INSTRUCTIONS FOR AUTHORS ............................................................................. 449

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
314
EDUCATION AS A HUMAN RIGHT
AGAINST HATE SPEECH AND
INTOLERANCE
Agnaldo Arroio
University of Sao Paulo, Brazil
E-mail: agnaldoarroio@yahoo.com
On January 24, 2019, the International Day of Education was celebrated for the first time.
One of the celebrations was a speech by the Secretary-General of the United Nations (UN),
in which Mr. António Guterres highlighted the role of education in combating hate speech,
intolerance in various aspects and also in xenophobia.
In the words of the Secretary-General of the UN: "Such a situation constitutes a violation
of his fundamental right to education. The world cannot afford to deprive a generation of
children and young people of the knowledge they will need to have a place in the economy of
the 21st century. "
In 2019, there are still 262 million children and young people who do not have access to
school, and most of these children and young people are girls who are in a situation of exclusion.
In 2015 between September 25 and 27, Heads of State and Government and senior representatives
from various countries met at United Nations Headquarters in New York when they celebrated
the 70th anniversary of the United Nations and decided on the new objectives’ development,
setting the 2030 Agenda.
The 17 Sustainable Development Goals (SDG) and 169 goals were announced to
demonstrate the scale and ambition of this new Universal Agenda. The Sustainable Development
Goals seek to realize the human rights of all and achieve gender equality and the empowerment
of women and girls. They are integrated and indivisible and balance the three dimensions of
sustainable development: economic, social and environmental.
We emphasize education, considering its potential to reduce inequalities and improve
people's health, achieve equality between men and women and end child marriage, protect
the planet's resources and also combat hate speech, xenophobia and intolerance, in addition to
cultivating a world citizenship.
Malala Yousafzai, a UN peace activist and messenger, made a statement that had a
world-class phrase when she said that "a child, a teacher, a book and a pencil can change the
world." According to the UN, 420 million people could be lifted out of poverty if all girls and
boys completed secondary school, breaking with cycles of intergenerational poverty.
UNESCO Director-General Audrey Azoulay said, "Without inclusive and equitable
quality education and opportunities for all throughout life, countries will not be able to break
the cycle of poverty that is leaving millions of children, young people and adults back". A recent
UNESCO report reveals that 617 million of children and adolescents cannot read or solve
basic math equations; in this group two-thirds are young people who attend school but are not
learning.
The Sustainable Development Goal - SDG 4 provides for the universalization of education
to occur by 2030, considering pre-school education and primary and secondary education. In
addition, it determines that, over the next 12 years, nations will have to achieve equal access for
all men and women to affordable, quality technical, vocational and higher education.
https://doi.org/10.33225/pec/19.77.314 ISSN 1822-7864 (Print) ISSN 2538-7111 (Online)

Agnaldo ARROIO. Education as a human right against hate speech and intolerance
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
According to UNESCO, the world is not on track to reach this agenda by 2030. In 315
addition to the numbers of out-of-school children or girls who are not learning, fewer than
40 percent of girls in sub-Saharan Africa complete high school, and about 4 million refugee
children and youths are out of school, their lives interrupted by conflict and loss.
International relations, especially among the great power states, are not avoiding political
tensions, destabilization and civil war, and between states in the world. Since 2011, many
things have changed in the world in this period, but little has been solved, since the protests in
Syria began that culminated in a civil war. A very relevant example of this is the impact of an
estimated 5.6 million refugees and 6.5 million internally displaced persons over eight years of
conflict, with no prospects for resolution.
Most of the current tensions are concentrated in the continents of Africa and Asia, with
a focus on the Middle East, a region known for its ancient and seemingly unsolved conflicts.
In addition to the states of violence in the Democratic Republic of Congo, Central African
Republic and Mexico; political instability in Lebanon and Egypt; and territorial disputes in
the Kashmir region between India and Pakistan and the South China Sea with Vietnam and
the Philippines. It is not by chance that the greatest flow of refugees comes from these regions.
The Rohingya Muslim group faces persecution and discrimination in their own country,
whose members, who account for more than one million people, are not even considered citizens
by the Myanmar government. The situation has been blamed by the UN as "ethnic cleansing"
and the humanitarian crisis has grown to alarming numbers, with Bangladesh now housing the
largest refugee camp with about one million Rohingya. Unfortunately, this is just an example,
because there are much more situations like this.
We agree with the director of UNESCO when she emphasizes that "Our challenge is
to make education work for all, promoting inclusiveness and equity in all areas, so as not to
leave anyone behind. This requires special attention for girls, migrants, displaced persons and
refugees, (in addition to) supporting teachers and making education and training more sensitive
to gender issues ".
However, considering the scale of the problem, we emphasize the urgency of States'
efforts in international aid, since the cost of not investing in education in these conflict regions
will result in divisions, inequalities and exclusions in societies, further aggravating problems,
with the massive displacements of these populations. A fully committed political commitment
to universal education is needed so that states and their institutions prioritize education by
ensuring that all girls and boys complete free, fair and quality primary and secondary education
leading to relevant learning outcomes and effectiveness. It also means that all girls and boys
have access to quality early childhood development, early childhood care and education, so that
they are ready for primary education.
Significantly reducing all forms of violence and related death rates everywhere, ending
abuse, exploitation, trafficking and all forms of violence and torture against children.
A fair, egalitarian society only occurs with the elimination of gender disparities in
education and ensuring equal access to all levels of education and vocational training for the
most vulnerable, including people with disabilities, indigenous people and children in situations
of vulnerability.
It is not enough that children have access to school, but education must be of a quality so
that all students acquire the knowledge and skills necessary to promote sustainable development
and a culture of peace, valuing cultural diversity.
Do we need to wait until 2030 to achieve sustainable development goals with a focus
on SDG 4 - Ensure inclusive and equitable quality education and promote lifelong learning
opportunities for all we have more of the conflicts and tensions that generate violence, exclusion,
and migratory flows?
ISSN 1822-7864 (Print) ISSN 2538-7111 (Online) https://doi.org/10.33225/pec/19.77.314

Agnaldo ARROIO. Education as a human right against hate speech and intolerance
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
316 References
United Nations – Sustainable Development Goals – 2030 Agenda. https://www.un.org/

sustainabledevelopment/ retrieved on June 2019.
Agnaldo Arroio PhD, Professor, University of Sao Paulo, Brazil & Editor-in-Chief of Problems of
Education in the 21st Century.
E-mail: agnaldoarroio@yahoo.com
Website: https://www.researchgate.net/profile/Agnaldo_Arroio

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
CHARACTERISTICS OF INDEPENDENT 317
SCHOOLS DIRECTED AT STUDENTS IN

NEED OF SPECIAL SUPPORT: A STUDY OF
SCHOOL WEBSITE PRESENTATION
Lotta Anderson
Malmö University, Sweden
E-mail: lotta.anderson@mau.se
Gunvie Möllås
Jönköping University, Sweden
E-mail: gunvie.mollas@ju.se
Lisbeth Ohlsson
Kristianstad University, Sweden
E-mail: lisbeth.ohlsson@hkr.se
Abstract
The aim of the research was to explore how 55 Swedish independent schools, directed at (or limited to)
students in need of special support (SNSS), describe their organisation, work and visions. The empirical
data of the research consisted of the schools’ website presentations, which were processed and analysed
in consecutive steps. The results showed that the students’ complicated school- and life situations
were often combined with disabilities mainly in the neuropsychiatric field. The majority of the schools
(76%) practiced both schooling and methods for treatment and care, differentiating their role from the
mainstream track. Neuropsychiatric and psychological perspectives had a significant influence, reflected
in how the schools describe their daily routines, therapeutic methods of treatment and access to specific
categories of staff. Small groups, individual instruction and competent staff were described as specific
features. Teaching content and didactic aspects were seldom highlighted. The focus on the websites was
on socialisation and subjectification while qualification, i.e. knowledge development, had a more limited
role. The study points to a need for further research exploring daily pedagogical practice in more depth
and calls for a greater focus on student perspectives. Consequences for learning contexts are discussed in
the concluding part of the article. The specialist role, the independent schools in the present study tended
to take on are most urgent issues to discuss in an educational context striving for equity and inclusive
learning environments.
Keywords: inclusive education, independent schools, students in need of special support, treatment
methods, website presentation.
Introduction
For some decades, independent Swedish schools have been able to conduct schooling
directed at students in need of special support (henceforth SNSS) although, in many cases
it is also possible for such schools to admit other students. The Education Act of 2010 gave
independent schools the right to limit their activities to SNSS, which at present also applies
to upper secondary schools (SFS, 2010: 800, chapter 10, 35§, chapter 15, 33§). All Swedish
schools are expected to achieve educational goals. However, statistics on study results show

Lotta ANDERSON, Gunvie MÖLLÅS, Lisbeth OHLSSON. Characteristics of independent schools directed at students in need of
special support: A study of school website presentation
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
318 that goal achievement in independent schools with limited student admission is considerably
lower than mainstream comprehensive schools as a whole. According to the Education Act,
students should be given special support within the teaching group that “the student normally
belongs to” (SFS, 2010: 800, chapter 3, 7§, 11§) as long as there are no other special reasons
for not doing so.
Questions about differentiation have been, and still are, central in discussions about all-
encompassing or separated schools both nationally and internationally (Persson & Persson,
2016). Opening schools to a diversity of students challenges regular/mainstream schools as well
as alternative educational contexts (Andersson, 2017; Ifous, 2015; Kotte, 2017). A key question
is how learning difficulties might be understood (Ainscow, 1998; Haug, 1998; Nilholm, 2012)
and how teaching in independent and public schools could be organised so that students see
themselves as participants in engaged learning and agents in their own lives (Ahlberg, 2015).
By analysing the websites of independent schools directing their schooling to SNSS, the
present study intends to contribute knowledge about how these schools define their target group,
present goals and visions, and describe their own organisation, content and implementation of
activities. Websites are the outward face of schools, where they market their activities through
the use of arguments vis-a-vis target group, profile, visions, and competence.
It is relevant to study the content of websites in relation to research on SNSS and education
(Dovemark, 2017; Rix, 2011). This is especially interesting if the independent schools targeted
at SNSS argue that mainstream schools fail to provide optimum education considering their
conditions and needs (Swedish National Agency for Education, 2014).
The Emergence of Independent Schools in Sweden
The administrative authority for the Swedish educational domain changed in the
early 1990s and today a diversity of private and public actors compete in the school market
(Magnússon, Göransson, & Nilholm, 2014). Independent schools make up an increasingly
large element within the compulsory school system, not least for the voluntary upper secondary
school market (Dovemark, 2017; Dovemark & Holm, 2017; Giota & Emanuelsson, 2011).
Among Sweden’s elementary schools, independent schools represent 17% (827/4839) and for
the upper secondary schools in the country, the equivalent figure is 33% (428/1313). Of the total
number of students in elementary schools, 15% (158 000) attend independent schools and the
equivalent proportion in secondary school is 26%, i.e. almost 88 000 students (Association of
Independent Schools, 2018).
In a survey of 686 independent schools in Sweden (response rate 79.8%, n=546)
(Göransson, Magnússon, & Nilholm, 2012) 55 schools defined themselves as specifically
directed to SNSS. For the group in question, SNSS made up 90–100% of the student base. Later,
a survey by the Swedish National Agency for Education (2014) showed that 61 elementary
schools and 7 upper secondary schools limited their admission to SNSS during the school year
2013/2014. Another 20 elementary and 6 upper secondary schools directed themselves to SNSS
although places could be offered to other groups. The increasing number is in line with the
general emergence of independent school units.
Independent schools are mainly established within densely populated areas (Magnússon
et al., 2014; West, 2014) and schools directed to SNSS are no exception. Almost half of their
students are residents in Stockholm municipality. Further, the majority of the students attend the
later years of elementary or upper secondary school. The failure of public schools to respond to
the needs of SNSS, combined with the increase of students with neuropsychiatric disabilities1
are two main causes for the growing number of independent schools according to the Swedish
National Agency for Education (2014).

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
Official statistics in Sweden (Swedish National Agency for Education, 2018b) show that 319
teacher ratio varies among administrative authorities and school units. Public schools have a
slightly higher teacher ratio (12.1 students/teacher) compared to independent schools (13.3
students/teacher). The proportion of teachers in independent schools with an academic degree
varies between 23 and 100% depending on the school, which can be compared to the average
proportion in public (66%) and independent (56%) schools (Swedish National Agency for
Education, 2018b).
Making comparisons between countries is impeded by the existing multitude of terms
and different systems. For example, Denmark’s ‘efterskole’ [after-school] is for young people
who failed in a mainstream school, and in Norway the establishment of (private) independent
schools has occurred as a result of legal changes during the last decade (Lovdata, 2016).
Finland and Iceland have limited private independent school systems, whereas two thirds of
the total student population in the Netherlands are found in private confessional schools (The
Structure of the European Education Systems, 2014/15). Independent schools in Australia
are steadily increasing through private organisations whose focus is on compensating for low
socioeconomic conditions (Zehavi, 2012). The Scottish private school system has old ancestries
(Swedish National Agency for Education, 2007) and in England independent schools may get
state support for admitting students with different diagnoses, such as dyslexia, and especially
gifted children (Rix, 2011).
Independent School Support for Students in Difficulties
In independent ‘charter schools’ in the US2, the extent and quality of support for SNSS
varies between schools and states (Bailey Estes, 2009). In spite of federal support for their
establishment, schools have failed to present innovative thinking for improved teaching and
didactic development.
In a survey of school leaders and their perceptions of how their schools address SNSS,
Taylor (2005) considered demographics, descriptions of the school assignments, the admission
of students, and strengths and weaknesses in the schools encounter with students’ varying
needs. The assignments were described by school leaders in terms of spiritual values, academic
excellence, catering for individual needs, and diversity. Taylor emphasised the central role
of leadership, confirmed also in Gous’s, Eloff’s and Moen’s (2014) qualitative study. School
leaders describe strengths such as limited group size, a high teacher ratio, flexible and devoted
teachers, and an inclusive philosophy. Obstacles mentioned were the absence of appropriate
education among staff, a syllabus that is too rigid, time factors, a lack of suitable localities
and equipment, and the costs associated with SNSS schools. Taylor (2005) emphasised the
need for systematic studies of teaching practice in independent schools in the US, focusing on
school leaders’ knowledge about special educational efforts as well as how the mapping of such
support is performed, and the quality of that mapping.
Using discourse analysis in a study of 78 websites from independent ‘specialist schools’
in England, Rix (2011) raised the question how the principle of the Salamanca declaration
(UNESCO, 1994) on inclusive education receiving all students in mainstream is possible to
apply in segregated school settings. West (2014) described a lack of comparative studies related
to policy issues in independent schools. Comparing Sweden and England she raised questions
about whether the privatisation of schools contributes to less segregation and increased
social unity, suggesting that it would be a mistake to presume that there is one single and
simple explanation for this. Similarly, Alexandersson (2011) expressed worries concerning
segregation of students due to reforms for establishing independent schools. Discussing the
increase in independent upper secondary schools in Sweden, Dovemark (2017) in her study
of 64 independent upper secondary schools, put forward that schools have become more

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
320 segregated. She noted that inequality has increased in spite of the requirement for curricula to
further knowledge development and democratisation. Competition means that differentiation
and brands are vital for independent schools to survive. Dovemark concluded that focusing on
diversity means positioning of students as different to suit the profile of the school in a discourse
of marketization rather than a discourse of democracy and citizenship. Values inscribed in the
Swedish curriculum to promote cooperation and justice were seldom evident in Dovemark’s
empirical data.
Malmqvist (2018) described municipalities that hand over the responsibility for
certain students to independent schools presenting themselves as specialised in, for example,
neuropsychiatric disabilities. This can be compared with the segregated contexts found in
public schools. Malmqvist studied ADHD classes and concluded that these contexts were
often strongly informed by neuropsychiatric and behavioural theories. Diagnoses, medication,
therapeutic methods of treatment, and reward systems had a great scope. The consequences of
handling school situations in segregated contexts led to the question: “In what way will ADHD
students have schooling that is based on their needs and that is beneficial for their future?”
(p.18).
Inclusive Education
Thomas (2013) claimed that inclusive learning environments are made up by school
practices that characterise the whole student population and its heterogeneity, focusing the
question on how separation relates to learning, fellowship, identity, and belonging. This, in turn
is associated with organisational prerequisites such as group size and the physical context of
the school.
Discourse on inclusive education often results in parents/guardians believing that their
child will no longer be secured the special teaching needed (Malmqvist, 2018; Rix, 2011).
Inclusion is thus associated with SNSS instead of a teaching and learning approach encompassing
all students (Ifous, 2015). An unambiguous definition of the concept is missing (Malmqvist,
2016; Nilholm & Göransson, 2013; Persson & Persson, 2016), but to discuss its meaning, the
concept can be contrasted with its opposite, i.e. with exclusion. Further, it is not merely about
who is doing the including or excluding, but also about the one who is experiencing oneself as
being included or excluded (Malmqvist, 2016). In-depth studies from a student perspective are
therefore needed.
Magnússon, Göransson and Lindqvist (2019) argued that interpretation of the phenomenon
of inclusive education should be placed at a general educational policy level where the measure
defining an inclusive school is dependent on political discourses and allocation of resources
on both national and local levels. Creating a common vision and ethos that acknowledges,
respects, and supports diversity of all people and values their perspective, calls for a discussion
on normative issues (Alexiadou, 2013). This depends on how school staff translate concepts
inherent in the school’s value base into action (Ahlberg, Andreasson, Assarson, & Ohlsson,
2011; Andreasson, Ohlsson, & Assarson, 2015) as much as there is a need for a broad, public
discussion engaging diverse groups in society (Alexiadou, 2013). Rix (2011) emphasised the
need for a theoretical analysis of which type of school has the capacity to create an inclusive
environment and he suggested new concepts and discourses to enable next steps towards
education for all.
Words associated with inclusion were given a prominent place in the majority of the
websites in Rix’s study. However, they took their point of departure from a deficit perspective on
the students’ innate abilities. The researcher concluded that those offering segregated education
can identify themselves as specialists with individually adjusted education, and legitimately
emphasise values such as inclusion and a mainstream agenda. These discourses need not mean

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
that the school’s practice has changed in any fundamental way. In their striving for inclusion, 321
many educators made substantial changes while others reinterpreted the inclusion concept to fit
their already established activity (Rix, 2011).
The specific arenas offered for learning and participation, and how the identity of
students is created, are dependent on the institutional practices they are part of and how these
practices handle the challenges that come with individual differences and diversity (Säljö,
2012). Transformations of identity shaping thus depend on the learning context in which
students find themselves and the kind of interaction made possible (Hjörne, van der Aalsvort &
de Abreu, 2012; Van der Aalsvort & Hjörne, 2012). When students are positioned as different
subjects, differentiation and dissimilarity become a prerequisite and a criterion for quality to
attract students in a competitive school market (Dovemark, 2017). However, this diversity
is interpreted to mean that different students fit in different schools specialised for that kind
of subject position. This could, according to some school leaders interviewed in Dovemark’s
study, lead to a strong homogenisation of student groups in schools and negative consequences
for fostering democratic values.
Conditions and Prerequisites for Learning and Development
Schooling directed to students in need of support is characterised by complexity. To

describe and interpret the independent schools’ websites and presentations of how they create
conditions and prerequisites for students’ learning and development we found it necessary to
combine some theoretical concepts and perspectives.
The underlying causes of a student’s school difficulties may be explained in different
ways. Thus, the actions taken are dependent on which perspective is dominant. Researching
discourses of inclusion and exclusion, Skidmore (2004) described a discourse of deviance and
a discourse of inclusion. Accordingly, Nilholm (2012) used the concepts of categorical and
relational, and a dilemma perspective to describe SNSS and the special educational context.
Von Wright (2000) chose ‘punctual’ and ‘relational’ to describe the view on students in
need of support and the school’s approach to and way of encountering these students. The
‘Communicative Relational Perspective’ (CoRP) is an example of a relational perspective
(Ahlberg, 2015; Möllås, 2009; Nordevall, 2011). Research taking CoRP as a starting point
studies relations between e.g., different levels, the societal-, school-, group- and individual level.
The levels interact and need to be mapped and analysed in order to optimise the prerequisites
for learning and teaching in relation to both students and staff. The way schooling is governed
and organised; and, how education is planned and carried through creates different conditions
and possibilities for students’ learning and development.
Relations exist in, and through, the practices that students and staff commonly share
(Bingham & Sidorkin, 2010), in this case independent schools. Both national (Aspelin, 2012;
Hugo, 2011; Möllås, 2009) and international studies (Biesta, 2010b; Raczynski & Horne, 2015)
have emphasised the importance of well-functioning interactions between the actors in the
school. The social environment (The National Agency for Special Needs Education and Schools,
2012; Tufvesson, 2007), i.e. shaping of relations, communication and cooperation, thus plays a
crucial role in student learning and participation (Ahlberg, 2015; Aspelin, 2012). Cooperation
may be looked at as an interplay between different organisations and their governing systems,
perspectives, and organisation (Danermark, 2004). Treatment, care, and concern characterised
the schools in the present research to different extents. This means that values and traditions
from such professions influenced the activities along with the legal documents, the Social
Services Act (SFS 2001:453) and the Health Care Act (SFS 1982:763).
Tufvesson (2007) put forward the ‘accessibility model’, which states that environmental
factors influence students in different ways when it comes to their ability to concentrate in a school

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
322 context. The external properties of the school building and physical qualities such as passages
between rooms, acoustics, furnishings, seating, group size and the size of the classroom affect
students’ concentration in a positive as well as a negative way. From this follows a necessity for
knowledge about the physical surroundings to make them accessible to all students and ensure
that any adjustments needed can be undertaken. The second part of the accessibility model is
as mentioned the social environment and the third the pedagogical context. Didactic strategies
such as organisation, content, and implementation of teaching create, together with the social
and physical surroundings, the conditions and prerequisites for optimal learning (The National
Agency for Special Needs Education and Schools, 2012; Tufvesson, 2007).
Alongside concepts such as inclusive education, CoRP and the accessibility model, three
main functions described by Biesta (2010a) as characteristics for teaching have been used as
theoretical tools in the present study to analyse the independent schools’ websites. Qualification
refers to a school’s knowledge assignments while socialisation is about integrating a person
into an existing social, cultural, and political context by transferring norms and values. The
third function, subjectification concerns the process of individualisation, i.e. becoming an
independent and responsible subject.
Problem of Research
The Swedish school legislation states that all students have the right to equal education
irrespective of the school they attend, where they live in the country or their different socio-
economic conditions. The Education Act (SFS (2010:800) is to be complied with by public
schools as well as the private ones. All students have the right to get guidance and encouragement
and if needed, extra adaptations and special support must be offered to the students within the
ordinary teaching group. Only in exceptional circumstances can separate teaching groups be
authorised. Despite this, several segregated solutions are found in Swedish schools (Malmqvist,
2018). Independent schools are of specific interest, due to their permission to conduct schooling
directed at or limited to SNSS (SFS, 2010: 800, chapter 10, 35§, chapter 15, 33§). These
schools seem to be an alternative when public schools fail to create prerequisites and conditions
for all students’ learning and development. Considering the infrequency of research focusing
independent schools directed at SNSS it is important to generate knowledge within this field.
The aim of the research was thus to explore how 55 independent elementary schools
present their schooling on their respective websites. The schools have themselves stated and
defined their activities as directed towards SNSS but most of them are also open to other
students.
Research questions were:
1 How is the target group defined and described?
2 How are the educational activities of the school described concerning organisation,
content and implementation?
3 How are goals and visions formulated?
Research Methodology
General Background
A qualitative approach characterized the design of the research based on a content

analysis of 50 independent schools’ websites. The Internet was expected to be a valuable source,
since schools’ home pages might be an effective way to market the visions and activities of the
schools. All printed pages resulted in extensive document analyses. This form of qualitative
research, often combined with other qualitative methods, is distinctive in so far as the web sites
exist before the researchers select them for further scrutiny and interpretation.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
Sample 323
The sample for the present research was derived from a previous national survey of
686 independent schools in Sweden (response rate 79.8%, n=546) (Göransson et al., 2012).
Responses indicated 55 independent schools targeted at SNSS. This group constituted the
sample of the present research. Following an introductory examination of responses from the
schools in question, a search for their websites was performed. Thus, the empirical data of the
study consisted of the schools’ presentations of their activities on websites, in total 1850 printed
pages, which were analysed in consecutive steps (Kvale & Brinkmann, 2009) described further
in the data analysis section.
Missing Data
The search for websites showed that one school had shut down since responding to the
questionnaire and for another the limited company had gone bankrupt. For three schools no
information could be found on the Internet and contacting the schools by telephone gave no
result either. The mapping and analysis of printed pages thus included 50 schools. A follow up
of the schools’ websites in 2018 showed that another seven schools had closed after the present
analysis was performed, a point that is further described in the results section.
Data Analysis

The data analysis was performed in several steps (Figure 1) (Kvale & Brinkmann,
2011). Initially the schools were sorted according to municipal area (SALAR, 2015, 2017).
Further data were processed and compiled then according to school form, ages of target group,
number of student places, teaching, treatment, owner of the school, and year of establishment.
Starting with the descriptions of the character of their settings, the schools were divided into
seven groups (Figure 2). Hereafter data were analysed according to each research question.
Descriptions of the target groups were divided into 13 categories, e.g. students needing
change of school, those with neuropsychiatric disabilities, difficulties with relationships, and
children of parents with addiction problems. The categories were then interpreted according
to the concepts of the ICF-CY: body, activity, and physical and social context factors (WHO,
2007), resulting in four groups (Figure 1). Each school’s description of their target group could
then be coded using these groups as follows: A, B, C, D, AB, BD and ABD (Figure 1).
Starting from the second research question the websites were read repeatedly by the
researchers and processed explanatorily while notes were made as mind maps and reflective
memos (Kvale & Brinkmann, 2011). Following the sorting of text content, three researchers
independently categorised the material in several analytic steps finally resulting in five themes
(Figure 1). Texts on the websites about the schools’ goals and visions were scrutinised in a
similar way.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
324
Figure 1. Data analyses.
Reliability and Ethical Aspects
Intersubjective analyses of data were used to strengthen the reliability of the study.
Extensive parts of the data were independently analysed by each researcher. Thereafter the
units of analysis were compared and further developed. Similarly, the text production was
characterised by common discussions and scrutinising. Access to the analysis of the survey
questionnaires was helpful in making comparisons and reconciliations with the intention to
strengthen reliability.
It should be noted that the current research of websites does not claim to describe how
daily work is organised and implemented in independent schools targeting SNSS or how well
the information on the websites corresponds to everyday practice. A follow up of the websites
was performed in 2018 to look for possible changes in the schools’ presentations. Where
websites were updated there is reason to assume that there might be differences in presentations
depending on how and by whom the updating was done. Websites are one way to market schools
and accordingly it is of interest to study their content.
Since schools in the present study often integrated treatment of different kinds, it
was sometimes difficult to discern clear information about the schooling in the independent
compulsory schools. The content as a whole has nevertheless contributed to viewing the schools
from a wider perspective. Principles of research ethics were consistently followed in the study
(Vetenskapsrådet, 2002, 2017). The data was publicly available, intended to inform the public
and especially interested parties. Irrespective of this, use of school names has been avoided in
the article.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
Research Results 325
Target Group for the Schools
According to the classification of Swedish municipalities produced by the Swedish

Association of Local Authorities and Regions (SALAR, 2017) most of the independent schools
in the study (74%) belonged to municipalities identified as large or medium-sized cities and
municipalities or commuting municipalities near them. The rest of the schools (26%) were
located within municipalities with a low commuting rate near medium-sized towns, smaller
towns/urban areas or in rural municipalities.
Six of the 50 studied schools required a diagnosis to admit students, i.e. neuropsychiatric
disabilities and/or dyslexia, while two schools clearly expressed that no diagnosis was needed.
In a third of the web texts, the concept ‘diagnosis’ was absent when describing the students in
the school. Instead words such as ‘difficulties’ or ‘problems’ appeared. Four schools exclusively
received boys and two only took girls. A few settings distinctly expressed that they did not admit
students who have addiction problems or a criminal record. About a quarter of the schools were
open to students with intellectual disabilities.
Symptoms of a complicated school or life situation were often the reason for choosing
an independent school directed towards SNSS3. In almost half of the settings (48%) there were
different combinations of groups A-D (Figure 1), indicating a multifaceted picture of problems.
A complicated school situation together with a disability (A+B) was most often described as
a reason for choosing the independent setting (ca. 78% of the schools). Seven of fifty schools
referred to earlier failure in mainstream schools where, for example, student health and social
services had been involved (group D). Six schools described their target group by referring to
the need for input and actions such as specially adjusted teaching aids and ways of working,
especially skills training or adjusted physical surroundings (group C).
Independent schools restricting admission often addressed students with neuropsychiatric
disabilities, thereby making the specific disability the starting point. Consequences of the
disability (physical and social environment factors) were presumably a cause of the student’s
problems in school- and life situations, which in turn led to schooling in an independent school
setting.
School Organisation, Content and Implementation
Organisation of Settings
Descriptions of the nature of the school settings varied. Categorisation was made from the
emphasis on schooling and care/treatment respectively. Of the 50 schools, 12 (24%) exclusively
offered schooling, in some cases combined with leisure time activity or short-term residential
housing. The remaining 38 schools (76%) were linked to some kind of treatment program.
Figure 2 shows the variation in time for students in the setting, ranging from solely school
hours to full time living. School settings, linked to institutions with residential care homes with
treatment according to the Social Act, are found within groups 5 and 6, which together make up
approximately half of the schools.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
326 1. Schooling only (9 schools)

8 2. Schooling with free-time activity/short term accommodation
7 1
(3 schools)
3. Schooling with treatment programs e.g. “therapeutic
school”
2 (7 schools)
6
4. Treatment program including school, daytime (6 schools)
5. Care and treatment program including school, weekly
accommodation (13 schools)
3 6. Care and treatment program including school, fulltime
accommodation (11 schools)
7. Family Therapy home (1 school)
8. Missing data (5 schools)
4
5
Figure 2. Characteristics of the independent schools in terms of organisation of

schooling, care and treatment.
Access to Staff and Competencies
There was a wide range of professional categories in the independent schools in the study.
Depending on their focus, a clear dividing line was seen between the settings. In about half of the
schools aimed at regular schooling (groups 1–2, Figure 2), the principal was often engaged as a
teacher. Besides class- and subject teachers, three schools stated that they had employed special
educators and/or special needs teachers, and in one case a ‘dyslexia pedagogue’. Whether the
staff members were university trained cannot be inferred from the texts and certain descriptions
such as ‘study pedagogue’ were not defined. A few schools emphasised that staff had “long time
experience of children in need of special support”. Access to educational tutorials or specific
education was described in a few cases.
In the schools included within groups 3–4 and 5–7, teachers made up a minor share of
the employees. Except for headmasters, the leadership was extended with, for example, heads
of activity, treatment and the boarding school. For groups 3–4, therapists and family processors
were important units. Approximately half of the schools in these groups mentioned access to
teachers. Except for leisure time leaders/coaches for young people, and student and teacher
assistants, there were a range of undefined roles such as healing pedagogues and coordinators.
Medical staff (e.g. psychiatrists) were frequently found in consulting roles at these schools.
In groups 5–7, a quarter of the schools did not describe their staff at all. Special needs
teachers were mentioned in a few cases, but special educators45 were employed on a part- or full-
time basis in a quarter of these schools. A couple of schools had access to special educational
competence ‘when necessary’. There was no information about other teachers. In cases where
treatment staff was defined, these included social workers and cognitive behavioural therapists.
Instructors and consultants featured in a range of areas, e.g. child and youth psychiatry,
neuropsychiatry, cognitive behaviour therapy, conversation therapy, dance and body therapy,
addiction therapy, musical counselling, and ART (Aggression Replacement Training).
All schools emphasised the specific competence and qualities of the staff as well as
access to ‘one-to-one teaching’, extra teacher resources and student assistants.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
Student Number and Ages 327
The limited number of students, small groups, and individual support were emphasised as
major advantages by the schools. A third of the schools did not mention the number of students
that their setting was planned for. There were 22 schools (42%) able to accept a maximum
number of 20 students. Eight schools (16%) could accept slightly more, i.e. 21–30 students,
while a few, e.g. a school specifically targeted at students with dyslexia, had room for more
than 100 students. The range of ages (pre-school to year 9) differed and four schools received
only pre-school class to year 6. The rest of the independent schools covered pre-school class
to year 9.
Creating Relations and Cooperation
A continuous interplay between adults and students in lessons, breaks, and leisure time
contributed to social accessibility, as reflected in the websites. Further, a trustful cooperation
with guardians/parents was considered a necessary precondition for students’ positive
development. In settings with treatment programs, parents most often took part as a resource
in processes for sustainable change, expressed in some cases as ‘hook-arms-with-parents
methodology’. Developmental dialogues, tailored action plans, and evaluations were important
tools strengthening cooperation. Cooperation with other actors (municipal and state) was not
mentioned on the websites.
The websites presented concepts related to a holistic view of students as they were
offered education and treatment. Each person’s abilities and needs were taken as a starting
point. A common basic view of students and values among the staff warranted a positive view
of students, which in turn promoted schoolwork. The schools considered that acknowledgment
of students’ interests and motivation were important ingredients. Ensuring that interactions are
engaging was essential for students to experience trust and participate fully in activities.
Ways of Working and Specific Methods
The main point in descriptions of teaching practice, i.e. the educational access, was the
organisation of activities inside and outside classrooms. Small groups, high staff ratio, individual
instruction and a variety of working methods were repeated arguments for a satisfactory
education. Fixed daily routines, timely well-adjusted working sessions, individually adjusted
schedules, structure, and predictability were put forward as essential in the majority of schools.
Physical accessibility was exemplified by stable workplaces, one’s own desk with storage space
and workbaskets.
Access to rooms and different contexts where students can be active was a recurring trait
in the presentations. Elements of bodily movement and aesthetics were present in the regular
daily schedule. Many schools arranged special outdoor activities, overnight excursions, cultural
activities (e.g. study visits, theatre and music events) and other events that aimed to strengthen
social relations and exposed students to meaningful leisure activities.
Schools with treatment programs mentioned a rich variety of methods, aiming to change
behaviour and give students tools to handle relations, feelings, conflicts, and norms. Pedagogical
elements, such as communicative exercises and learning in interaction with others, were also
described. The importance of practical ingredients interwoven with theoretical approaches was
emphasised along with obtaining knowledge about everyday activities such as mealtimes. There
was seldom any account of specific educational methods, strategies or didactic aspects, except
in general terms, e.g. ‘tailored pedagogy, thoughtful pedagogical principles, and clarifying
instruction’ or ‘teaching Swedish is more laboratory [concrete] than in other schools’.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
328 Governing and Guiding Documents
There were guidelines governing schools that ran settings with treatment activities,
in addition to the application of regulatory documents applied in mainstream schools. The
curriculum for compulsory school (Swedish National Agency for Education, 2018a) was the
regulatory framework most often mentioned, followed by different individual action plans
as well as other documents governing treatment activities such as the Social (SFS 2001:453)
and Health Care Acts (SFS 1982:763), and activity and equality plans. The UN Convention
on Rights for persons with disabilities (United Nations - Disability, 2006) and the Salamanca
Declaration (UNESCO, 1994) were more rarely considered. The Education Act (SFS 2010:800)
was not referred to a great extent as a foundation for the settings.
Formulating Goals and Visions
Of 50 independent schools, 41 have formulated goals and visions using varying concepts,
e.g. main goal, operating principle, aim and main objective. The same school could have several
different wordings. Goal descriptions were usually related to the actions suggested or already
implemented to reach a certain goal. One school spoke to the students themselves: “The goal for
the school is that you, after 9 years are ready to succeed with your studies in upper secondary
school, academically as well as socially.”
In the presentations of goals and visions, different themes emerged: knowledge (themes 1
and 2) on the one hand, and social and personal development (themes 3 and 4) on the other, in
both cases from a student perspective. These themes were characterised by both short and long
term perspectives. Moreover, an activity perspective appeared (theme 5) with emphasis on what
the school ought to achieve or how the activity should develop. The same school sometimes
expressed several themes in its goal description, exemplified in Table 1.
Table 1. Descriptions of goals and visions.
Thematization of descriptions of goals and con-

tent. Number of schools where these are present Examples of statements from the school websites
in one or more statements
The students should achieve the best study results as

Knowledge development (short term)
possible; passing at least in all subjects and entering into a
14 schools/19 statements
national program in upper secondary school.
Knowledge development (long term) To be able to apply to higher education; desire for further
12 schools/15 statements learning; having mastered effective tools for further learning.
Being active and able to manage as much as possible on
one’s own; every student should be strengthened in her/his
Social and personal development (short term) 27
own security and trust in oneself; strengthen and increase
schools/51 statements
students’ feeling of meaningfulness, manageability and
comprehensibility.
Ability to form a meaningful life; promote development to be-
Social and personal development (long term) 20
come a responsible person and citizen; to have a home and
schools/29 statements
live an independent life or be able to manage employment.
Allocate schools with a common denominator; activities with
Organisational development 20 schools/40 state- individually adjusted teaching; closeness, cooperation, and
ments integration with regular/mainstream settings; schools with the
child in the centre.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
The social and personal development theme dominated both short- and long-term 329
perspectives (80 statements in themes 3–4). This was considered relevant given that the majority
of schools ran a therapeutic activity with care and therapy as central elements. In comparison,
the knowledge theme stood out in less than half of the descriptions (34 statements in themes
1–2). The expressed visions were usually found in the activity theme while more concrete
formulated goal statements emanated from a student perspective.
Theme 5 contained content from the websites from an organisational perspective. It
considered the profile of the school as well as perspectives representing foundational values
such as equity, participation and equal rights. Focus was on safety and trust in oneself as a
student as well as the school´s respect for staff, students and parents/guardians. The descriptions
of aims and visions also directed attention to a proactive mission, i.e. preparing students for a
return to their municipal school. The meaning of the texts on the websites mainly focused on the
individual student, staff member, and parent/guardian. How school difficulties relate to group-,
school- and society level was seldom described.
A Follow up Research of Websites from Independent Schools
A follow up of the websites in summer 2018 showed that seven of the schools had
shut down since the Swedish Schools Inspectorate called in their authorisation. One of the
schools missing a website in the first analysis now had a presentation of its activity on the
web. The analysis also showed that two independent schools had changed their names. Another
tendency was for the change of owner or fusion into a bigger corporate group. According to the
Association of Independent Schools (2018), one single actor could have 187 school settings but
the most common is one unit, which is true for 587 actors. The number of independent schools
directed at SNSS is not clear.
Discussion
The aim of the present research was to explore how 55 independent schools present
their activities on their respective websites. The results showed that the students’ complicated
school- and life situations were often combined with disabilities mainly in the neuropsychiatric
field. Schools restricting their acceptance mostly directed themselves to students presenting
problems within the neuropsychiatric field and the majority of these schools ran both schooling
and treatment programs.
A continuous interplay between staff and students during school hours, and trustful
cooperation was considered a prerequisite for students’ positive development and for physical,
social, and pedagogic accessibility. The small student groups, individual instruction, high staff
ratio, staff competence and varied ways of working were emphasised as specific characteristics
of the schools. It was less common for learning content and didactic aspects to be mentioned
on the websites.
When analysing presentations of goals and visions different themes emerged, with social
and personal development dominating, followed by knowledge and operational development.
One conclusion is that the texts concentrated on the individual level without any obvious focus
on how school difficulties relate to group-, school- and society level.
Segregating Solutions
Inclusion and exclusion processes will always occur to different degrees in different school
contexts (Malmqvist, 2016). They happen in both public and independent schools (Malmqvist,
2018). However, an increase in segregating solutions for students whose needs cannot be met

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
330 in the mainstream does not harmonise with the intentions of the Education Act (SFS 2010: 800,
kap.3) although such groups are not forbidden in law. Some of the independent schools point
out their successes and opportunities in supporting the development of students by referring to
the failure of students in mainstream/public schools. From that point of view, the emergence of
independent schools directing themselves to SNSS (Swedish National Agency for Education,
2018b; Stockholms stad, 2014) could be regarded as a consequence of the inability of municipal
schools to meet students’ varied needs (cf. Rix, 2011). Malmqvist’s study (2018) provides
examples of municipal schools handing over the responsibility for certain students by sending
them to an independent school, specialised in, for example, neuropsychiatric disabilities.
The results showed that schools restricting their admission often addressed students with
neuropsychiatric problems. They described their students largely as students with a disability.
The organisation of activities could be compared to a permanent segregated group, something
that is against the goals for inclusion and limits students’ possibilities for a diverse togetherness
and participation (cf. Ahlberg, 2015; Möllås, 2009; Nordevall, 2011). Thus, it is not primarily
a relational view where contextual factors on different levels are mapped and acted on. Instead,
disability is the focus, i.e. a discourse of deviance with great impact (Skidmore, 2004). From a
punctual perspective, focus is on what and not who the student is (von Wright, 2000).
The majority of the schools (76%) are linked to some kind of treatment activity and
its regulations. The neuropsychiatric paradigm largely distinguished these schools, giving
them an expert role aside from the mainstream track. Specific competencies in the medical and
psychological field were highlighted in the presentations such as ‘access to psychotherapist’ or
‘specialist in psychiatry’ (cf. Malmqvist, 2018; Rix, 2011). Methods described by the schools
were characterised by care and treatment inputs, which aimed to change student behaviour. The
visions and goals primarily focused on socialisation and subjectification (Biesta, 2009), i.e.
social and personal development. Qualification as learning and knowledge development got a
lesser role mirrored in the schools’ staffing.
Regardless of the organiser, concerns have grown regarding the consequences segregated
school contexts may bring (Alexandersson, 2011; Bailey Estes, 2009; Dovemark, 2017;
Dovemark & Holm, 2017; Giota & Emanuelsson, 2011; Göransson, Malmqvist, & Nilholm,
2012). There is a risk that schools conserve a categorical or punctual view (Nilholm, 2012;
von Wright, 2000). The student becomes an object of therapeutic inputs in order to return to
mainstream schooling (Rix, 2011). However, this return was not an explicit goal dominating
the descriptions on websites for the schools in the study. There are occasional examples from
earlier studies (Göransson, Magnússon, & Nilholm, 2012). The extent to which students return
to mainstream schools cannot be answered at this stage but requires further studies.
Follow Up and Evaluation
Malmqvist (2018) points to the need for pedagogical follow-up studies examining the
consequences for schooling influenced by neuropsychiatric and psychological research from a
long-term perspective. Correspondingly, we find it necessary to follow and evaluate the impact
on students admitted to independent schools directing or restricting their activities to SNSS.
West (2014) also requests more studies exploring and problematizing the contribution to social
coherence by independent schools.
Research is required to study the students’ genuine participation and their opportunities
to perceive themselves as committed and learning subjects (Ahlberg, 2015). What consequences
arise for a student’s identity development and future if schooling is first and foremost characterised
by difficulties, e.g. having dyslexia, concentration difficulties, Asperger’s syndrome or ADHD
(Dovemark, 2017; Dovemark & Holm, 2017; von Wright, 2000)? It is ultimately about how
the basic values of the curriculum and a democratic idea of variety is interpreted and realised

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
in everyday education, something that applies to all schooling (Dovemark, 2017; Andreasson, 331
Ohlsson, & Assarson, 2015).
A further question is whether the school choice underpinning the emergence of
independent schools (Prop., 1991/92:95), increases the actual possibilities of all students and
parents/guardians. Could it be that independent schools are the only remaining alternative when
mainstream municipal schools are not working for a student (Swedish National Agency for
Education, 2014)?
Regardless of principal organiser, all students have a right to develop and be successful
in their learning (Bailey Estes, 2009). However, goal achievement in independent schools
restricting admission of students does not show results in that direction (Swedish National
Agency for Education, 2014). In spite of the schools’ qualities and special investments, limited
goal achievement remains, reducing possibilities for future studies. There is probably a range
of underlying causes for this lack of achievement, but it suggests that the schools may in fact
not be specialists (Rix, 2011; Swedish National Agency for Education, 2014). Professionals
possess competence within their respective areas and carry with them conceptions of what is
possible and what is not (Anderson & Östlund, 2017). Against this background, the low ratio
of teachers and special educators in many schools must be considered when discussing goal
achievement.
Creating Relations and Organising Teaching
In the presentations from the independent schools, the importance of well-functioning

social relations was emphasised and cherished in several activities. There are good models for
mainstream schools to follow in terms of what is done to create good relations, togetherness and
successful working interactions (cf. Ahlberg, 2015; Aspelin, 2012; Biesta, 2010b; Hugo, 2011;
Raczynski & Horne, 2015). Cooperation with parents/guardians, a common basis of values,
taking care of students’ interests and motivation, and a positive approach are all emphasised
as strengths by schools in the present study. The question is whether this can be viewed as
something unique for independent schools directed at SNSS. Rather, it should be a common
sign of all schooling regardless of principal organiser. Some parts are governed by Swedish
law and are thus a requirement to be fulfilled by all schools. Academic and social needs
should be attended to (cf. Bailey Estes, 2009). The emphasis on creating relationships in the
studied schools raises questions about whether the Swedish educational focus on knowledge
requirements, grades and merits has led to social aims being neglected in mainstream schools,
which in turn has led to consequences for students’ learning.
The importance of relations is in line with the foundational idea of CoRP where
communication, participation, and learning are viewed as a triad (Ahlberg, 2015). The school
presentations did not highlight the importance of learning content and didactics for learning
itself as a social phenomenon (Säljö, 2012). This signals that knowledge development is placed
in the background in most of the schools in the present study. The strategies described mainly
dealt with how teaching should be organised (small groups, high staff ratio etc.) and concrete
measures often advocated by neuropsychiatric research, e.g. set daily routines, individual
schedules, and predictability. The rich variety of treatment methods confirms the influence of
neuropsychiatry and psychology (cf. Malmqvist, 2018).
Limitations and Further Research
The present study has clear limitations. The website text was formulated by the schools
themselves. We cannot determine whether the content is a true reflection of the reality of the
schools. That would require further studies conducting interviews with the school leaders

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
332 and other staff as well as observations of the teaching occurring in classrooms. This study
also cannot determine whether student participation is higher in the schools studied than in
others. As the schools market themselves to SNSS, they can gain from listening to the voices of
students and their experience of physical, social and pedagogical/educational accessibility. The
perspectives of parents/guardians also need to be studied to gain a holistic picture. There was
scant information available on the websites about whether an independent school serves as a
permanent solution for earlier complicated schooling or if active efforts are made for the return
of students to their regular school. This should be of keen interest in further research.
Conclusions
An underlying neuropsychiatric and psychological paradigm seems to exist emphasising

the expert role of the independent schools, often described as an alternative when the mainstream
schools have failed to meet students’ different needs. About three quarters of the studied schools
were linked to therapeutic activities and they presented a rich variety of methods, aiming to
change behaviour. In contrast, specific educational methods, strategies or didactic aspects
were seldom described. In line with this, the social and personal development of the students
dominated the goals.
The present research contributes knowledge about independent schools directing
themselves to SNSS and what they announce they can offer to this group of students. Nevertheless,
the findings call for further research in order to close the research gap. It is necessary to follow
and evaluate the impact on students admitted to independent schools directing or restricting
their activities to SNSS. Regardless of principal organiser, the consequences of segregated
solutions and the individual centred focus must be scrutinised and discussed. The responsibility
of both independent and municipal schools for SNSS, and for equity and increased inclusion is
therefore a high priority.
Notes
The present research is part of the research project Fristående skolors arbete med elever
i behov av särskilt stöd [Independent schools and their work with students in need of special
support], financed by The Swedish Research Council, Stockholm (project number 2008-4701).
The data used to support the findings of this study are available from the corresponding authors
upon request. The authors declare that there are no conflicts of interest regarding the publication
of this paper.
The authors thank Michelle Pascoe, PhD, from Edanz Group (www.edanzediting.com/
ac) for editing a draft of this manuscript.
References
Ahlberg, A. (2015). Specialpedagogik i ideologi, teori och praktik - att bygga broar [Special education in
ideology, theory, and practice – building bridges]. Stockholm: Liber.
Ainscow, M. (1998). Would it work in theory? Arguments for practitioner research and theorizing in the
special needs field. In C. Clark, A. Dyson, & A. Millward (Eds.), Theorising special education
(pp. 7–20). London: Routledge.
Alexandersson, M. (2011). Equivalence and choice in combination: The Swedish dilemma. Oxford
Review of Education, 37(2), 195–214. Retrieved from https://www.tandfonline.com/doi/abs/10.
1080/03054985.2011.559379.
Alexiadou, N. (2013). Privatising public education across Europe – Shifting boundaries and the politics
of (re)claiming schools Education Inquiry, 4(3), 413-422. https://dx.doi.org/10.3402/edui.
v4i3.22610.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
Anderson, L., & Östlund, D. (2017). Assessments for learning for students with Intellectual 333
Disability. Problems of Education in the 21st Century, 75(6), 508–524.
Andersson, H. (2017). Möten där vi blir sedda: en studie om elevers engagemang i skolan [Meetings
where we are seen. A study about engagement in school] (Doctoral thesis, Malmö University).
Malmö, Sweden. Retrieved from http://hdl.handle.net/2043/23423.
Andreasson, I., Ohlsson, L., & Assarson, I. (2015). Operationalizing equity: The complexities of equity
in practice. Education, Citizenship and Social Justice, 10(3), 266–277.
Aspelin, J. (2012). How do relationships influence student achievement? Understanding student
performance from a general, social psychological standpoint. International Studies in Sociology
of Education, 22(1), 41–56. https://doi.org/10.1080/09620214.2012.680327
Assarson, I., Ahlberg, A., Andreasson, I., & Ohlsson, L. (2011). Skolvardagens komplexitet: en studie
av värdegrundsarbetet i skolans praktik [School day complexity: A study of value-based work in
school practice]. Stockholm: Swedish National Agency for Education.
Association of Independent Schools (2018). Statistik [Statistics]. Retrieved 27/06 2018 from http://www.
friskola.se/faktaom/statistik.
Bailey Estes, M. (2009). Charter schools and students with disabilities. How far have we come? Remedial
and Special Education, 30(4), 216–224.
Biesta, G. (2009). Good education in an age of measurement. On the need to reconnect with the question
of purpose in education. Educational Assessment, Evaluation & Accountability, 21(1), 33–46.
https://doi.org/10.1007/s11092-008-9064-9
Biesta, G. (2010a). Good education in an age of measurement: Ethics, politics, democracy. Boulder,
Colorado: Paradigm Publishers.
Biesta, G. (2010b). “Mind the Gap!” Communication and the Educational Relation. In C. Wayne
Bingham, & A. M. Sidorkin (Eds.), No education without relation (pp. 11–22). New York: Peter
Lang Publishing.
Bingham, C. W., & Sidorkin, A. M. (2010). No Education without Relation. New York: Peter Lang.
Danermark, B. (2004). Samverkan en fråga om makt. [Cooperation – a question about power.] Örebro:
Läromedia.
Dovemark, M. (2017). Utbildning till salu - konkurrens, differentiering och varumärken. [Education
for sale – competition, differentiation, and brands]. Utbildning & Demokrati [Education and
Democracy], 26(1), 67–86.
Dovemark, M., & Holm, A.-S. (2017). Pedagogic identities for sale! Segregation and homogenization
in Swedish upper secondary school. British Journal of Sociology of Education, 38(4), 518–532.
Giota, J., & Emanuelsson, I. (2011). Specialpedagogiskt stöd, till vem och hur? Rektorers hantering av
policyfrågor kring stödet i kommunala och fristående skolor [Special educational support, for
whom and how? School leaders’ handling of policy questions on support in public/municipal and
independent schools] (RIPS: Rapporter från Institutionen för pedagogik och specialpedagogik,
nr 1) [Reports from the department of education and special education]. Göteborg: Göteborgs
universitet, Institutionen för specialpedagogik [Gothenburg University, Department of education
and special education].
Goes, J. G., Eloff, I., & Moen, M. C. (2014). How inclusive education is understood by principals in
independent schools. International Journal of Inclusive Education 18(5), 535–552. https://doi.or
g/10.1080/13603116.2013.802024
Göransson, K., Magnússon, G., & Nilholm, C. (2012). Challenging traditions? Pupils in need of special
support in Swedish independent schools. Nordic Studies in Education, 32, 262–280.
Göransson, K., Malmqvist, J., & Nilholm, C. (2012). Local school ideologies and inclusion: The case
of Swedish independent schools. European Journal of Special Needs Education 28 (1), 49–63.
https://doi.org/10.1080/08856257.2012.743730
Haug, P. (1998). Pedagogiskt dilemma: Specialundervisning [Pedagogical dilemma: Special education].
Stockholm: Swedish National Agency for Education.
Hjörne, E., van der Aalsvort, G., & de Abreu, G. (2012). Exploring Practices and the Construction of
Identities in School–An Introduction. In E. Hjörne, G. van der Aalsvort & G. de Abreu (Eds.),
Learning, social interaction and diversity exploring identities in school practices (pp 1-7).
Rotterdam: Sense Publishers.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
334 Hugo, M. (2011). Från motstånd till framgång: att motivera när ingen motivation finns [From resistance
to success: Motivating when there is no motivation]. Stockholm: Liber.
Ifous. (2015). Från idé till praxis: vägar till inkluderande lärmiljöer i tolv svenska kommuner [From
idea to praxis: Roads to inclusive learning environments in twelve Swedish municipalities].
(Forskarnas rapport [Reports from the researchers] 2015:2). Retrieved from http://www.ifous.se/
app/uploads/2013/02/201509-Ifous-2015-2-slutversion2f%C3%B6rwebb.pdf
Kotte, E. (2017). Inkluderande undervisning. Lärares uppfattningar om lektionsplanering och
lektionsarbete utifrån ett elevinkluderande perspektiv [Inclusive education and teachers’
perceptions of lesson planning and lesson work from a student inclusive perspective] (Doctoral
thesis, Malmö University, Malmö, Sweden). Retrieved from http://hdl.handle.net/2043/23228.
Kvale, S., & Brinkmann, S. (2009). Den kvalitativa forskningsintervjun [The qualitative research
interview]. Lund: Studentlitteratur.
Lovdata. (2016). Lov om frittståande skolar (friskolelova)[Law on independent schools]. Retrieved 02/03
2017 from https://lovdata.no/dokument/NL/lov/2003-07-04-84.
Magnússon, G., Göransson, K., & Lindqvist, G. (2019). Contextualizing inclusive education in
educational policy: the case of Sweden. Nordic Journal of Studies in Educational Policy. https://
doi.org/10.1080/20020317.2019.1586512.
Magnússon, G., Göransson, K., & Nilholm, C. (2014). Similar Situations? Special needs in different
groups of independent schools. Scandinavian Journal of Educational Research, 59(4), 377–394.
https://doi.org/10.1080/08856257.2012.743730.
Malmqvist, J. (2016). Working successfully towards inclusion – or excluding pupils? A comparative
retroductive study of three similar schools in their work with EBD. Emotional and Behavioural
Difficulties, 21(4), 344–360.
Malmqvist, J. (2018). Has schooling of ADHD students reached a crossroads? Emotional and Behavioural
Difficulties, 23(4), 389-409. https://doi.org/10.1080/13632752.2018.1462974.
Möllås, G. (2009). “Detta ideliga mötande”- En studie av hur kommunikation och samspel konstituerar
gymnasieelevers skolpraktik [“Always these meetings” – A study of communication and
interaction constituting the school practice of upper secondary students] (Doctoral thesis,
Jönköping University, Jönköping, Sweden). Retrieved from https://www.diva-portal.org/smash/
get/diva2:275770/FULLTEXT01.pdf.
Nilholm, C. (2012). Barn och elever i svårigheter – en pedagogisk utmaning [Children and students in
difficulties – a pedagogical challenge]. Lund: Studentlitteratur.
Nilholm, C., & Göransson, K. (2013). Inkluderande undervisning – vad kan man lära av forskningen?
[Inclusive education – what can you learn from research?]. Stockholm: Specialpedagogiska
skolmyndigheten [Special educational school authority].
Nordevall, E. (2011). Gymnasielärarens uppdrag som mentor: En etnografisk studie av relationens
betydelse för elevens lärande och delaktighet [The upper secondary school teacher’s assignment
as mentor: An ethnographic study of the significance of relations for pupils’ learning and
participation] (Doctoral thesis, Jönköping University, Jönköping, Sweden). Retrieved from
https://www.diva-portal.org/smash/get/diva2:414302/FULLTEXT01.pdf.
Persson, B., & Persson, E. (2016). Inkludering och socialt kapital. Skolan och ungdomars välbefinnande
[Inclusion and social capital. School and the welfare of young people]. Lund: Studentlitteratur.
Prop. 1991/92:95. Om valfrihet och fristående skolor [On liberty of choice and independent schools].
Retrieved from https://data.riksdagen.se/fil/5277FE6D-7C0B-47CB-AE25-7B2234DE5FBE.
Raczynski, K. A., & Horne, A. M. (2015). Communication and interpersonal skills in classroom
management. How to provide the educational experiences students need and deserve. In E. T.
Emmer & E. J. Sabornie (Eds.), Handbook of classroom management (pp. 387–408). New York:
Routledge.
Rix, J. (2011). Repositioning of special schools within a specialist personalised educational marketplace
– the need for a representative principle. International Journal of Inclusive Education, 15(2),
263–279. https://doi.org/10.1080/13603110902795476.
SALAR (2015). Classification of the Swedish municipalities, 2011. Stockholm: Swedish Association of
Local Authorities and Regions.
SALAR (2017). Classification of the Swedish municipalities, 2017. Stockholm: Swedish Association of
Local Authorities and Regions.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
SFS (1982:763). Hälso- och sjukvårdslagen [Health and Medical Services Act]. Stockholm: 335
Socialdepartementet [Social Department].
SFS (2010:800). Skollagen [Education Act]. Stockholm: Utbildningsdepartementet [Education
department].
SFS (2001:453). Socialtjänstlagen [Social Services Act]. Stockholm: Socialdepartementet [Social
department].
Skidmore, D. (2004). Inclusion – the dynamic of school development. Maidenhead: Open University
Press.
Stockholms stad [City of Stockholm]. (2014). Skolmarknadsrapport: Fristående resursskolor i Stockholm
(2014-05-23) [Report on school markets: independent resource schools in Stockholm]. Stockholms
stad [City of Stockholm]: Utbildningsförvaltningen [Department of education].
Swedish National Agency for Education (2014). Fristående skolor för elever i behov av särskilt stöd
[Independent schools for students in need of special support] (Rapport 409). Stockholm: Swedish
National Agency for Education.
Swedish National Agency for Education (2018a). Curriculum for the compulsory school, preschool class
and school-age educare 2011. Stockholm: Swedish National Agency for Education.
Swedish National Agency for Education (2018b). Statistik i tabeller [Statistics as tables] (2016/17).
Retrieved 28/06 2018 from https://www.skolverket.se/skolutveckling/statistik/om-skolverkets-
statistik.
Säljö, R. (2012). Schooling and spaces for learning. In E. Hjörne, G. van der Aalsvort, & G. de Abreu
(Eds.), Learning, social interaction and diversity–exploring identities in school practices (pp.
9–14). Rotterdam: Sense Publishers.
Taylor, S. S. (2005). Special education and private schools. Principals’ points of view. Remedial and
Special Education, 26(5), 281–296.
The National Agency for Special Needs Education and Schools (2012). Tillgänglighetsmodell. Retrieved
26/11 2018 from http://www.spsm.se/sv/stod-i-skolan/tillganglighet/tillganglighetsmodell/.
The Structure of the European Education Systems 2014/15. Retrieved from http://eacea.ec.europa.eu/
education/eurydice/documents/facts_and_figures/education_structures_EN.pdf.
Thomas, G. (2013). A review of thinking and research about inclusive education policy, with suggestions
for a new kind of inclusive thinking. British Educational Research Journal 39(3), 473–490.
https://doi.org/10.1080/01411926.2011.652070.
Tufvesson, C. (2007). Concentration difficulties in the school environment – with focus on children with
ADHD, Autism and Down’s syndrome. Lund University: Environmental and Energy Systems
Studies.
UNESCO (United Nations Educational, Scientific and Cultural Organization) (1994). The Salamanca
Statement and Framework for Action on Special Needs Education. Retrieved 27/06 2018 from
http://www.unesco.org/education/pdf/salama_e.pdf.
United Nations - Disability (2006). Convention on the Rights of Persons with Disabilities (CRPD).
Retrieved 27/06 2018 from http://www.un.org/disabilities/documents/convention/crpd_swedish_
corrected.pdf.
Van der Aalsvort, G., & Hjörne, E. (2012). Learning, Social Interaction and Diversity–Future Challenges.
In E. Hjörne, G. van der Aalsvort & G. de Abreu (Eds.), Learning, social interaction and diversity
– Exploring identities in school practices (pp. 223–229). Rotterdam: Sense Publishers.
Vetenskapsrådet [The Swedish Research Council]. (2002). Forskningsetiska principer inom humanistisk
samhällsvetenskaplig forskning [Research ethics in humanistic and social research]. Stockholm:
Vetenskapsrådet [The Swedish Research Council].
Vetenskapsrådet (2017). God Forskningssed [Good research tradition]. https://www.vr.se/analys-
och-uppdrag/vi-analyserar-och-utvarderar/alla-publikationer/publikationer/2017-08-29-god-
forskningssed.html.
West, A. (2014). Academies in England and independent schools (“Fristående Skolor”) in Sweden:
Policy, privatisation, access and segregation. Research Papers in Education 29(3), 330–350,
http://dx.doi.org/10.1080/02671522.2014.885732.
WHO (2007). International Classification of Functioning, Disability and Health – Children and Youth
Version. ICF-CY. Retrieved 07/12 2016 from http://www.who.int.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
336 von Wright, M. (2000). Det relationella perspektivets utmaning. En personlig betraktelse. [The challenge
of the relational perspective– a personal viewing] In Swedish National Agency for Education
(Red.), Att arbeta med särskilt stöd, några perspektiv [Working with special support, some
remarks] (pp. 6–19). Stockholm: Swedish National Agency for Education.
Zehavi, A. J. (2012). Veto players, path dependency, and reform of public aid policy toward private
schools: Australia, New Zealand, and the United States. Comparative Politics 44(3), 311–330.
https://doi.org/10.5129/001041512800078922.
Endnotes
1 ADHD, autism spectrum disorders and Tourette syndrome are some of the most common
neuropsychiatric disabilities. People with neuropsychiatric disabilities have different cognition, i.e. the
person perceives information and experiences, and processes sensory impressions in a different way.
People surrounding the individual must therefore understand and adjust to the needs of those with
neuropsychiatric disabilities so that those affected can function in their everyday life and education.
Adjustments made are dependent on the individual. For students with a different cognition to develop
and learn in the best way a thorough pedagogical mapping is needed. From this mapping, it should be
clear what adjustments and special support the individual student needs (National Association Attention,
Attention, www.attention.se).
2 Charter Schools and Students with Disabilities, Center for Law and Education under contract
to the Council of Parent Attorneys and Advocates (COPAA), USA
3
4 iii Expressed in IF-terminology (WHO, 2007), a description of activity-level.
5 ͮiv There are two different occupational groups working with special educational support in
Sweden: special teachers and special educators [special pedagogues].
v KBT (Cognitive Behaviour Therapy), ART (Aggression Replacement Therapy), SET (Social
and Emotional Training), KIT (Cognitive Affective Training), TBA (Applied Behaviour Analysis), T4C
(Thinking for a Change) and ADAD (Adolescent Drug Abuse Diagnosis). Educational elements that are
used to different degrees include FMT (Functional Music Therapy), TEACCH (Treatment and Education
of Autistic and related Communication handicapped CHildren) and methods for communication training
such as PECS (Picture Exchange Communication System).

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
337
Received: March 20, 2019 Accepted: June 03, 2019
Lotta Anderson PhD, Senior Lecturer in Special Needs Education, Malmö University, Faculty of Education
and Society, Nordenskiöldsgatan 10, SE-205 06 Malmö, Sweden.
E-mail: lotta.anderson@mau.se
Website: http://forskning.mah.se/id/luloan
ORCIDid: 0000-0003-1616-7545
https://orcid.org/0000-0003-1616-7545
Gunvie Möllås PhD, Senior Lecturer in Special Needs Education, Jönköping University, School of
Education and Communication, P.O. Box 1026 SE-551 11 Jönköping, Sweden.
E-mail: gunvie.mollas@ju.se
Website: https://ju.se/forskning/forskningsinriktningar/larandepraktiker-i-och-utanfor-
skolan-lps/ccdju.html
ORCID id: 0000-0001-7823-557X
https://orcid.org/0000-0001-7823-557X
Lisbeth Ohlsson PhD, Senior Lecturer in Special Needs Education, Kristianstad University, Faculty of
Education, SE-291 88, Kristianstad, Sweden.
E-mail: lisbeth.ohlsson@hkr.se
Website: https://www.hkr.se/personal/lisbeth.ohlsson
ORCID id: 0000-0002-7242-1424
https://orcid.org/0000-0002-7242-1424

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
338
DEVELOPING OF FACTOR STRUCTURE
FOR LEARNING STRATEGIES OF
ESTONIAN STUDENTS IN MATHEMATICS
AT THE UNIVERSITY LEVEL
Indrek Kaldo
Estonian Business School, Estonia
E-mail: indrek.kaldo@ebs.ee
Kandela Õun
University of Tartu Pärnu College, Estonia
E-mail: kandela.oun@ut.ee
Abstract
This research reports learning strategies of the first-year Estonian university students in mathematics.
The data were collected during two years from 440 university students of different disciplines. The
respondents were among students who take at least one compulsory mathematics course during their first
study year. The participants filled out a Likert-type questionnaire that was developed using previously
published instruments. The aim of this research was to examine the 69-item LIST questionnaire first time
for Estonian university students. By means of an exploratory factor analysis, 9 factors out of 12 were
confirmed. The research confirmed most of the components identified in earlier studies. It validates the
use of the instrument in further studies of learning strategies at the university level in Estonia. This gives
a positive signal about the usefulness of the instrument, as the component structure remains stable in
different populations.
Keywords: learning strategies, LIST questionnaire, mathematics education, mathematics related affect,
university mathematics.
Introduction
Research into mathematics education at the tertiary level may be itself an interesting
field of research and may give rise to useful results for teachers in all educational levels to
apply to their teaching. This research is so far, the only study of students’ learning strategies in
mathematics in Estonia at the university level and until now the area has been unexplored in
Estonia. One of the strategic objectives of the Estonian higher education strategy is to motivate
students to study natural and exact sciences and technology at the tertiary level (Estonian
Ministry of Education and Research, 2006). Estonian students showed excellent achievement in
the PISA studies, and Estonian students know that mathematics is important (Estonian Ministry
of Education and Research, 2013; 2017). PISA 2012 and 2015 showed that Estonia's basic
school students rank among the best in the world while being at the absolute top in Europe
(Estonian Ministry of Education and Research, 2017), but at the same time they conclude that
mathematics is boring (Kislenko, 2009). However, at the university level natural sciences, pure
science and technology are not popular fields of study and the dropout rate is high.
The research at the tertiary level in Estonia is limited to a few research papers (Kaldo
& Hannula 2012; Kaldo, 2014), which indicated that females are more motivated to study

Indrek KALDO, Kandela ÕUN. Developing of factor structure for learning strategies of Estonian students in mathematics at the
university level
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
mathematics. These recent studies (Kaldo & Hannula, 2014; Kaldo, 2014) showed that more 339
than two-thirds of the students think that what they are learning in mathematics is interesting.
The problem is that more than half of the students do not understand everything that they
have done in mathematics over the last year (Kaldo, 2014). This calls for a change in learning
strategies in mathematics at the university level.
In the following are given the selection of approaches to capture learning strategies
which reflect the importance of affective and motivational issues. Pintrich and DeGroot (1990)
developed a questionnaire “The Motivated Strategies for Learning Questionnaire” (MSLQ)
to measure the types of learning strategies and academic motivation used by college students
and uses a Likert scale. There are essentially two sections to the MSLQ, a motivation section,
and a learning strategies section. In the MSLQ the learning strategy section includes 31 items
regarding students' use of different cognitive and metacognitive strategies (Pintrich, Smith,
García & McKeachie, 1991). By Griese, Lehmann & Roesken-Winter (2015) the Approaches
to Studying Inventory (ASI) by Entwistle and Ramsden (1983) and its refinements (ASSIST
by Tait, Entwistle & McCune, 1998; ALSI by Entwistle & McCune 2004) feature the main
distinction of categorizing learning behaviour as being of either strategic or of apathetic
approach (Griese, Lehmann & Roesken-Winter, 2015). Weinstein and Palmer (2002) described
another self-report instrument to assess students’ learning strategies, which is the Learning
and Study Strategies Inventory (LASSI). This instrument LASSI (Weinstein & Palmer, 2002)
covers thoughts, behaviours, attitudes and beliefs in relation to successful learning that can also
be fostered by interventions (Griese et al., 2015).
Another well-known questionnaire is the German LIST questionnaire (Wild & Schiefele,
1994), which is based on the same classification as MSLQ and takes up aspects from LASSI
as well (Griese et al., 2015). LIST questionnaire has been modified and tested several times
since 1994 and has been applied in the context of many subjects, including mathematics
(Liebendörfer, Hochmuth, Schreiber, Göller, Kolter, Biehler, Kortemeyer, & Ostsieker, 2014;
Griese et al, 2015). Griese et al. (2015) examined the 69-item LIST questionnaire for 2374
STEM students from different engineering courses at Ruhr-Universität Bochum in Germany,
typically predominantly males. Gómez-Chacón, Griese, Rösken-Winter and Gonzàlez-Guillén
(2015), explored by means of the LIST questionnaire learning strategies for two samples of
113 Spanish and 159 German engineering students. Griese et al. (2015), research focuses on
engineering students in their first semester at the university. Out of the students questioned,
77.70 % are males, 22.30 % females in their study. In the paper Griese et al. (2015), learning
strategies are understood “as all kinds of planned and conscious learning behaviour and the
attitudes behind it, involving observable actions (e.g. solving tasks, asking questions, taking
notes) as well as thought processes (e.g. planning, reflecting) on the basis of both cognitive and
affective-motivational dispositions”.(p. 2)
Therefore, the question is raised about how students use different learning strategies
in mathematics. The latest research in Germany (Griese et al., 2015) gives one instrument,
but no previous researches have explored Estonian university students’ learning strategies
in mathematics. The aim of this research was to use some of the published instruments on
mathematical learning strategies first time in Estonia at university level. This instrument was
then used to confirm its applicability in Estonia at the university level. In this research, the
following research question was particularly pursued: What kinds of factor structure in learning
strategies in mathematics do students from Estonia hold at the university level? We hope to get
a reliable instrument and then later to respond to the research questions that will help solve the
societal problems that were pointed out as the motivation to study and decreasing the dropout
rate in universities. We must also concern later ourselves with helping students learn how to
study mathematics because students must also accept some of the responsibility instead of
relying only on the lecturers.

university level
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
340 Research Methodology
General Background
We are influenced by the work of Griese et. Al (2015) and for our research in Estonia,
the decision was made in favour of the Griese et al. (2015) 69-LIST questionnaire which
is modified and up-to-date. As there are no analogous Estonian questionnaires on learning
strategies, this research opted for the LIST questionnaire, thus hoping for the further asset of a
parallel instrument for different countries. The quantitative data were gathered in 2017 and 2018
from students of five universities of Estonia. Tallinn University of Technology (approximately
70,000 alumni) is the only technological university in Estonia, University of Tartu is Estonia's
national university (around 13,000 students) and belongs to the top 1% of the world's most-cited
universities and research institutions. The Estonian University of Life Sciences (number of
students 2500) is one of top 100 universities in the world in the field of agriculture and forestry,
Estonian Business School is the oldest privately owned business university in the Baltics with
more than 1500 students. Estonian Aviation Academy (approximately 300 students) is a state-
owned professional higher education institution educating and training specialists for Estonian
aviation enterprises and organisations.
Sample of Research
This research was carried out in Estonia at the university level. A nationally representative
study provides a strong basis for research at the tertiary level as well as for the conclusions and
educational implications. In order to gain a complete picture and to get a representative sample,
almost all the universities in Estonia were covered (a total of 5 universities in 2 cities: Tallinn
and Tartu). In Estonia, there are only a few universities and they focus on different subjects.
Data collected during two years (in the 2017 and 2018 spring semester) to get more data for
analyses. The participants were 440 volunteer bachelor students taking at least one first-year
compulsory math course at the university level at the universities introduced previously. The
questionnaire was completed during the mathematics lectures that were compulsory for the
students and participation was voluntary. There were 234 males and 206 females; 329 students
studying in Estonian and 111 students studying in English (Figure 1).
Figure 1. Counts (on coloured areas) and percentages of respondents’ socio-

demographic data.

university level
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
According to Estonian Educational Information System (www.ehis.ee), open for 341

everybody, no confidential information included) there were 1461 bachelor students in year
2015/2016 who have mathematics course in their study program so the database is presenting
population quite well. All results presented in groups and analysed used statistical methods.
Instrument and Procedures
The LIST questionnaire used in this research was developed in 1994 as part of a research
project in Germany. The statements in the questionnaire are grouped into 13 topics (Wild &
Schiefele, 1994). Griese et al. (2015) modified the original LIST questionnaire to 69-items LIST,
they removed the scale Critical Checks because it did not seem appropriate for mathematics
at the beginning of the university. By the same reason the 69-items LIST was used because
this sample was first-year university students. In the research, a quantitative (questionnaire)
research strategy was used. The questionnaire was used to study students’ learning strategies in
mathematics.
The questionnaire used in Estonia was translated before the pilot study into Estonian
and back to English (Kaldo & Õun 2019, submitted). As one of the aims of the pilot study
was to make a comparative analysis, then the translation had to have been carried out with a
high degree of caution. Gorard’s (2001) suggestion was followed during the study where he
recommends that:
… if you are working in one language and translating your instrument into another
language before completion (a common process for overseas students), then use the techniques
of back translation as well. In this, the translated version is translated back into the original
language by a third person as a check on the preservation of the original meaning. (p. 91)
The study in Spain (cf. Gómez-Chacón et al., 2015), showed that LIST keeps its qualities
when being used in another country: “after being translated into English and then into Spanish,
the cognitive and metacognitive scales from LIST kept their reliability, an indication for the
questionnaire’s universal applicability” (Gómez-Chacón et al., 2015).
Since the purpose of the research was to confirm the earlier scales on learning strategies
(Griese et al., 2015), the original scales from the earlier research were used and their reliabilities
were computed and the same component names were used: Organizing, Elaborating, Repeating
etc. The Exploratory factor analysis used for computing factors. That statistical method has
three main uses: 1) to understand the structure of a set of variables; 2) to construct an instrument
to measure an underlying variable; and 3) to reduce a data set to a more manageable size while
retaining as much of the original information as possible (Field, 2009).
Moreover, the structure of the learning strategies on mathematics was explored through
calculating correlations among the reliable components.
Participants filled in a questionnaire on paper. The students were asked to respond on a
4-point Likert scale: strongly disagree, partly disagree, partly agree, and strongly agree). The
students were given at least 30 minutes to fill in the questionnaire and told the questionnaire
was anonymous and we collected 440 questionnaires.
Since the purpose of the research was to confirm the 69-item LIST of students’ learning
strategies in mathematics (Griese et al., 2015),
Data Analysis

The items from the earlier research were used and the reliabilities of the modified scales
were computed (Table 1).
In this research Field’s (2009) directions were followed. The decision to use the
exploratory factor analysis for the questionnaire came from the following reasons: 1) the

university level
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
342 previous questionnaires were not tested in a similar population; i.e., these were used in Estonia
for the first time; 2) the sample in this study is large and varied enough to make an exploratory
analysis compelling. In this research, an exploratory factor analysis was done, which revealed
factors similar to the earlier studies. For the exploratory factor analysis, the maximum
likelihood method with direct oblique rotation was used to determine useful and statistically
robust dimensions regarding this construct.
This method of factor analysis allows for the making of inferences from sample to
population; the sample of 440 students is, therefore, large and adequate enough. Oblique
rotation is used when factors are allowed to correlate (Field, 2009). The factors of the
learning strategies in mathematics cannot be regarded independently of each other; therefore,
correlations among factors should be allowed. In that case, an oblique rotation will lead to a
better estimation of factors since it derives factor loadings based on the assumption that they are
correlated (Fabrigar, Wegener, MacCallum, & Strahan, 1999). The Kaiser criterion is based on
the idea that the eigenvalues represent the amount of variation explained by a factor and that an
eigenvalue of 1 represents a substantial amount of variation and therefore the recommendation
is to retain all factors with eigenvalues greater than 1 (Field, 2009). The program SPSS with the
Kaiser criteria “eigenvalue > 1” gave a suggestion to use 17 factors. Field (2009) argued that
with a sample of more than 200 participants, the Catell’s scree-test provides a fairly reliable
criterion for factor selection. According to the Cattell’s scree–test, after an inspection of the
scree plot, the proper number of factors appeared to be between 7 and 8. A 12-factor solution
for the whole survey was used, because a 12-factor solution corresponded with the number of
factors predicted from the original studies. Another reason was that some factors contained only
two items factor solutions or their Cronbach alphas were low. Items which had communalities
of less than 0.3 were removed, because these lowest communalities are not significant (Hair,
Anderson, Tatham, & Black,1998). Moreover, the structure of the view of mathematics was
explored through calculating correlations (Hinkle, Wiersma, & Jurs, 2009) between the reliable
components in SPSS. At least moderate correlations coefficients that are greater than 0.5 are
presented in this study. In addition, we also calculated the mean scores and standard deviations
for the whole sample (n=440) on each of the components.
Reliability and Theory of Uniting Two Database
Reliability characterises the stability, consistency and suitability of the methodology

used. Reliability shows how well the results of repeated measurements (by either the same
researcher or different researchers) carried out in the same circumstances coincide (Kask,
2009; Laius, 2011; Kaldo 2015). Reliability also indicates whether a certain indicator measures
consistently and continuously (Kask, 2009; Laius 2011). In other words, how reliable is the
result of the measurement (Kask, 2009; Laius, 2011)? In this study, the Cronbach’s alpha was
used as a measure of the internal consistency of the instrument and its subscales, which is the
most widely used measure (Hair et al., 1998). If the reliability coefficient is 0.70 or higher, it is
considered "acceptable" in most social science research situations (Hair et al., 1998).
There are many studies using data collected during different years by the same
questionnaire and it is important to compare the results sameness before uniting different year’s
databases. For continues data the best method is a two sample (or an independent sample) T-test
and it is a commonly utilized design for a straightforward comparison of two independent
groups in psychological research (Goenen, Johnson, Lu, & Westfall, 2005; Kruschke, 2013).
By Jeon and De Boeck (2017) in a two-sample setting, testing H0 based on the independent
sample T-test. The test statistic can be computed as , where and are the sample means for
Groups 1 and 2, respectively, and is the estimated SE of the difference between the means. The
t-statistic follows the Tv distribution with the degrees of freedom v=N1+N2-2. The two-sided p

university level
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
value can be computed as ; that is, the area under the sampling distribution of the t-statistic to 343
the left and the right of the absolute test statistic value when the null distribution is true. (Jeon,
& De Boeck, 2017) The statistical program SPSS Statistics 24.0 was used for the data analysis.
Results of Research
The summarized results are presented in Table 1. The Cronbach’s alpha showed reliability.
The mean scores and variances for the whole sample (n = 232) on each of the components were
also calculated. The original Cronbach’s alpha is the alpha which is used in earlier studies of
previously published instruments (Wild & Schiefele, 1994).
Table 1. The twelve factors of the students’ learning strategies in mathematics.
Factors Original Cronbach’s

Std.
(number of Sample item Cronbach’s alpha in the Mean
deviation
items) alpha research
F1 Organizing I go over my notes and structure the

0.82 0.786 2.68 0.56
(8) most important points.
F2 Elaborating I think of practical applications of new

0.77 0.796 2.73 0.54
(8) concepts.
F3 Repeating
I read my notes several times in a row. 0.73 0.747 2.42 0.54
(7)
F4 Metacogni-
I plan in advance in which order I want to
tion: Planning 0.64 0.644 2.58 0.61
work through the subject matter.
(4)
F5 Metacogni- I ask myself questions on the subject

tion: Monitoring matter in order to make sure that I have 0.64 0.628 2.37 0.61
(4) understood everything correctly.
F6 Metacogni-
Confronted with a difficult subject matter
tion: Regulating 0.64 0.666 2.76 0.60
I adapt my learning strategy accordingly.
(3)
F7 Effort I make an effort even though the subject

0.74 0.772 2.84 0.49
(8) matter may not suit me well.
F8 Attention When I am learning I notice that my

0.90 0.903 2.58 0.70
(6) thoughts tend to stray.
F9 Time manage-
ment I work according to a schedule. 0.83 0.707 2.35 0.57
(3)
F10 Learning I work in a place that makes it easy to

0.71 0.740 2.83 0.54
Environment (5) concentrate.
F11 Peer Learn- I work on tasks together with my peer

0.82 0.85 2.56 0.63
ing (7) students.
F12 Using Refer- I search for explanatory material if cer-

0.72 0.806 3.05 0.64
ence (4) tain facts are not completely clear.

university level
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
344 The Cronbach’s alpha used to measure reliability of factor items. It is commonly used
as a measure of the internal consistency or reliability of factors for a sample of examinees.
The mean scores and variances for the whole sample (n = 440) on each of the components
were also calculated. In our study, the factor analysis confirmed 9 factors of 12. In this study,
the factor analysis confirmed factors: F1 Organizing, F2 Elaborating, F3 Repeating, F7 Effort,
F8 Attention, F9 Time management, F10 Learning Environment, F11 Peer Learning and F12
Using Reference. Three factors did not confirm for low reliability: F4 Metacognition: Planning;
F5 Metacognition: Monitoring and F6 Metacognition: Regulating. By using the same number
of questions as in the initial factors, in the factors F9 Time management, F10 Learning
Environment the reliability was less than 0.7 and therefore we removed one question in both
factors and we got reliable factors.
Compared with pilot study, three factors did confirm low reliability: F4 Metacognition:
Planning; F5 Metacognition: Monitoring and F6 Metacognition: Regulating.
To compare two-year sample similarities independent samples t-test was carried out
(Table 2). There were statistically significant differences between first and second year in F1,
F7, F8 and F12 but test results for these factor items and also for factors did not confirm big
differences of means by years (the mean differences were 0.17-0.29) so there is no reason to take
these two year samples as separate databases. In the first year questionnaire, some questions
needed recast for better understanding, so some differences could appear from that. Reliability
analysis confirmed the same factor structure for both years so one can conclude these two-year
samples can be united as one.
Table 2. Factors t-test results.
Group Statistics t-Test

Factors Year N Mean Std. Deviation Std. Error Mean Significance
2017 232 2.6061 0.55572 0.03649
F1 .005
2018 208 2.7539 0.54557 0.03783
2017 232 2.6945 0.53589 0.03518
F2 .119
2018 208 2.7744 0.53406 0.03703
2017 232 2.3822 0.54823 0.03599
F3 .144
2018 208 2.4577 0.53202 0.03689
2017 232 2.5420 0.58675 0.03852
F4 .114
2018 208 2.6322 0.60626 0.04204
2017 230 2.3402 0.59623 0.03931
F5 .328
2018 208 2.3970 0.61697 0.04278
2017 232 2.7763 0.57281 0.03761
F6 .561
2018 208 2.7430 0.62929 0.04363
2017 232 2.8959 0.48878 0.03209
F7 .008
2018 208 2.7713 0.49023 0.03399
2017 232 2.5042 0.68433 0.04493
F8 .017
2018 208 2.6637 0.70833 0.04911
2017 232 2.3103 0.56131 0.03685
F9 .107
2018 208 2.3978 0.57322 0.03975
2017 232 2.8138 0.52381 0.03439
F10 .573
2018 208 2.8430 0.56333 0.03906

university level
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
2017 232 2.5695 0.62946 0.04133 345

F11 .824
2018 207 2.5560 0.63720 0.04429
2017 232 3.1070 0.58647 0.03850
F12 .035
2018 207 2.9791 0.68317 0.04748
Initially, the structure of the students’ learning strategies was obtained (Table 3). Relations
between the factors were calculated for the confirmed nine factors.
Table 3. Correlations among the factors.
F1 F2 F3 F7 F8 F9 F10 F11 F12

F1 Organizing 1
F2 Elaborating .357** 1
F3 Repeating .504* .194* 1
F7 Effort .451** .267** .389** 1
F8 Attention -.134** -,105* -.033 -.289** 1
F9 Time management .284** .216** .259** .346** -.112* 1
F10 Learning Environ-
.313** .238** .222** .347** -.130** .320** 1
ment
F11 Peer Learning .244** .133** .182** .133* .032 .071 .1597 1
F12 Using Reference .317** .339** .269** .429** -.103* .151** .2267** .227** 1
Table 3 shows that nearly all dimensions correlate statistically significantly with each
other. All correlations with the sign ** are significant at the level .01 (2-tailed). Correlations
with the sign * are significant at the .05 level (2-tailed). The results of the correlation analysis
showed that nearly all the factors correlated statistically significantly with each other. However,
the strength of the correlation in the survey (Hinkle, Wiersma & Jurs, 2009) varied from little,
if any (.00 to .29) to low (.30 to .49) and moderate (.50 to .70). Moderate correlations are the
following factors: Organizing (F1) and Repeating (F3) were found to correlate with a coefficient
of .504. The correlations of the rest of the factors are weak. In the pilot study the same results
were obtained.
Discussion
Research into mathematics education at the tertiary level may be itself an interesting
field of research and may give rise to useful results for teachers in all educational levels to
apply to their teaching (Alsina, 2001; Abdulwahed, Jaworski & Crawford, 2012). Based on
studies carried out by researchers in other countries, it is clear that students’ learning strategies
in mathematics are important areas in mathematics education and need attention in an Estonian
context. This study is so far the only investigation of students’ learning strategies in mathematics
in Estonia at the university level and until now the area has been unexplored in Estonia. If we
look at researches at the tertiary level in other countries in the field of learning strategies in
mathematics, then we cannot find many studies (for example, Gómez-Chacón et al., 2015;
Entwistle & McCune, 2004). The latest research in Germany confirmed the instrument as
acceptable instrument at the tertiary level.
Research question: What kinds of factor structure in learning strategies in mathematics
do students from Estonia hold at the university level?

university level
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
346 In the research, we used the instruments developed by Griese et. al (2015) and this is the
German LIST questionnaire, which is based on the same classification as MSLQ and takes up
aspects from LASSI as well and it is up to date questionnaire for the university level students.
The Cronbach’s alpha is commonly used as a measure of the internal consistency reliability of a
questionnaire. If the reliability coefficient is 0.70 or higher, it is considered to be "acceptable" in
most social science research situations. In the Table 1 nine factors had a high Cronbach’s alpha
and their reliability for Estonian university students was confirmed. The reliability of three
factors used in the study was not confirmed. However, those that were not found to be reliable
were not far from the threshold level. The standard deviation of the item responses was very
low, which may have contributed to the low reliability of the scale in this sample.
This research concluded with exactly the same results (Wild & Schiefele, 1994). The
difference between the study of STEM students’ learning strategies (Griese et al., 2015) was
that in their study Metacognition factors were confirmed.
The self-criticism of our data collection is that the questionnaire was handed in on paper.
For the manual insertion of data in the file for SPSS professional help was used. For web-based
questionnaires, there is no need to insert data in the file, but the weakness is that in this case
it is hard to get a large sample size. This was the reason why paper and pen were used for the
questionnaires.
The scales had been previously tested on high school students and university students
outside of Estonia, which suggest that the differences in reliability can be related to differences
between samples (age, level of study, field of study, culture) or the translation of the items. This
claim is coherent with Diego-Mantecón et al.’s (2007) conclusion that questionnaires can be
sensitive to variables such as student age, gender and nationality.
Conclusions
Based on researches carried out by researchers in other countries, it is clear learning

strategies in mathematics are important areas in mathematics education and need attention in an
Estonian context. In this research the sample size and representativeness of all the universities
is one of the strengths of the research. One task of the research was to check the reliability
of the questionnaire. That means that we can use the questionnaire later for the survey. Nine
reliable factors for students’ learning strategies in mathematics were obtained. Based on the
study analysis, the structure of the first-year baccalaureate students’ learning strategies in
mathematics is coherent with the structure from other researches’ (Wild & Schiefele, 1994;
Griese et al., 2015) structures. This gives a positive signal about the usefulness of the instrument
for learning strategies in mathematics, as the component structure remains stable in context of
Estonia population at university level.
The study confirmed same aspects of learning strategies as Griese (2015) reported.
Learning strategies in mathematics are understood as all kinds of planned and conscious learning
behaviour and the attitudes behind it, involving observable actions as well as thought processes
on the basis of both cognitive and affective-motivational dispositions, can be identified and
validly measured as separate components of Estonian university students’ learning strategies
in mathematics. However, some robust differences in scale reliabilities beg for caution when
importing instruments to new cultural contexts.

university level
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
References 347
Abdulwahed, M., Jaworski, B., & Crawford, A. R. (2012). Innovative approaches to teaching mathematics
in higher education: A review and critique. Nordic Studies in Mathematics Education, 17(2), 49-
68.
Alsina, C. (2001). Why the professor must be a stimulating teacher. In D. A. Holton (Ed.), The teaching
and learning of mathematics at the university level: an ICMI Study (pp. 3–12). Dordrecht: Kluwer
Academic Publishers.
Entwistle, N., & McCune, V. (2004). The conceptual bases of study strategies inventories. Educational
Psychology Review, 16(4), 325–346.
Entwistle, N., & Ramsden, P. (1983). Understanding student learning. London: Croom Helm.
Estonian Ministry of Education and Research. (2006). Estonian Higher Education Strategy 2006-2015.
Retrieved from www.hm.ee/index.php?popup=download&id=7653‎.
Estonian Ministry of Education and Research. (2013). The PISA 2012 survey results summary. Retrieved
from http://www.hm.ee/index.php?popup=download&id=12474
Estonian Ministry of Education and Research. (2017). Summary of PISA 2015 survey results for Estonia.
Retrieved from https://www.hm.ee/sites/default/files/pisa_2016_booklet_eng.pdf
Field, A. (2009). Discovering statistics using SPSS (3rd Ed.). London: Sage Publications.
Goenen, M., Johnson, W. O., Lu, Y., & Westfall, P. H. (2005). The Bayesian two-sample t test. American
Statistician, 59, 252–257.
Gómez-Chacón, I. M., Griese, B., Rösken-Winter, B., & Gonzàlez-Guillén, C. (2015). Engineering
students in Spain and Germany—varying and uniform learning strategies. In Vondrova, N.,
& Krainer, K. (Eds.), Proceedings of the 9th Congress of European society for Research of
Mathematics Education. (2117–2123) Prague: ERME.
Gorard, S. (2001). Quantitative methods in educational research. London: Continuum.
Jeon, M., & De Boeck, P. (2017). Decision qualities of Bayes factor and p value-based hypothesis testing.
Psychological Methods, 22(2), 340-360
Griese, B., Lehmann, M., & Roesken-Winter, B. (2015). Refining questionnaire-based assessment of
STEM students’ learning strategies. International Journal of STEM Education, 2(1), 12.
Hair, J. F., Anderson, R. E., Tatham, R. L., & Black, W. C. (1998). Multivariate data analysis (5th Ed.).
Prentice Hall.
Hinkle, D. E., Wiersma, W., & Jurs, S. G. (2009). Applied statistics for the behavioural sciences. Belmont:
Wadsworth Publishing.
Kaldo, I. (2014). What kinds of view of mathematics do students from Estonia hold? Nordic Studies in
Mathematics Education, 19(2), 101-122.
Kaldo, I. (2015). University Students’ View of Mathematics in Estonia. Doctoral dissertation, Tallinn
University, Estonia.
Kaldo, I., & Hannula, M. S. (2012). Structure of students’ view of mathematics in Estonia. Nordic Studies
in Mathematics Education, 17(2), 5-26.
Kaldo, I., & Hannula, M. S. (2014). Gender differences favouring females in university students’ views of
mathematics in Estonia. Nordic Studies in Mathematics Education, 19(1), 3-22.
Kask, K. (2009). A study of science teacher development towards open inquiry teaching through an
intervention programm. Doctoral dissertation, Tartu University, Estonia.
Kislenko, K. (2009). Mathematics is a bit difficult but you need it a lot: Estonian pupils’ beliefs about
mathematics. In J. Maaß and W. Schlöglmann (Eds.), Beliefs and Attitudes in Mathematics
Education: New Research Results (pp. 143-164). Rotterdam: Sense Publishers.
Kruschke, J. K. (2013). Bayesian estimation supersedes the t test. Journal of Experimental Psychology:
General, 142, 573–603.
Laius, A. (2011). A longitudinal study of science teacher change and its impact on student change in
scientific creativity and socio-scientific reasoning skills. Doctoral dissertation, Tartu University,
Estonia.

university level
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
348 Liebendörfer, M., Hochmuth, R., Schreiber, S., Göller, R., Kolter, J., Biehler, R., Kortemeyer, J.,
& Ostsieker, L. (2014). Vorstellung eines fragebogens zur erfassung von lernstrategien in
mathematikhaltigen studiengängen. [Presentation of a questionnaire for the acquisition of
learning strategies in mathematical study courses] In Roth, J. & Ames, J. (Eds.) Beiträge zum
mathematikunterricht 2014 (pp. 739–742). Münster: WTM.
Pintrich, P. R., & Smith, D. A. F., García, T., McKeachie, W. J. (1991). A Manual for the Use of the
Motivated Strategies for Learning Questionnaire (MSLQ). Retrieved from https://eric.
ed.gov/?id=ED338122.
Pintrich, P. R., & DeGroot, E. V. (1990). Motivational and self-regulated learning components of
classroom academic performance. Journal of Educational Psychology, 82, 33-40.
Tait, H., Entwistle, N., McCune, V. (1998). ASSIST: A reconceptualisation of the approaches to studying
inventory. In C Rust (Ed.), Improving student learning: improving students as learners (pp. 262–
271). Oxford: Oxford Centre for Staff and Learning Development.
Weinstein, C. E., Palmer, D. R. (2002). Learning and study strategies inventory (LASSI): user’s manual
(2nd ed.). Clearwater: H & H Publishing.
Wild, K. P., & Schiefele, U. (1994). Lernstrategien im studium. Ergebnisse zur Faktorenstruktur
und Reliabilität eines neuen Fragebogens [Learning strategies of university students: Factor
structure and reliability of a new questionnaire]. Zeitschrift für Differentielle und Diagnostische
Psychologie, 15, 185–200.
Received: March 20, 2019 Accepted: June 06, 2019
Indrek Kaldo PhD, Associate Professor, Estonian Business School, A. Lauteri 3, Tallinn 10114, Estonia.
E-mail: indrek.kaldo@ebs.ee
Website: http://www.ebs.ee
Kandela Õun MSc, Lecturer, University of Tartu Pärnu College, Ringi 35, 80012 Pärnu, Estonia.
E-mail: kandela.oun@ut.ee
Website: http://www.pc.ut.ee

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
CLASSIFICATION OF ENGINEERING 349
STUDENTS’ SELF-EFFICACY TOWARDS

VISUAL-VERBAL PREFERENCES USING
DATA MINING METHODS
Citra Kurniawan, Punaji Setyosari, Waras Kamdi, Saida Ulfa
State University of Malang, Indonesia
E-mail: airakurniawan@gmail.com, punaji.setyosari.fip@um.ac.id,
waras.ft@um.ac.id, saida.ulfa.fip@um.ac.id
Abstract
The purpose of this research was to build a classification model and to measure the correlation of self-
efficacy with visual-verbal preferences using data mining methods. This research used the J48 classifier
and linear projection method as an approach to see patterns of data distribution between self-efficacy
and visual-verbal preferences. The measurement of the correlation of engineering students’ self-efficacy
with visual-verbal preferences using the data mining method approach gets the result that self-efficacy
does not correlate with visual-verbal preferences. However, engineering students’ self-efficacy influences
the achievement of initial learning outcomes. Visual-verbal preference is more influenced by students’
interest in images so it can be concluded that self-efficacy affects the initial results of learning but does
not have a correlation with visual-verbal preferences. The results of the decision tree provide the results
that are easily understood and present a correlation between self-efficacy and visual-verbal preferences
in a visual form.
Keywords: self-efficacy, visual-verbal preferences, data mining.
Introduction
Every student has different levels of self-efficacy because they have different initial
abilities and learning experiences. Self-efficacy influences people ‘s belief to face failure and
try harder in achieving success. Success can build a robust belief in the level of confidence. If
someone achieves success easily, then he/she are easily discouraged by failure because the level
of confidence requires experience to overcome the problems that occur. The experience gained
when dealing with problems becomes capital to help improve self-efficacy (Bandura, 1994).
Self-efficacy refers to the ability of someone who uses prior experience references to solve
problems (Boswell, 2013). Individuals who have high self-efficacy have high confidence to deal
with problems, while individuals with low self-efficacy have fears of facing failure (Wu, Tsai,
& Wang, 2011). Students try to process information and appraise their self-efficacy from ability
and learning experience. The success of students to overcome problems can increase self-
efficacy and reduce failure (Schunk, 2003). Self-efficacy is formed from experience, common
experiences, social persuasions, and physiological reactions (Jordan, Amato-henderson, Sorby,
& Donahue, 2011). Self-efficacy in the engineering field is very important. Bandura (1997)
explained that self-efficacy determines the action to be chosen, how much effort is made to
solve the problem, how can they survive in failure and realize the level of self-achievement
(Bandura, 1997; Marra & Bogue, 2006). Engineering students need quantitative skills to prepare

Citra KURNIAWAN, Punaji SETYOSARI, Waras KAMDI, Saida ULFA. Classification of engineering students' self-efficacy towards
visual-verbal preferences using data mining methods
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
350 themselves to face problems in engineering courses. Self-efficacy contributes to academic

performance even though the factors of problem-solving ability and intellectual ability also
influence learning outcomes (Aleta, 2016). The factors that influence confidence in success
in engineering students are mastery experiences, vicarious experiences, social persuasions,
and physiological states. Classroom and curricular practice influence students’ engineering to
have self-confidence, retention, and success (Hutchison, Follman, & Bodner, 2005). Bandura
revealed that engineering self-efficacy was measured using the developed self-efficacy scale
(Bandura, 2006). Carberry et al. (2010) showed three findings to measure self-concept in student
engineering, namely (1) measurement design considers the factors of self-efficacy, motivation,
anxiety, and outcome expectation; (2) self-efficacy depends on the experience of the engineer
students; (3) High correlation between self-efficacy with motivation, expertise and outcome
expectation (Carberry, Hee-Sun, & Ohland, 2010).
Engineering courses are often associated with things related to configuration, symbols,
codes, and topology. Engineering students have process information based on their preferences.
When Engineering students have presented a content consisting of images and text, it will try
to process which content matches their preferences (Kurniawan, Setyosari, Kamdi, & Ulfa,
2018). If a student who has visual preferences processes information from the content, then
the first time he is looking for is image content. If the content displayed is only in the form of
text, he will still try to process information even though the content does not match his learning
preferences (Peterson, 2016; Plass, Chun, Mayer, & Leutner, 1998). Visualization is essential
in learning. Visualization can simplify the information that is difficult to understand (Sudatha,
Degeng, & Kamdi, 2018). A student always has different preferences in processing information.
Also, students also have self-efficacy in dealing with problems in the information processing
process. Someone who has high self-efficacy tends to be able to make more efforts in processing
information if the information presented does not match his preferences. Therefore, research is
needed, which aims to identify the self-efficacy that a person has seen from the visual-verbal
preferences approach.
Problem of Research
Data mining methods aim to determine the classification model by determining data
classes and grouping examples based on similarity attributes. Previous research measured
correlation using descriptive statistical methods, such as in the research of measuring cognitive
style visualizer correlations to the achievement of learning outcomes in design modeling
and performance (Pektaş, 2013). In the research Pektaş (2013) used the analysis of variance
analysis (ANOVA) to determine whether cognitive style has any effect on design modelling and
performance. However, descriptive statistics method is not optimal for drawing correlations
in the form of data visualization so that the tendency of one variable/class cannot be seen.
Therefore, the measurement of correlation of self-efficacy to visual-verbal preferences with data
mining methods is needed as an alternative method besides descriptive statistical methods. This
research was tried to build a classification model for experimental data that has been collected.
This research discusses the analytical method for measuring the correlation of engineering
students’ self-efficacy if it is associated with visual-verbal preferences.
Research Focus
The focus of this research was: (1) building a data mining classification model, and (2)
measuring the correlation of engineering students’ self-efficacy and visual-verbal preferences
using data mining methods.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
Research Methodology 351
General Background
This study used experimental research with data mining methods for educational data.
Data mining in this research is a classification technique assisted by WEKA data mining software
(Abernethy, 2010; Witten, Frank, Hall, & Pal, 2017) and Orange data mining (Demšar et al.,
2013). Data mining classification in this research uses decision tree-J48 (WEKA) and Linear
Projection (Orange) classification techniques. WEKA data mining provides various methods for
classifying (Kabakchiev et al., 2017). This research used Decision Tree-J48 to do classification.
Decision tree-J48 is the implementation of the C4.5 algorithm in WEKA data mining. The
C4.5 algorithm has a method for breaking nodes into several nodes based on the similarity
of attribute data. Linear projections provide an overview of the linearity correlation between
engineering students’ self-efficacy and visual-verbal preferences displayed in graphical form.
Sample
Participants in this research were 250 engineering students, with details of 72 female
participants and 178 male participants, as shown in Table 1.
Table 1. Distribution of participant.
Preferences
Self-Efficacy
Verbal Visual Total
High Self-efficacy
Female 4 28 32
Male 26 47 73
Low Self-efficacy
Female 12 28 40
Male 28 77 105
Figure 1 shows the preferences and attributes of self-efficacy for each gender, where
participants who had high self-efficacy consisted of 32 female and 73 male. Participants who
had low self-efficacy included 40 female and 105 male. Preference has two attributes, namely
visual and verbal, while self-efficacy has two attributes, namely high and low. The measurement
of the self-efficacy scale uses the self-efficacy instrument developed by Bandura (2006), where
students who obtain value on means from the measurement results are grouped into groups
of students with high self-efficacy, while students who obtain the value under means from the
measurement results grouped into groups of students with low self-efficacy.
Instrument and Procedures
Participants in this research were given several tests aimed at measuring the preferences
of visual-verbal, self-efficacy, and pre-tests. Test results were processed using data mining
methods, namely decision trees, and linear projection classifications. Visual-verbal preference
was measured using a visual-verbal questionnaire (VVQ) developed by Richarson (1977) which
contained the VVQ category (Richardson, 1977) and Kirby (1988) that developed question
item of VVQ (Kirby, Moore, & Schofield, 1988). While the self-efficacy measurement used a

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
352 self-efficacy questionnaire (Bandura, 2006). Meanwhile, the analysis of the pre-test results in
this research has used the questions given in the Cisco Networking Academy Program, Chapter
1 to measure the initial value of the pre-test (Cisco Systems, 2003). Data mining process data
classified into five classes, each of which is self-efficacy, gender, interest in images, preferences,
and pre-test results, can be seen in Figure 1.
Figure 1. Class of data.
Self-efficacy has two attributes, namely “high” and “low,” gender has two attributes,
namely “male” and “female,” image interest has two attributes, namely “interested” and “not
interested,” preference has two attributes, namely “verbal” and “visual.
Data Analysis
The classification phase in data mining consists of three stages, namely (1) experimental
data; (2) modelling; (3) evaluation (Demšar et al., 2013). Experimental data were presented
data in five classes and their attributes. Experimental testing data use 10-fold cross validation
because this test is effective for limited data (Kabakchiev et al., 2017). The classification
developed in this research can be seen in Figure 2.
Figure 2. Classification model.
When the classification algorithm functions, data were distributed in two data
sets consisting of training data and test data. The algorithm runs ten times and produces a
classification model, as shown in Figure 3.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
353
Figure 3. Data mining task implementation-classification.
The model classification in Figure 2 shows the class of data grouped based on the
similarity of attributes to form a decision tree model. Outliner data were processed by the
linear projection method. This method displays linear projections from classes labelled data.
The projection in question is to generalize graphical projections and consider the effect of
projections on geographical objects (Orange Data Mining, 2015). Rules classify attributes
and attributes by making algorithm models in decision making. The algorithm for building a
decision tree model can be seen in Figure 4.
Figure 4. Model of a decision tree classifier.
The decision tree model in this research predicted the possibility of a class formed by the
attributes possessed by the instance. The class was the status owned by the instance. The Class
is often referred to as conclusions from data. Attributes were information that a class has. The
decision tree model can be seen as follows:
If self-efficacy=’high’ then interest in image object=’no’

Else if preferences=’ visual.’
If self-efficacy=’high’ and preferences=’ visual ‘then interest in the image object
Else if pre-test=’high’

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
354 Research Results
The number of instances in this research was 250 data instances. Each instance has a
class, namely self-efficacy, gender, interest in the object image, and preference. This research
uses data mining methods with classification techniques to identify classes, attributes, and
examples. Research data can be seen, as shown in Figure 5.
Figure 5. Instance data of the research.
Figure 5 shows an example of data in each class with each pre-test value. The Class has
105 high-value attributes, while the class that had a “low” attribute has an instance of 145. The
distribution of self-efficacy can be seen in Figure 6.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
355
Figure 6. Class of self-efficacy.
The Class “gender” had a distribution of ‘male’ attributes consisting of 178 instances
and ‘female’ consisting of 72 instances. The distribution of the class “gender” can be seen in
Figure 7.
Figure 7. Class of gender.
Class “interest in the image” had a distribution of attributes ‘yes’ consisting of 145
instances and ‘no’ consisting of 105 instances. The distribution of the class “interest in the
image” can be seen in Figure 8.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
356
Figure 8. Class of interest in the image.
Figure 9 shows that class “preferences” had a distribution of ‘visual’ attributes as many
as 180 instances, while ‘verbal’ attributes have as many as 70 instances.
Figure 9. Class of preferences.
Participants involved in this research then followed the pre-test to test the initial ability
level. The minimum value obtained by participants was 60, and the maximum value obtained is
95, where the mean value is 76.288, and the standard deviation is 8.184, as shown in Figure 10.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
357
Figure 10. Means of pre-test.
J48 Classifier Analysis
The J48 classifier is a data mining method that implements a C4.5 algorithm to build
a decision tree model. The decision tree model is created to form a classification model
(Bhuvaneswari, Prabaharan, & Subramaniyaswamy, 2015). The level of accuracy obtained in
this research is 66%, and the mean absolute error (MAE) is 0.3855. MAE serves to measure the
accuracy of predictions by averaging errors (the absolute value of errors). The analysis process
in this section uses WEKA data mining to form a classification model. The classification
process in WEKA data mining produces a confusion matrix. The confusion matrix is a method
for measuring classification performance. The classification system performance describes how
well the system classifies data. The confusion matrix can see the results of 2 lines. The first
line, “41 64” shows that there are (41 + 64) instances class self-efficacy ‘high‘ and all right are
classified as self-efficacy ‘high.‘ In the second line, “21 124” shows that there are (21 + 124)
instances class self-efficacy ‘low‘ and all are classified as self-efficacy ‘low,‘ as in Figure 11.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
358
Figure 11. Confusion matrix.
Data visualization Analysis
The research used data visualization with a decision tree and a linear projection approach
to obtain a classification model. Linear projection is a machine learning method that refers to
the number of populations from an instance. The linear projection method presents information
about statistical correlations, about the linearity of specific aggregate values. Learning
preferences of students have a relationship with student interest in images, while the level of
self-efficacy does not correlate with the visual-verbal preferences, as shown in Figure 12.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
359
Figure 12. Linear projection of data visualization obtained for the data class.
Figure 12 shows that male students have more interest in images when compared to
female students. The result is reinforced by the results that show that male students who have
visual preferences have more numbers compared to female students. Grouping in the decision
tree is divided based on the value of achievement on the results of the pre-test, consisting of two
groups, namely students with a value of> 85 and students with a value of ≤ 85. Students who
have pre-test> 85 have high self-efficacy. Whereas in students who have the results of pre-test ≤
85 are distinguished based on the students’ interest in the image, as shown in Figure 13.
Figure 13. The Decision tree of correlation learning outcomes with the results of
the pre-test and self-efficacy.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
360 Figure 13 shows that interest in images can affect the results of pre-test (> 80) if students
have high self-efficacy. So, it can be concluded that high self-efficacy directly affects the
achievement of the pre-test results, on the other hand, the interest in images and visual-verbal
preferences do not influence the achievement of the results of the pre-test. Students who have
visual preferences with interest in images can get good pre-test results if there is a high self-
efficacy factor.
Discussion
This research showed that participants who have a low level of self-efficacy are 58%, and
participants who have a high level of self-efficacy are 42%. However, the results of the research
showed that there is a correlation between the level of efficacy of visual-verbal preferences, as
indicated by the variation of the results obtained, as shown in Figure 14.
Figure 14. Overall accuracy of classifiers.

The decision tree method can present a correlation and classification model based on
the similarity of attributes of data instances. It is consistent with previous research that shows
that this method can help establish a classification model for experimental data (Kabakchiev
et al., 2017). The classification model classifies data instances with their respective attributes
(Demšar et al., 2013; Singh, Naveen, & Samota, 2013). The decision tree method in this study
can predict groups in each example where each sample is grouped according to the associated
agreement (results see Figure 13). It agrees with Apte and Weiss (1997), which state that
classifications in data mining are used to predict problems and group problems based on the
objectives to be used (Apte & Weiss, 1997). Decision tree involves the use of training sets to
build problem prediction models and classify input data (Singh et al., 2013). The classification
model was developed based on predictive algorithms by classifying populations into branches
consisting of root nodes, internal nodes, and leaf nodes (Yan-yan Song & Ying Lu, 2015).
An additional alternative method that can be used is the linear projection method. Linear
projection can display statistical projection information and describe the tendency of one class to
another class. The implementation of data mining methods, such as decision tree-classification,
and linear projection, are beneficial for measuring correlations between classes and attributes
of experimental data.
This research found that self-efficacy affected the results of the pre-test which agreed
with what was revealed by Abosede and Adesanya, who revealed that self-efficacy influences
performance and problem-solving abilities (Abosede & Adesanya, 2017). Other studies
also found results that engineering self-efficacy had a significant correlation with academic
achievement (Aleta, 2016).

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
Conclusions 361
Classification techniques in data mining methods designed to classify data instances aim
to build a classification model for experimental data. Classification forms a decision tree to
group data instances based on attribute attributes. The class results of the class form the model
according to the prediction of the problem presented. This research involved engineering self-
efficacy towards visual-verbal preferences about the results of the pre-test. There is a need for
analysis of experimental results such as self-efficacy towards the tendency toward visual-verbal.
This research proposes an alternative method to measure the correlation of self-efficacy with
a visual-verbal preferences approach called the J48 classifier technique and linear projection.
J48 classifier is an algorithm used to construct a decision tree with a statistical classifier,
while a linear projection presents information about the correlation of linearity to several
measured variables. The measurement results show that self-efficacy correlates with the results
of visual-verbal preferences. The proposed method can also be applied to the measurement of
correlation with more classes and large data instances in the database. This research found that
data mining methods, especially decision trees, were able to be used in analysing correlations
between several variables. The decision tree method can be used as an alternative method
besides the statistical method in measuring the correlation between variables. Further research
is expected to develop other data mining methods specifically for processing educational data.
Acknowledgments
This research is supported by the Domestic Postgraduate Education Scholarship (BPP-

DN) from the Ministry of Research, Technology, and Higher Education.
References
Abernethy, M. (2010). Data mining with WEKA, Part 2: Classification and clustering (No. 2). IBM
developer Works. Retrieved from ibm.com/developerWorks/.
Abosede, S. C., & Adesanya, A. O. (2017). Contributions of self-efficacy and problem-solving skills on
secretaries’ job performance in Ogun State public service, Nigeria. Journal of Education and
Practice, 8(11), 109–116.
Aleta, B. T. (2016). Engineering self-efficacy contributing to the academic performance of AMAIUB
engineering students: A qualitative investigation. Journal of Education and Practice, 7(27), 53–
61.
Apte, C., & Weiss, S. (1997). Data mining with decision trees and decision rules. Data Mining, 13. https://
doi.org/10.1016/S0167-739X(97)00021-6.
Bandura, A. (1994). Self-Efficacy. In I. V. S. Ramachaudran (Ed.), Encyclopedia of human behavior (Vol.
4, pp. 71–81). New York: Academic Press. https://doi.org/10.1002/9780470479216.corpsy0836.
Bandura, A. (1997). Self-efficacy: The exercise of control. Harvard Mental Health Letter (Vol. 13). New
York: W.H Freeman Company. https://doi.org/10.1007/SpringerReference_223312.
Bandura, A. (2006). Guide for constructing self-efficacy scales. In F. Pajares & T. Urdan (Eds.), Self-
Efficacy beliefs of adolescents (pp. 307–337). Greenwich: Information Age Publishing.
Bhuvaneswari, T., Prabaharan, S., & Subramaniyaswamy, V. (2015). An effective prediction analysis
using J48. ARPN Journal of Engineering and Applied Sciences, 10(8), 3474–3480. Retrieved
from www.arpnjournals.com.
Boswell, S. S. (2013). Undergraduates’ perceived knowledge, self-efficacy, and interest in social science
research. The Journal of Effective Teaching, 13(2), 48–57.
Carberry, A. R., Hee-Sun, L., & Ohland, M. W. (2010). Measuring engineering design self-efficacy.
Journal of Engineering Education, 99(1), 71–79. https://doi.org/10.1002/j.2168-9830.2010.
tb01043.x.
Cisco Systems. (2003). Cisco Networking Academy Program CCNA 1 and 2 Companion Guide.
Indianapolis: Cisco Press.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
362 Demšar, J., Curk, T., Erjavec, A., Hočevar, T., Milutinovič, M., Možina, M., … Zupan, B. (2013). Orange:
Data Mining Toolbox in Python. Journal of Machine Learning Research, 14, 23492353.
Hutchison, M. A., Follman, D. K., & Bodner, G. M. (2005). Shaping the self-efficacy beliefs of first-
year engineering students: What is the role we play? School of Engineering Education Graduate
Student Series, 12. Retrieved from http://docs.lib.purdue.edu/enegs/12.
Jordan, K. L., Amato-henderson, S., Sorby, S. A., & Donahue, T. L. H. (2011). Are there differences in
engineering self-efficacy between minority and majority students across academic levels? In New
Research & Trends for Minorities in Engineering (pp. 13–26). Vancouver: American Society for
Engineering Education. Retrieved from https://www.asee.org/public/conferences/1/papers/659/
download.
Kabakchiev, C., Kabakchieva, D., Garvanov, I., Behar, V., Kabakchiev, K., Rohling, H., … & Yarovoy,
A. (2017). Data mining classification of cars based on GPS shadows in Forward Scatter Radar
systems. In The 18th International Radar Symposium IRS 2017. Prague: DGON. https://doi.
org/10.23919/IRS.2017.8008217.
Kirby, J. R., Moore, P. J., & Schofield, N. J. (1988). Verbal and visual learning styles. Contemporary
Educational Psychology, 13(May 2014), 169–184. https://doi.org/10.1016/0361-
476X(88)90017-3.
Kurniawan, C., Setyosari, P., Kamdi, W., & Ulfa, S. (2018). Electrical engineering student learning
preferences modelled using k-means clustering. Global Journal of Engineering Education, 20(2),
140–145.
Marra, R., & Bogue, B. (2006). Women engineering students’ self efficacy – A longitudinal multi-
institution study. In Proceedings of the 2006 WEPAN Conference (pp. 67–78). Pittsburgh:
WEPAN-Women in Engineering Programs and Advocates Network. Retrieved from https://
journals.psu.edu/wepan/article/view/58479/58167.
Orange Data Mining. (2015). Linear Projection: A linear projection method with explorative data analysis.
Retrieved September 7, 2018, from https://docs.orange.biolab.si/3/visual-programming/widgets/
visualize/linearprojection.html.
Pektaş, Ş. T. (2013). Correlations between the visualizer/imager cognitive style and achievement in
digital modeling tasks. Procedia - Social and Behavioral Sciences, 116(2014), 5055. https://doi.
org/10.1016/j.sbspro.2014.01.1072.
Peterson, M. O. (2016). Schemes for integrating text and image in the science textbook: Effects on
comprehension and situational interest. International Journal of Environmental and Science
Education, 11(6), 1365–1385. https://doi.org/10.12973/ijese.2016.352a.
Plass, J. L., Chun, D. M., Mayer, R. E., & Leutner, D. (1998). Supporting visual and verbal learning
preferences in a second-language multimedia learning environment. Journal of Educational
Psychology, 90(1), 25–36. https://doi.org/10.1037/0022-0663.90.1.25.
Richardson, A. (1977). Verbalizer-Visualizer: A cognitive style dimension. Journal of Mental Imagery,
1, 109–126.
Schunk, D. H. (2003). Self-efficacy for reading and writing: Influence of modelling, goal setting, and self-
evaluation. Reading and Writing Quarterly, 19, 159–172.
Singh, D., Naveen, H., & Samota, J. (2013). Analysis of data mining classification with decision tree
technique. Global Journal of Computer Science and Technology, 13(13), 1–6.
Sudatha, I. G. W., Degeng, I. N. S., & Kamdi, W. (2018). The Effect of Visualization Type And Student
Spatial Abilities On Learning Achievement. Journal of Baltic Science Education, 17(4), 551–563.
Witten, I. H., Frank, E., Hall, M. A., & Pal, C. J. (2017). Data Mining: Practical machine learning tools
and techniques. (C. Kent, Ed.) (4th ed.). Cambridge: Morgan Kaufmann Publishers. https://doi.
org/10.1016/C2009-0-19715-5.
Wu, C.-S., Tsai, L., & Wang, P. (2011). Correlation between technological creativity, self- efficacy
and knowledge sharing among athletes. International Journal of Management and Marketing
Research, 4(2), 77–84.
Yan-yan Song, & Ying Lu. (2015). Decision tree methods: Applications for classification and prediction.
Shanghai Archives of Psychiatry, 27(2), 130–135. https://doi.org/10.11919/j.issn.1002-
0829.215044.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
Received: April 09, 2019 Accepted: June 10, 2019 363
Citra Kurniawan Student of Graduate School, State University of Malang, Malang, Indonesia.
(Corresponding author) E-mail: airakurniawan@gmail.com
Punaji Setyosari Professor at the Department of Educational Technology, State University of Malang,
Malang, Indonesia.
E-mail: punaji.setyosari.fip@um.ac.id
Waras Kamdi Professor at the Department of Mechanical Engineering Education,

State University of Malang, Malang, Indonesia.
E-mail: waras.ft@um.ac.id
Saida Ulfa Assistant Professor at the Department of Mechanical Engineering Education, State
University of Malang, Malang, Indonesia.
E-mail: saida.ulfa.fip@um.ac.id

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
364
THE FORMATIVE RESEARCH OF
ECUADORIAN UNIVERSITY
TEACHING STAFF
Derling Mendoza Velazco
National University of Education, Ecuador
E-mail: derling.mendoza@unae.edu.ec
Irma Abrigo Córdova, Janela Romero Chávez, Fredy Cueva Bravo

International University of Ecuador, Ecuador
E-mail: irabrigoco@uide.edu.ec, karomeroch@uide.edu.ec, frcuevabr@uide.edu.ec
Magda Francisca Cejas

University of the Armed Forces, Ecuador
E-mail: mfcejas@espe.edu.ec
Abstract
University research is the way in which lecturers can perceive, analyze and be illustrated about unknown
topics or something that has been omitted and that is why it is important and is the key in the academic
researcher´s life, the gathering of information, the analysis and the search for a solution to an existing
or detected problem. This research includes three phases: the first, where the search, collection and data
analysis is proposed; the second, where models of teacher formation and theorization are included and the
third, where the final idea of formation of university researchers and lecturers is proposed. This research
was developed from a qualitative perspective, since it is implemented in an epistemic conceptualization
that corresponds to its nature, highlighting the interest of deepening the intuition of the object research.
As key informants, the participation of 15 (fifteen) lecturers from different universities of Ecuador is
highlighted. The constant comparison of theories and grounded theory was applied for the analysis of
the subtracted data, in order to make a theoretical approach through the emerging categories and sub-
categories, through the operating software Atlas.ti version 8; with the aim to contribute to the theoretical
and methodological basis. The subcategories were derived: professional teaching function and novel
teaching profile. The conclusions mention the idea or model of formation for higher education teaching
staff. It also mentions the importance of improving the formation of lecturers through scientific research
to promote knowledge and strategies that require the need to theorize, produce and increase the level of
thinking.
Keywords: continuous research, complex thinking, educational innovation, educational theory, formation
of teaching staff, university education.
Introduction
In the educational practice, the lecturer’s participation in higher education is a dimension

of special interest. The praxis is a complex and systemic action. It is demonstrated with a
gallery of events that sometimes is far from strengthening the processes. In education there
are strong barriers that are created by repetition. The fact in a culture is assumed as the norm
in the organization and is impregnated by the novice teaching staff. The daily routine of the
formation process becomes cyclical: moments with multiple nuances of weakness that become
operational barriers in the professionalization of the students. What the teaching staff does or

Derling MENDOZA VELAZCO, Irma ABRIGO CÓRDOVA, Janela ROMERO CHÁVEZ, Fredy CUEVA BRAVO, Magda FRANCISCA CEJAS.
The formative research of Ecuadorian university teaching staff
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
does not do, their successes or failures in educational spaces are translated into levels of student 365
formation (Cid, Pérez, & Zabalza, 2009).
It is necessary for the teaching staff to take possession of knowledge and apply it in their
work, for the very nature of their practice. The didactic offers important contributions to the
andragogy diversity in university education. There are important approaches with actions close
to innovation, deepening and transforming the teaching process in relation to learning from the
reality of the classroom and the events of daily formation. This situation requires processes of
comprehension, creation, transformation and orientation of the educational practice (Martínez,
Yániz, & Villardón, 2018).
Research Focus
The Theory of complex thinking
According to Peñaranda (2016) complexity is a network (complex: what is the network

as a whole) of inseparable heterogeneous components associated: it presents the paradox of
the one and the multiple. Complexity is, in fact, the events, actions, interactions, retroactions,
determinations, chances, which constitute our phenomenological world. Morín (1998)
emphasized that complex thinking has its essence in the lost tradition of approaching the world
and the human being from a hermeneutic point of view, that is, interpretive and comprehensive.
For the mentioned researcher, the complex thought is first of all a thought that relates and is
conceived as a form of rationality in the approach of the world and of the human being. It
consists of the union of the parts with the whole, through the establishment of relations between
them, taking into account their differences.
The complex thought unites, gathers, relates and approaches the processes in their
constant dynamism and change. Likewise, its difficulty of action lies in the fact that it must
face the interweaving, the solidarity of the phenomena with each other, the fog, the uncertainty
and the contradiction (Morín, 1999). Until the first half of the 20th century, most sciences
had specialization and abstraction as a mode of knowledge; there was also the reduction of
knowledge of a whole to the knowledge of the parties. The key concept was determinism, the
application of mathematical logic to the problems of what was experienced and in the social life
(Itzel, 2018). From there, that a reform of thoughts becomes possible, it is separated and has to
be replaced by the thought that distinguishes and unites. The disjunctive and reductive thought
must be replaced by a “complex thought” (Morín, 1999). This reform of thoughts is addressed
and has a quite clear implication with education. Educational systems must begin this process
of reform that transcends from simplicity to complexity (Maldonado & Gómez, 2011).
Consequently, the theory of complex thinking is relevant to the research, because
it analyzes the process of formation of subjects who learn (university teaching staff). Both
formation and society are a complex issue, which implies a multiplicity of factors in relation that
make it a dynamic, varied and diverse issue. Rosario (2018) explained that in human formation
there are multiple factors involved in different systemic relations: culture, society, institutions,
community, subjects that learn. Therefore, the human formation of the lecturer cannot be seen
from a single aspect. The formation must be seen from the relations of factors articulated from
the subject, the individual and the social dimension.
University teaching staff require complex thoughts in education, according to Rodríguez,
Gil, and García, (1999). Diversity and mobility of so much information is difficult to harmonize,
because it jumps from simple to complex. Complexity informs reality and the resulting
educational process is another. This process depends on many vicissitudes and diverse elements
that disorient educational work.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
366 The importance that is assumed in the research is based on complexity and complementarity.
According to Morín (1998), “the aspiration to the totality is an aspiration to the truth. The
recognition of the impossibility of the totality is a very important truth, for which the totality is,
at the same time, the truth and not the truth” (p.137). The orientation of the andragogic practice
carried out by university teaching staff, due to its practice, has little real reflection based on the
elements that make up this complexity.
As a result, it is required a type of knowledge to understand that parties depend on the
whole and vice versa. The present research was based on a complex theory that does not isolate
phenomena, but integrates them into a single set (Ramírez, 2013). The andragogic practice
should not focus only on an encyclopedia that transmits the cognitive where Machado (2018)
claimed that true higher education is a meeting between human beings where life itself is the
main actor.
It is necessary to question the traditional instances and to look for the complex perspective,
since the contents treated in class must aim at the transformation of the educational subject,
these are concretely presented with a certain degree of value. Consequently, each educational
content must be proposed not only as objective knowledge, but also as the freedom of the
subject to appreciate it as valuable to himself (Veytia, 2014). This brings with it the idea of
change, since in the sciences and in the social disciplines the conceptions of man, knowledge,
education and formation have to be continuously formed and transformed. The theories and
concepts are transitory according to Chacón, (2012), educational conceptions and pedagogical
practices do not anticipate or do not evolve simultaneously because they lose their meaning and
reason of being.
Changes in teaching staff’ thinking for their formation are a legitimate and necessary
process that must be operated from the moment they are required, and not wait for that change to
come from the highest governmental and institutional spheres. In the field of teacher formation,
a rationality like that of complex thinking is required, in keeping with the constant and changing
world. Martínez, (2018) and Ramírez, Suárez and Torres, (2017), emphasized that each country
requires innovative subjects that adapt easily to changes with a flexible attitude, with capability
to establish professional relations between processes and systems, teaching staff must have the
ability to approach new possibilities, people with qualities present in complex thinking, which
approach reality in all its complexity, to understand it and potentially transform it.
Orientation for teaching staff
Orientation is understood as the permanent process of acquiring, structuring and

restructuring behaviors (knowledge, skills, values) for the performance of a certain function.
In the case of educational action, orientation becomes an improvement of the teaching staff,
where Sanchez (2017) defined orientation as a development in university teaching staff with
the commitment, vocation and application of guidance in teaching. During the university
career leading to the diploma and later on updating formation courses and graduate, the
lecturer acquires and consolidates knowledge and specialized skills. Due to this, Jimenez and
Montecinos (2018) established that higher education teacher’s formation high social relevance
is a dynamic, permanent and continuous process, closely linked to the teaching practice.
The university lecturer is a professional whose daily practice is full of unknowns to
which he does not respond with preconceived formulas. Lecturers structure their knowledge,
skills and values so that they can solve the problems that arise in the educational space on a
daily basis (Iglesias, Lozano, & Roldán, 2018). In other words, the reflection about their work
in the institution allows the learning got by the orientation in a formal and informal way to take
concrete forms and acquire a meaning that ensures its social transcendence.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
In a modern cultural perspective, the teaching staff is the epicenter where action and 367
programming are born. Currently, it focuses both on the transmission of values and on the
stimulation of the student’s personal development, so that the latter can be a culture builder
and not a continuator of it (Tejada, 2008). In this way, it is perceived that the teaching staff’s
continuous formation relays on their preparation and professional emancipation so that, through
a reflective criticism, lecturers can develop an effective teaching style that promotes meaningful
learning in students. In this way, innovative action thinking is achieved by working as a team
with colleagues to develop a common research project. Montero and Gewerc, (2018) stated that
today’s society imposes a challenge for teaching staff, where the path for lecturers whose work
and mission are related to the demands of their social context.
Teaching staff formation is a complex, critical and constructivist perspective. It implies
as a strategic objective, an investigative conception of the teaching work, for the integration
and reconstruction of meanings coming from diverse epistemological sources. This includes
phenomenological knowledge, ethical and ideological, disciplinary, didactic, socio-political
related to school content and meta-disciplinary, as well as personal conceptions, according to
what was expressed by Tejedor (2018) and Calle (2018).
Due to this, teaching staff formation allows the necessary conceptual tools so that the
environment is analyzed and relevant knowledge to social needs is built. Imbernón, (2017)
claimed that during the high education teacher orientation; the subject must become a researcher
of the needs and the potentials of the community. The teaching staff must be able to work as a
team in the realization of a shared task, he must reflect on its practice to transform or modify
it, taking into account the educational knowledge he has, the set of knowledge that must be re-
contextualized and the tools it has available to carry out its educational function.
Research Problem
The formation of the teaching staff is not a checking of the teaching formulas. Nor
should it be formation in specific disciplines. It has to be the space that receives the higher
education teacher’s concern to transcend through reflection, to clarify his position regarding
educational issues. The role must be the social dynamic to understand the world according to
Enkvist, (2016). Therefore, there must be a space where the teacher in formation or in service
can become aware of himself, aware of his work, of the world and confirm his commitment to
his students and their learning process.
After analyzing different theoretical sources, the problem in the research arises because,
Ecuadorian education does not give priority to the union between research and formation. In
this way, researchers need to interpret a theoretical approach that demonstrated the importance
of continuous research.
Research Aim
From there, the importance of current research aimed to make a theoretical approach to
formation and continuous innovation of the Ecuadorian university teaching staff. For this, the
research was developed based on qualitative paradigm. The purpose of research was to find a
relevant data of the situations that occur during the development of the educational practice.
The continuous theoretical comparative method and the application of the grounded theory
were applied. Emphasis was placed on the individual and subjective aspects of the experience, a
method suggested by Hernández, (2016). The phenomena were studied as they are experienced
and perceived by the teaching staff of the research, describing the meaning of the experiences
lived by the educational teaching staff who perform andragogic practices in the context of
higher education. The phenomenology of everyday life was considered, because each social
fact is only understandable in its context to intuit and give meaning, through the opinions of
the informants.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
General Background
This research was based on the qualitative interpretive paradigm. Yuni and Urbano (2005)
pointed out that this qualitative interpretive paradigm “obeys the philosophical conception from
which it is nourished, and which provides the foundations on what is real and the possibilities
of knowing it”. (p.107) The researchers carried out the data collection in the universities where
the informants perform their daily teaching tasks. It was made with the intention of being able
to observe directly without the presence and influence of other elements that could distort the
information provided.
Due to the nature of the qualitative research, interviews were conducted, recommended
by Calduch (2014) to obtain information from the informants at the time of expressing their
opinions on the issues raised for the research. Each teaching staff answered in a pleasant and
measured way to the topics, using the knowledge they possess according to their area of academic
preparation. Once the required information was compiled, the researchers proceeded to show
their own interpretations, taking into account the opinions obtained. The time determined for
the total development of the investigation was six months. The research started on October 1st,
2018, until the 29th of March 2019.
Informants
For the selection of the informants of this research, the researchers took the criteria
recommended by Mendieta (2015), where there is no need for a statistical selection because it
is a study with a qualitative approach. After an analysis of the best universities working in the
city of Quito, Ecuador. Eight (8) universities were selected according to different criteria. First,
for the amount of research and scientific publications published. Second, because of the number
of students and teaching staff enrollment. Finally, for the diversity of careers offered by each
of the universities. The universities chosen were San Francisco University of Quito, Pontifical
Catholic University of Ecuador, University of the Armed Forces, National polytechnic school,
Central University of Ecuador, University of the Americas, National University of Education,
and UTE University.
One (1) letter of invitation was written for each chosen university, requesting the
participation of 2 (two) lecturers who possessed the following qualities: experience in scientific
research and possession of an academic degree superior or equal to the master’s degree. It was
also written that the lecturer would be interviewed in no more than 20 (twenty) minutes. In
1 (one) university only one professor did not agree to participate. Thus, the total number of
participants was 15 (fifteen) university informants.
Techniques and Instruments
In order to collect the necessary information that would respond to the emerging concerns
of this research about educational research, the collection of information was carried out in a
first phase. The researchers applied a semi-structured interview to the informants (view table 1).
Roth (2016) pointed out that this type of qualitative interviews should be simple, understandable
and opened, without pre-established categorizations. Based on this, the informants expressed in
the best way the data required by the researchers without any influence from the perspectives at
the time of performing the analysis.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
Table 1. Interview questions applied to research informants. 369
Universidad UTE Data:
Interview No.: Informant:
Question´s
1. How can teachers update themselves professionally?
2. Does the university allow you continuous formation?
3. Is research development important?
The interviews were recorded in Moving Picture Experts Group (MPEG) format. For
the transcription of the audios the researchers used Dragon Natural Speaking software version
12. Later, the texts were coded. The coding complies with the researcher’s ethical standards for
the production and dissemination of data, a method recommended by Márquez (2001). Each
testimony was presented as: First informant interview (I.1). Sequentially, every last informant
was coded (I.15).
Subsequently, it was proceeded to analyze the data obtained by contrasting the
information. A contrast was made between the data obtained by the informants and the
theoretical foundations investigated. The operating system of qualitative analysis Atlas.ti
current version, recommended by Charmaz (2010), for qualitative research was run. From the
contrast or triangulation of the information, categorizations came out and were analyzed by
the researchers. Emerging categories, theoretical underpinnings, and researcher analyses were
triangulated. In this way, the theoretical approach was established that gave an answer to the
proposed objective. This research method was recommended by Kohlbacher (2006) to make a
theoretical approach.
Research Results
Category Educational Innovation
Figure 1 shows the category as innovation educational. The results were described
through a structural network.
Figure 1. Structural network of the educational innovation category issued by

the Atlas.ti operating system.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
370 Based on the testimonies, the subcategory “Social Adaptation” emerges. For the
interviewees, “adaptation” has always prevailed as a topic of interest imposed by the university.
The informant (I.2) expressed “in my university they always want there to be a link with
society”, in the same way for (I.5), “A teacher must adapt to changes in society as well as to
needs”.
Most of the narratives presented by the informants consider that educational innovation
is an adaptation to the demands of today’s society. It is necessary for teaching staff to be
prepared to train a student who knows how to respond to social demands, which are advancing
more rapidly every day. In agreement on the theme of adaptation according to (I.12), “is of
sum importance... it is necessary to reflect on teaching practice in order to modify that which
is no longer necessary”, in this way, according to the interviewee, educational reflection
makes it possible to speed up changes in the search for improvement. But for (I.14), there
is “methodological innovation... supposes a motivation for the teaching staff since it implies
facing new challenges”, these challenges are presented as the subcategory “Difficulties or
problems”. Also, according to (I.15) “Innovation must be an essential part, if we do not want
to continue giving master classes where the student does not feel protagonist of it”. Motivation
and creativity are fundamental axes of educational innovation, as an aspect highlighted in the
manifestos, where teaching staff should not fall into routine and therefore should motivate
students.
In the narratives, it was identified that educational innovation is related to sub-codes
that emerge from didactic implementation. According to (I.1) “I should have updated myself
for my students, because, I use a lot of visual material to make learning more meaningful”.
Other significant aspects identified in the informants’ manifestos is the development of methods
used to innovate in the classroom. (I.3) said, “I learned to use the materials provided by digital
networks and many materials I have found on the Internet”. The acquisition of new methods
is easy to adapt for teaching staff who have their work motivation. According to (I.6) there
are emotional obstacles when he said, “sometimes I am afraid to ask, but I have to go to my
classmates for advice on new methods or I use formation innovation courses”. The fear is
expressed of being catalogued as a teaching staff not updated in the use of digital tools. For this
reason, informants are forced to resort to personal and professional updating.
The informants reported a meaningful knowledge, so stressing that learning and
formation are part of innovation in the classroom. It also depends on the shared experiences of
other colleagues, especially by the working groups they have formed. Among the difficulties
or problems raised is spontaneous learning. For (I.10), “the best way of updating is to design a
parallel formation program, but the university considers that we are all prepared”, simultaneously
according to (I.8), “the digital whiteboard is an attractive innovation, I should have learned
spontaneously, because we were not oriented to the use of these resources”. For the informants,
the need for an orientation design or project is fundamental.
As a factor of difficulty time emerges, manifested by (I.11), “basically time... passes
very quickly and does not provide for learning and putting into practice what I would like”. The
same is true for (I.6), which states: “I am completely in favor of updating teaching staff, but
although family life and such formation are not very compatible”. According to the teaching
staff interviewed, the universities invest money in material resources, but it is not orientation
programs or formative updating, considered as a difficulty to develop innovations in the
activities.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
Category Curricular Aspect 371
Figure 2. Structural network of the curricular aspect category issued by the

Atlas.ti operating system.
Figure 2 (two) demonstrates the structural network of the category “curricular aspects”.
This category is supported by the sub-categories “scarce reflection and “formation models
and approaches”. The second category is based on key testimonies from teacher formation
universities and universities where teaching staff work. For (I.4), “the university requires
personnel with a doctorate degree, but when I request time to begin enrollment, I was told that
I did not have one”. According to the informant (I.9), “curricular changes in current Ecuadorian
higher education do not allow for a teaching staff with a master’s degree; we must all have a
PhD”. The participants manifest the need to carry out academic studies of professional superior,
as a requirement of the institution where they work.
On the contrary, for the informant (I.12), who stated that “the university always forces
us to carry out formation courses” and for (I.13) who stressed, “When we have vacations or
weekends we are summoned for orientation workshops, oh is our time respected”. The informants
highlight in a not very pleasant way the university invitations to the constant update. Considered
general updating workshops and those that are offered on a mandatory basis. The free time is
not valued or respected by the institutions according to the teaching staff. The informant (I.15),
who attends these formation courses states, “I have to go to have a good curriculum”, also
(I.7) stressed “we have to take advantage of these seminars, they are the help of our university,
because personally they have a high price”. Two aspects of formative reception are presented in
this way: the teaching staff who accept and participate in the formation offered by the university
and those who reject it. From the point of view (I.5), confessed “I start to think and say to
myself, if we do not have updating and formation. What would become of us?
An evaluation of the effectiveness of the courses or workshops applied by the universities
is established, according to (I.1) when expressed “Our university institute facilitates the
formation, the problem is in the lack of content”. The lack of clarity or purposes of these are not
stipulated from the beginning of the courses, which cause discomfort for the activities carried
out. The discomfort of necessary courses or not, is exposed by (I.3), when expressed “the last
workshop we had, consider it useless, if we had been asked what we needed, everything would
be different”. The teaching staff perception of the opportunities offered by universities for
continuing education is not very well accepted. Demonstrating that participation or autonomy
is not promoted, the development of capacities for reflection, innovation, research, especially
debate, is not allowed.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
372 For (I.14). “In this way, informants can identify the needs and the type of skills required,
responding that they negatively justify the university’s obligation to only comply with a
curricular requirement”. Similarly, for (I.10), “the department of human talent does not request
a consultation on our formation, it does so in an authoritarian manner”, the arguments expressed
demonstrate that the courses do not consider the context or the social reality in which the
university educational practice is developed, since this influences it in a decisive manner. From
the perspective of teaching staff, opinion is not taken into account in the design of study plans
and programs.
According to (I.13), “I believe that these opportunities provided by the formation
institution are ambiguous, because they only think of the evaluation of the ministry of higher
education. For the teaching staff, there is no congruence between what is proposed and what is
done on the part of the university authorities. For the lecturers, their main interest is to safeguard
the academic evaluation score. The evaluations are described by (I.11) when he stated that “I
only hope that this year, when carrying out evaluations and categorizations, they do not make
radical changes in the programs like the previous time”. The participants emphasize that the
programs can be improved, but adaptation requires a new curricular orientation design. Other
informants, as (I.8) expressed “when evaluations are carried out, there are modifications in the
system, everything deteriorates and we have to learn again”, so it was emphasized that constant
changing of university programs, the result of implicit evaluations, then more updating is needed
to know the legal support of the existing reforms. Some informants stated that everything arises
from incompetent policies and, therefore, not even the system itself has a clear idea of what
type of teaching staff is in demand in Ecuadorian society.
Category Teaching Staff Formation
The category “teaching staff formation” is generated in relation to cause and effect
(see image 3). The relationship between teaching staff formation, consecutive changes due
to educational innovation and curricular changes. In this scheme, formation is an external
mechanism that acts outside of practice. This practice is stigmatized and devalued, for the
informant (I.12), that said “in spite of having updated the curriculum, the practice, activities and
strategies in force in my university are traditional”. Emphasis is placed on the informant’s oral
presentation, that the frontal teaching methods are of an academic type.
Universities of the traditional type propose academic models of a pedagogical order,
however, for (I.11), “I think that the university has a great influence on the formation of a
teacher”, just as for (I.2), “in the university where I studied my pre-grade degree we were not
infused with research”. These testimonies represented that the practices of teaching staff face
diverse situations and problems. In order to solve them, they turn to their theoretical knowledge
as well as to their incipient practical skills, to articulate the demands and characteristics of the
university context. The lack of an educational-research system generates a negative impact
for university graduates who expose, (I.1), “In truth, in undergraduate they always gave me
practical lessons, but in research they were only theoretical”. In this way, the sub-category
“new profile of the teaching staff” emerges, where the new university lecturers must build their
“knowledge of the trade”. This arises from the interaction between theoretical knowledge and
direct experience in the university environment with the students. The intellectual entry profile
of educational praxis is constituted by (I.15), when stated “I learned by teaching my students”.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
373
Figure 3. Structural network of the teaching staff formation category issued by

the Atlas.ti operating system.
Due to limited practical experience, beginning teaching staff rely on formally acquired
models in their formation to understand, analyze and cope with practice situations, or on their
first learning matrices. The possibility of recognizing the components that interact in a given
educational situation, depends in part on whether these elements have been previously presented
to them during their formation. In this sense, the formation of university teaching staff is
presented as a preparation and professional emancipation. This preparation is considered as the
profiles of the teaching staff, according to (I.1). “We have to increase the capacity to elaborate,
criticize, reflect and create an effective teaching style”. The teaching and learning styles are
upgraded thanks to research according to (I.3), when expressing, “by doing good research we
can give meaning to learning, change, strengthen and improve our teaching system”. In the same
way, the informant emphasizes that meaningful learning must be promoted in the students and
an innovative reflection-action must be achieved, working in team with colleagues to develop
a common educational project.
Faced with these assertions, according to (I.10) “teaching staff not only continue to be
trained, they must also reflect on whether research is important or not”, the results issued by the
informants demonstrated the need for a reflective and innovative teaching staff, an ideal lecturer
whose formation is developed in the context of his or her research work. With regard to research
work, (I.6) stated that “teaching staff formation is a field of knowledge and research focused on
the study of the processes by which teaching staff learn and develop their professional skills for
the formation of other subjects”. For this, according to the informant, teaching staff formation
is fundamental, through a model focused on the immediate context of the work, where they are
trained to analyze the educational system and develop their practice as innovation work.
In the aspect of educational work, lecturers expressed the fear of change, according
to (I.14) when indicated “I see that new and old professionals are always differentiated by
practice and evaluation, it would be comforting to create a project for the exchange of ideas
or strategies. Thus, the informants expressed the timely need for innovation. The informant
raised a new model of continuous professional formation in education. This formation model
focuses on the recognition that is experienced and novice teachers provide prior knowledge and
experiences. These experiences that are put into play in new learning situations are: knowledge
is also considered as an entry profile.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
374 As for the testimonies of informant on the importance of research, for (I. 3), “research
application is required, but the coordination of research only requires projects...they do not share
their methodological design strategies, they have material and they do not cooperate”. From
this point of view, there is a need for an organizational change in the distribution of educational
research models that facilitate teaching staff updating, that is, access to comprehensive and
participatory formation. These actions should be supported by a reflection on the daily work of
teaching staff, since the lack of research models is a weakness expressed by informants.
The study projects requested by the university coordinators according to (I.5), “require
a technical methodological character”. Based on the diversity of elements that make up the
research models, the formation of the Ecuadorian teaching staff obeys fundamentals associated
with systems theory. According to (I.13) “when developing a research project, I am always
asked to make changes, which I have to fix or correct, but I am never told what to do”. This is
established with a logical and pragmatic approach, in which many teaching staff universities
expressed their reluctance to become academically professional because they are considered
to have absolute knowledge, especially in the technical field, but not in the educational field.
For the purposes to be promoted professional formation, according to (I.9), he stated “my
colleagues do not accept a postgraduate degree, for fear of designing a doctoral thesis”. From
a critical point of view, there is a need for professional incentive and intellectual promotion by
universities. In the same way (I.4) he stated, “I can professionalize myself, cover my expenses,
I would do all academic development, everything but thesis”, there is a latent fear to carry out
a research, project, research, degree work or doctoral thesis, which hinders the objectives of
the higher education system, under which the commitment of professional formation must be
assumed, in order to address the learning needs that educational practice demands.
Discussion
The purpose of research was to determine the different contexts that make up the formation
and continuous innovation of Ecuadorian university teaching staff. With the theoretical support
of different researchers who support the complex educational thought as an essential model
in the educational role. This model makes it possible to mobilize thought, generate reflective
actions and produce spaces of multicultural and pluralist convergence.
Teaching staff formation becomes much more pertinent when subjects reflect on their
existing and necessary capacities (Escobar & Escobar 2016; Calvo, 2019), in the same way with
the application of meaningful learning. Developing research as a theoretical construct between
formation and innovation, however it is underlying several theories that are of a domain of the
faculty at various levels according to Contreras, (2016). The theoretical contribution mentioned
above and the participation of 15 (fifteen) informants made it possible to identify the three
categories that respond to the objective set at the beginning of the research.
In the first place, the category “Educational Innovation”, considered by teaching staff
as a constant update in their formation to adapt to changes in society (Iglesias, Lozano &
Roldán 2018). Likewise, the participants stated that innovation in their universities is a great
opportunity for professional improvement. The narratives extracted from the participants show
the need for formation through courses offered by the university administration. On the other
hand, other informants prefer to acquire strategies through self-learning.
After analyzing the textual testimonies that constitute the category “curricular aspects”,
it was possible to determine that the university and curricular evaluations are not disseminated
from a methodological or investigative vision of advisory or participatory depth, only from a
political aspect that is compliant with the norms or laws pre-established (Freire et al., 2018). All
the emergent changes affect in a considerate way the formation of the university teaching staff.
The models and approaches to formation are still based on abstract paradigms with knowledge
that changes rapidly and in increasingly diverse and difficult contexts.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
Finally, the third category was “teaching staff formation”. According to the results 375
presented by the informants, it was revealed that there were barriers that prevented professional
formation. Like the informants for García, Cerdas and Torres (2018), there must be an effective
mediation with the different educational agents, which allows the development of a formation
process that corresponds to the important social work that takes place in the universities. For
this, it is essential that, at a general level, universities introduce a process of reflection, in
order to approach a real knowledge of this formation task. Teaching staff formation is part of
the research. This is the starting point of a profound change with strategic characteristics that
allow an epistemological and methodological reconstruction in the formation of each subject,
whose role is formation through research. Therefore, the Ecuadorian universities of teaching
staff formation in their model of formative philosophy must instill in their curriculum the
requirements of research, where continuous research is established from the beginning of their
future role.
Conclusions
It is established a theoretical approach based on the categories described in the previous

section, where Ecuadorian university education requires changes and innovative adaptations
on higher education teachers. The curricular models are the product of constant evaluations
that slow down the professionalization of the teaching staff. These adaptations and emerging
changes require an extra-academic effort on the part of the faculty for their understanding
and application. In this way, a model of teaching staff formation is required that effectively
contributes to the optimization of research practice and knowledge.
The investigative thought is a form of knowledge or skill derived from the university
educational praxis. Their formation is consolidated with knowledge that is confirmed or
restructured as a result of the experiences that take place in the educational context and in the
daily work of education. In this way, the experience should be contributed with an accumulation
of events that strengthen the professional maturity of the teaching staff and progressively their
formation.
Ecuadorian universities should include research in conjunction with educational theories
and academic philosophy. So, there is a systematic search for knowledge, in order to induce
solutions to different problems. By introducing research as a formation approach, this becomes
a standard practice from the andragogical point of view. To do this, the research allows lecturers
to approach methodologically different realities of the educational field. The continuous research
on the teaching staff is one of the pillars of significant re-orientation, allowing to conceptualize
with a theoretical-practical approach, the existence of weaknesses that can be improved with
the research of formative type.
References
Calduch, R. (2014). Métodos y técnicas de investigación internacional [International research methods

and techniques]. Madrid: Universidad Complutense de Madrid.
Calle, L. (2018). Educación Superior: la alfabetización en géneros discursivos [Higher education: Literacy
in discursive generes]. Educação & Realidade, 43(2), 629-651. doi: 10.1590/2175-623665114.
Calvo, M. (2019). Pensamiento complejo y transdisciplina [Complex thinking and transdiscipline].
Sophia, colección de Filosofía de la Educación, 26(1), 307-326. doi: http://doi.org/10.17163/
soph.n26.2019.09.
Chacón, A. (2012). Andragogy as a discipline propelling knowledge in higher education. Educare, 16(1),
15-26. Retrieved from: http://www.revistas.una.ac.cr/index.php/EDUCARE/article/view/3729.
Charmaz, K. (2010). Constructing Grounded Theory. A practicel guide through qualitative analysis.
Washington DC: Sage.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
376 Cid, A., Pérez, A., & Zabalza, M. (2009). Las prácticas de enseñanza declaradas de los “mejores
profesores” de la universidad de Vigo [The declared teaching practices of the “best teaching staff”
at the University of Vigo]. Revista Electrónica de Investigación y Evaluación Educativa, 15(2),
1-29. Retrieved from: http://www.uv.es/RELIEVE/v15n2/RELIEVEv15n2_7.htm.
Contreras, F. (2016). El aprendizaje significativo y su relación con otras estrategias [Meaningful learning
and its relationshipwith other strategies]. Horizonte de la Ciencia, 6(19), 130-140. doi: 10.26490/
uncp.horizonteciencia.2016.10.210.
Enkvist, I. (2016). El complejo oficio del profesor [The complex office of the teacher]. Madrid: Fineo
Editorial.
Escobar, R., & Escobar, M. (2016). La relación entre el pensamiento complejo, la educación y la pedagogía
[Relation among complex thinking, education and pedagogy]. Administración y Desarrollo, 46(1),
88-99. doi: https://doi.org/10.22431/25005227.62.
Freire, J., Páez, M., Núñez, M., Narváez, M. & Infante, R. (2018). El diseño curricular, una herramienta
para el logro educativo [Curriculum design, a tool for educational Achievement]. Revista de
Comunicación de la SEECI, 45(15), 75-86. doi: http://doi.org/10.15198/seeci.2018.45.75-86.
García, J., Cerdas, V. & Torres, N. (2018). Curriculum management in Costa Rican Schools: An analysis
from teaching staff and administration team’s perception. Educare Electronic Journal, 22(1),
1-28. doi: http://dx.doi.org/10.15359/ree.22-1.11.
Hernández, R. (2016). Metodología de la investigación [Research Methodology]. México: McGraw Hill.
Iglesias, M., Lozano, I. & Roldán, I. (2018). La calidad e innovación educativa en la formación continua
docente: un estudio cualitativo en dos centros educativos [Educational quality and innovation
in continuous teacher formation: a qualitative study in two schools]. Revista Iberoamericana de
Educación, 77(1), 13-34. Retrieved From: https://rieoei.org/RIE/article/download/3090/3952/.
Imbernón, F. (2017). Ser docente en una sociedad compleja. La difícil tarea de enseñar [Being a teacher
in a complex society. The Difficult Task of Teaching]. Barcelona: Graó.
Itzel, Á. (2018). Enseñanza y aprendizaje en el siglo XXI Metas, políticas educativas y currículo en seis
países [Teaching and learning in the 21st century. Metas, políticas educativas y currículo en seis
países, by Fernando M. Reimers and Connie K. Chung (editors)]. Perfiles Educativos, 11(159),
212-217. Retrieved from: http://perfileseducativos.unam.mx/iisue_pe/index.php/perfiles/article/
view/58776/51594.
Jiménez, F. and Montecinos, C. (2018). Diversity, management models and initial teacher formation:
formative challenges from a social justice perspective. Revista Brasileira de Educação,
23(e230005), 1-21. doi: http://dx.doi.org/10.1590/s1413-24782018230005.
Kohlbacher, F. (2006). The use of qualitative content analysis in case study research. Forum Qualitative
Sozialforschung. Forum: Qualitative Social Research, 7(1). doi: http://dx.doi.org/10.17169/fqs-
7.1.75
Machado, J. (2018). El modelo andragógico: una concepción epistémica para los estudios de prosecución
en las Universidades politécnicas territoriales [The andragogical model: an epistemic conception
for further studies in territorial polytechnic universities]. Centro de investigación y estudios
gerenciales, 32, 36-56. Retrived from: http://www.grupocieg.org/archivos_revista/Ed.%20
32(36-56)-Machado%20Jos%C3%A9_articulo_id369.pdf.
Maldonado, C. & Gómez, N. (2011). El Mundo de las Ciencias de la Complejidad: Una Investigación
sobre Qué Son, Su Desarrollo y Sus Posibilidades [The world of complexity sciences: An
investigation into what they are, their development and their possibilities]. Bogotá: Editorial
Universidad del Rosario.
Martínez, D. (2018). Evaluar el pensamiento crítico en Educación para la Ciudadanía: Propuesta para
contextos masificados. Proposal for Mass Contexts [Evaluate critical thinking in Education for
Citizenship]. Didacticae, 3, 131-144. doi: 10.1344/did.2018.3.131-144.
Martínez, M., Yániz, C. & Villardón, L. (2018). Self-evaluation and teaching reflection for the
improvement of the professional competence of teaching staff in the knowledge society. Revista
de Educación a Distancia, 56(10), 2-30.
Márquez, A. (2001). La ética del investigador en la producción y difusión del conocimiento científico [The
ethic of the researcher in the production and diffusion of scientific knowledge]. Revista Venezolana
de Gerencia, 6(16), 632-650. Retrieved from: https://www.redalyc.org/pdf/290/29061608.pdf.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
Mendieta, G. (2015). Informantes y muestreo en investigación cualitativa [Informants and sampling in 377
qualitative research]. Investigaciones Andina, 17, 1148-1150. Retrieved from: http://www.redalyc.
org/pdf/2390/239035878001.pdf.
Mendoza, D. (2018). Estrategias de enseñanza y su efectividad en los procesos de aprendizaje en los
estudiantes de turismo de la Universidad Iberoamericana de Ecuador [Teaching strategies and
their effectiveness in the processes of learning in the students of tourism of the Iberoamerican
University of Ecuador]. Espacios, 39(43), 25-39. Retrieved from: http://www.revistaespacios.
com/a18v39n43/a18v39n43p25.pdf.
Montero, L. and Gewerc, A. (2018). La profesión docente en la sociedad del conocimiento. Una mirada
a través de la revisión de investigaciones de los últimos 10 años [The teaching profession in the
knowledge society. A look through the review of research from the last 10 years]. Revista de
Educación a Distancia, 56(3), 1-22. doi: http://dx.doi.org/10.6018/red/56/3.
Morín, E. (1998). Introducción al pensamiento complejo [Introduction to complex thinking]. Barcelona:
Gedisa Editorial.
Morín, E. (1999). La cabeza bien puesta [The head well placed]. Buenos Aires: Buena visión.
Peñaranda, J. (2016). Pensamiento complejo de edgar morin: nueva visión del conocimiento [Edgar
Morín‘s complex thought: a new vision of knowledge]. Revista Ingenio, 11(1), 129-137. Retrieved
from: http://revistas.ufpso.edu.co/index.php/ringenio/article/view/332/227.
Ramírez, J. (2013). Humanización del aprendizaje en la era de la Información: una arista andragógica
[Humanizing Learning in the Information Age: An Andragogical Edge]. Actualidades
investigativas en educación, 13(3), 1-18. doi: 10.15517/aie.v13i3.12029.
Ramírez, M., Suárez, L. and Torres, J. (2017). El docente y su relación con la investigación e innovación
educativa: estudio con docentes del Instituto Politécnico Nacional [The teacher and his
relationship with research and educational innovation: study with teaching staff of the
Instituto Politécnico Nacional]. Investigação Qualitativa em Educação, 1, 930-939. Retrieved
from: https://proceedings.ciaiq.org/index.php/ciaiq2017/article/view/1414/1371.
Rodríguez, G., Gil, J. & García, E. (1999). Metodología de la investigación educativa [Methodology of
educational research]. España: Aljibe.
Rosario, A. (2018). La propuesta de Newman para una formación humanista [Newman‘s proposal
for a humanist formation]. Iglesia, Comunicación y Cultura, 3(1), 22-35. doi: 10.1080 /
23753234.2018.1426994.
Roth, W. (2016). Analyzing the qualitative data analyst: A naturalistic investigation of data interpretation.
Forum: Qualitative Social Research, 16(3), 1-19. doi: http://dx.doi.org/10.17169/fqs-16.3.2415.
Sanchez, P. (2017). The educational orientation in the university from the perspective of teaching staff.
Universidad y Sociedad, 9(2), 39-45. Retrieved from: http://rus.ucf.edu.cu/index.php/rus
Tejada, J. (2008). Perfil docente y modelos de formación. Estrategias didácticas innovadoras. Recursos
para la formación y el cambio [Teacher profile and formation models. Innovative teaching
strategies. Resources for formation and change]. Barcelona: Octaedro.
Tejedor, F. (2018). Investigación educativa: la utilidad como criterio social de calidad [Educational
research: Usefulness as a social criterion of quality]. Revista de Investigación Educativa, 36(2),
315-330. doi: https://doi.org/10.6018/rie.36.2.326311.
Veytia, M. (2014). La andragogía y la competencia en el manejo de herramientas tecnológicas, dos pilares
clave para el estudiante de posgrado en modalidad virtual [Andragogy and competence in the
handling of technological tools, two key pillars for the postgraduate student in virtual modality].
Revista Iberoamericana de Producción Académica y Gestión Educativa, 1, 1-16.
Yuni, J., & Urbano, C. (2005). Investigación etnográfica. Acción de investigación [Ethnographic Research.
Investigation action]. Argentina: Brujas.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
378
Received: April 21, 2019 Accepted: June 12, 2019
Derling Mendoza Velazco Doctor in Education, Research Professor, National University of Education,
(Corresponding author) Ecuador.
E-mail: derling.mendoza@unae.edu.ec
Website: https://orcid.org/0000-0001-8275-3687
Irma Abrigo Córdova Doctor in Accounting and Business Sciences, Research Professor, International
University of Ecuador, Loja, Ecuador.
E-mail: irabrigoco@uide.edu.ec
Janela Romero Chávez Master in International Business and Foreign Trade Management, Professor,
International University of Ecuador, Ecuador.
E-mail: karomeroch@uide.edu.ec
Fredy Cueva Bravo Master of Business, Educational Research, International University of Ecuador,
Ecuador.
E-mail: frcuevabr@uide.edu.ec
Magda Francisca Cejas Researcher and Teacher of the Department of Economic, Administrative and Trade
Sciences, University of the Armed Forces (ESPE), Ecuador.
E-mail: mfcejas@espe.edu.ec

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
ON PERCEPTION OF COMPUTER ALGEBRA 379
SYSTEMS AND MICROSOFT EXCEL BY

ENGINEERING STUDENTS
Natalia M. Mezhennaya, Oleg V. Pugachev
Bauman Moscow State Technical University, Russia
E-mail: natalia.mezhennaya@gmail.com, opugachev@yandex.ru
Abstract
This research presents the analysis of image of the popular program packages: CAS (Computer Algebra
Systems) Matlab, Mathcad, Mathematica, and the spreadsheet ‘Microsoft Excel’ among students of
engineering specializations studying in Bauman Moscow State Technical University (BMSTU). The
research has explored perception of program packages by all students (N=164), and by male students
(N=110) and female students (N=54) separately. Image of the program packages, convenience of their
interfaces, usage scenarios, and applicability for solution of tasks of certain types have been explored
via questioning the students. Moreover, the students assessed likelihood of using the programs in the
future. It was established that all the program packages under consideration can be successfully used
in the process of education, but some of them (Matlab and Mathematica) require additional lessons to
learn their interface and peculiarities. Gender differences in perception of the program packages do not
determine the education outcomes. The differences of perception of the programs are mainly caused by
their popularity and the students’ skills of using certain programs. The following gender differences have
been detected: in the female students’ sample, the programs Mathcad and Excel are leading, Matlab is
performed a little worse; Matlab is associated with the statement “difficult / inconvenient interface”. In
the male students’ sample, the three programs Matlab, Mathcad, and Microsoft Excel got nearly equal
grades. Female students estimate future usage of Microsoft Excel higher than male students do. The CAS
Mathematica takes the lowest places for both samples; as a possible cause of such result, it has been
found that Mathematica is not widespread as compared to the other programs under consideration.
Keywords: computer algebra system, education statistics, engineering education, mathematical
education, Mathcad, Mathematica, Matlab, Microsoft Excel.
Introduction
Students in contemporary universities have to study in large and very heterogeneous

classes. Differences between students may grow quite substantial, due to short time of education
(e.g., during completing master’s degree) and saturation of education programs. Hence it is
necessary to elaborate teaching means purposed to eliminate these differences and adapted
for students having trained up to different levels and having various motivation and interests.
Moreover, one should design the teaching means to make them convenient for students’
independent work.
It is well known (Peng, Hong, & Mason, 2014) that any teaching means should involve
students into the education process and motivate them to achieve the stated purposes of
education. The researches (Barba, Kennedy, & Ainley, 2016; Kalyar, Ahmad, & Kalyar, 2018)
have demonstrated the importance of motivation and its influence on education outcomes. The
research (Ivaniushina, Alexandrov, & Musabirov, 2016) has considered the regression model of
the linkage between motivation and students’ expectations and the level of confidence in their
possibilities, depending on their gender, age, and previous education successes; it has established
a significant direct linkage between expectation of success and the education outcome.

Natalia M. MEZHENNAYA, Oleg V. PUGACHEV. On perception of computer algebra systems and microsoft excel by engineering
students
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
380 The basic reason why teaching of engineering disciplines is more expensive as compared
to other subjects, e.g., humanitarian disciplines, is the necessity of labs. Experiment analysis
teaches the practical aspects of the course and gives students the experience of collecting and
real-time analysis of empiric data. Nowadays the majority of education institutes use computer
modeling instead of expensive maintaining of labs (Erokhin, Sadikova, Zhdankina, Korzhuev,
& Semenov, 2018).
One of possible didactic methods purposed to solve the stated problems is usage of
specialized computer packages: programming languages (Peng et al., 2014); CAS – Computer
Algebra Systems (Buteau, Jarvis, & Lavicza, 2014; Mezhennaya & Pugachev, 2018; Särvari,
2005); spreadsheets (Neuwirth, 1995; Haspekian, 2005; Ibrahim, 2009; Sanford, 2018);
program packages for mobile devices (Harrison & Lee, 2018), e.g., GeoGebra (Albano & Dello
Iacono, 2018; Jacinto & Carreira, 2017), Wolfram Alpha (Weisstein, 2014), etc.
The research (Buteau et al., 2014) has explored the attitude of the modern scientific society
to CAS and established that STEM specialists (STEM – Science, Technology, Engineering, and
Mathematics) regard CAS as an integral part of research methods of modern mathematics;
therefore, one of the main drivers of penetration of CAS in education is their usage in future
research work. Hereby the research (Buteau et al., 2014) has revealed that CAS are used in the
education process mainly as a means of solving tasks, and quite rarely as a means of current
and final control; the decision to use CAS and/or to use a certain type of CAS is mainly a
teacher’s individual initiative. Many researchers describe the methods of penetration of CAS in
the education process, see e.g. (Särvari, 2005). The didactic aspects of using CAS in teaching
certain mathematical disciplines is the topic of discussion in (Mezhennaya & Pugachev, 2018;
Vasilieva, 2011).
The research (Cretchley, Harman, Ellerton, & Fogarty, 2000) has dealt with university
first year students’ attitude to usage of Matlab-based teaching means. The research has
established that it is possible to overcome the difficulties of primary study of the interface
and syntax system rather quickly. They have compared the students’ attitude to the Matlab-
based teaching materials and the students’ skills in the beginning and in the end of a half-year
term. They have found significant positive influence of penetration and usage of the education
technique, due to the growth of the students’ interest in the process of education. Hereby they
have detected certain gender differences in attitude to and usage of Matlab. In (Durán et al.,
2007) the attitude of students of electro-technical specialization to Matlab-based materials
has been researched, general positive influence of these materials on education outcomes and
contentedness of students has been detected. The research (Barkatsas, Gialamas, & Orellana,
2016) has evaluated students’ attitude to education with using CAS in general; the results of
the research demonstrate positive influence of using CAS on achieving education outcomes
and satisfaction of students with the education process. Usage of CAS establishes the balance
between the students’ assessment of their skill of computer-assisted or analytical task solution
and the real level of their training in mathematics; this factor (Morán-Soto & Benson, 2018)
increases the level of education outcomes.
Another advantage of CAS is the possibility to create interactive templates (Mezhennaya
& Pugachev, 2018; Mezhennaya & Pugachev, 2019) used by both students and teachers, saving
time of independent work, composition and renovation of task banks (Vlasova, Popov, &
Pugachev, 2016), checking and discussing the tasks fulfilled. Moreover, due to CAS usage,
students are active participants of the education process.
According to (Broley, Caron, & Saint-Aubin, 2018), one of the difficulties of CAS
penetration is a significant fragmentation of students’ skills. The same situation takes place with
respect to traditional programming languages. The research (Broley, Caron, & Saint-Aubin,
2018) has detected high volatility of students’ skills of using program packages for solving
tasks. Due to large variety of program packages and various factors influencing their choice by

students
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
any student, it is difficult to provide a necessary level of mastering a certain program package 381
in a large collective of students.
In order to overcome the difficulties described, it is possible to use teaching means
based on spreadsheets, e.g. the widespread program package Microsoft Excel. Such packages
installed in the majority of display rooms are accessible for students, and their basic usage is
taught already in the last school years (Gorbacheva, Smirnova, & Potekhin, 2008). In (Beare,
1991) it was established that usage of Microsoft Excel-based teaching materials makes the
education process problem-oriented and therefore leads to its general activation.
The research (Malone, Schunn, & Schuchardt, 2018) has established that education
based on the use of spreadsheets should not be uniform for all students, it is necessary to
consider their previous training levels. Namely, for students having lower level of primary
computer training, education outcomes would be better if they study spreadsheets before
studying mathematical subjects. In converse, for the students having trained with computers
very well, one should hold both courses in parallel.
The research (Mezhennaya & Pugachev, 2018) has described a possible technique of
using CAS in teaching the course of probability theory. The Mezhennaya & Pugachev have
established that the demonstration materials created by teachers with usage of interactive
templates in Wolfram Mathematica are well comprehensible for students; the education
outcomes of the students taught with the CAS appeared significantly higher as compared
to the students who had not used these supplementary materials. At the same time, students
considered usage of the templates as quite difficult. There were two possible reasons: a) some
students were not motivated to master and modify the template files suggested, b) the system
Wolfram Mathematica was not widespread among engineering students. Hence it is necessary
to carry out a research of students’ attitude to popular program packages, including the purpose
to detect resemblances and differences between students whose native language is Russian and
foreign students having learned Russian.
In elaboration of teaching materials, it is also necessary to consider possible gender
differences of their apprehension and usage. In (Ivaniushina et al., 2016) gender differences in
learning a university course “Data analysis” have been researched; the Ivaniushina, Alexandrov,
& Musabirov have noticed that men are more self-assured, but the difference of education
outcomes of men and women is not statistically significant. The research (Tikhomirova, 2017)
of spatial thinking of students with different levels of mathematical training has established that
a respondent’s gender has practically no influence on the amount of correct answers.
On the other hand, gender differences in apprehension and usage of teaching materials
are possible. For example, (Davidovitch & Yavich, 2018) have found gender differences in
students’ perception of tablets used in education. Namely, male students attribute more weight
to tablets in both cognitive and emotional aspects of their application.
The research (Zeldin & Pajares, 2000) has revealed that verbal influence and indirect
experience were basic factors of women’s self-confidence to work in STEM-subjects; assumable
importance of these sources of self-effectiveness can be stronger for women in men-oriented
spheres than in other women’s activities.
The research (Peng et al., 2014) has studied the students’ choice of strategies of passing
tests; hereby some gender differences in self-effectiveness have been detected. Gender
differences in the choice of education strategies and self-effectiveness have been also revealed
in (Ozan, Gundoglu, Bay, & Celkan, 2012). The research (Zhan, Fong, Mei, & Liang, 2015)
has established that gender differences take place in a group work as well. For example, male
students demonstrate better results when working in a mix-gender group than in a single
gender group. Hereby, male students prefer to work in mix-gender (gender-balanced or gender-
majority) groups, whereas female students prefer single-gender and gender-minority groups.
Gender differences take place in the attitude to education but not in education outcomes.

students
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
382 The aim of the present research was to study popularity, perception and usage scenarios
(and possible gender differences in them) of several widespread program packages (Matlab,
Mathcad, Mathematica, Microsoft Excel) among students. The research has included
engineering students of the 3-rd and 4-th years completing bachelors’ degree and of the 1-st
year completing master’s degree. All the students studied in Bauman Moscow state technical
university (BMSTU). The first factor determining the choice of programs is the set of program
packages installed in display rooms in the University; it is also important which of them teachers
use in education in various specializations, and which licenses are available for students. The
image of program packages was under research by questioning in two groups of male and
female students. The results obtained have undergone analysis and comparison.
General Background
The present research has dealt with popularity, perception and usage scenarios of several
program packages often used in education: Matlab, Mathcad, Mathematica, and Microsoft
Excel, among engineering students taught in BMSTU in the autumn term of the 2018-2019
year. Moreover, the researchers analyzed similarities and differences between male and female
students. For this purpose, the researchers have carried out an anonymous questioning of the
students, in order to clarify their attitude to the program packages mentioned. After that, they
have proceeded a statistical analysis of the data obtained. The presented results can be useful for
the choice of engineering students’ education strategies based on using one or several program
packages. The research has included the data collected in December 2018, the statistical analysis
has been carried out in January–March 2019.
Sample
The researchers have questioned students taught in the faculty of Special Machine-
building of BMSTU in the 3-rd and 4-th years completing bachelors’ degree and in the 1-st year
completing master’s degree. All the students have studied in the autumn term in 2018. The group
under research consisted of 164 students (110 male and 54 female students); each of them had
used at least one of the program packages Matlab, Mathcad, Mathematica, or Microsoft Excel
during their study (in audience or in independent work). The research included the questioning
during the first two weeks of December 2018. Table 1 presents the demographical data of the
sample. Figure 1 shows the histogram of age distribution in each group.
Table 1. Demographical data of the whole sample and of male and female
samples separately.
All Male Female

Mean age 21.84 21.97 21.57
Median age 22.00 21.50 22.00
Minimum age 19.00 19.00 19.00
Maximum age 38.00 38.00 33.00
Standard deviation 2.50 2.73 1.95

students
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
383
Figure 1. Proportions of the students of each age category of male and female.
Of the total amount of 250 questionnaires, 182 came back. The data of 164 questionnaires
satisfying the given criterion have undergone statistical analysis.
Instruments and Procedures
The questionnaire shown in Table 2 was used as a research tool. The questionnaire
included the standard demographical block and several special questions about the usage of
computer programs in education. The special questions included the block (Question 5) asking
which program packages the student had used in education and the block about their usage
scenarios (Question 6). The questionnaire suggested the students to assess convenience of using
each of the packages for the following purposes:
a) Solution of tasks in mathematics, physics, and special subjects.
b) Preparing presentations.
c) Using on mobile devices (smart phones and tablets).
d) Using in display rooms (or rooms equipped with stationary computers) and in ordinary
lecture halls and seminar rooms.
e) Interface friendliness and applicability of the program packages for solving problems
requiring much resource.
The last block (Question 7) asked which program packages the students were going to
use in the future.

students
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
384 Table 2. The questionnaire suggested to students.
An anonymous survey of students with special questionnaires described in detail further

was carried out. If a student has used no one of the program packages, the research excluded
his/her questionnaire.
Each student received a personal questionnaire printed on paper and filled it individually
(without the teachers’ help). The questioning was anonymous, and the students knew that it
would not influence their rating points. The students returned the filled questionnaires to the
teachers of their groups.

students
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
Data Analysis 385
The filled questionnaires were carefully processed and checked whether the students
had filled them correctly. Questions 1-4 should be answered and contain no empty items;
Question 5 should contain at least one mark; Question 6 should be filled only in concern with
the program packages mentioned in Question 5, so it might contain empty cells. The answer to
Question 7 was independent on the previous questions and could contain empty cells too. The
questionnaires not satisfying at least one of the demands listed have been excluded from the
research.
The standard methods of statistical analysis (descriptive methods of statistics, analysis
of contingency tables, correspondence analysis) have been applied to the sample of 164
questionnaires. The programs Statistica 12.0, Mathematica 11.3, and Microsoft Excel 2016
were used in the analysis.
Research Results
Usage of the Programs
Table 3 shows the percentage of students having used each of the packages, and the
p-values calculated from comparison of the proportions in the two groups.
Table 3. Percentages of students having used each of the program packages,

for the groups of male and female students, and the corresponding
p-values calculated for the difference between the two proportions (in
the corresponding column).
Matlab Mathcad Mathematica MS Excel Total

Male 78.18 88.18 19.09 90.91 110
Female 59.26 81.48 27.78 90.74 54
p-value .0113 .2456 .2064 .9717
Table 3 shows that the most popular among both male and female students is MS Excel.
The second is Mathcad and the third is Matlab. The CAS Mathematica was the least popular.
To the researchers’ opinion, the possible reasons why almost all students of both genders
used Microsoft Excel are the following:
a) It is widespread.
b) The education program includes many tasks, which students cannot solve without
using a personal computer. For their solution teachers often choose Microsoft Excel
as the most simple and acceptable software.
c) Usually it is not required to install Microsoft Excel specially, since it is presented
in the majority of display rooms; moreover, it is compatible with other freeware
spreadsheets which students can use in their independent work.
d) The basis of working with Microsoft Excel has been taught in many schools.
The package Mathcad is not far behind Microsoft Excel.
The research has revealed a significant (at level 5%) difference only in the percentages of
male and female students using Matlab. The ratio between the percentages of male and female
students using this package is approximately 4 to 3.
With respect to CAS Mathematica, the situation is the opposite. Female students use
it more than male students (the ratio of the shares is about 3 to 2), but there is no significant
difference at level 5%.

students
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
386 In order to analyze possible gender differences in perception of the program packages
and their usage scenarios, the researchers have performed further statistical analysis. For this
purpose, for each program package and each statement in Question 6, the quantities of students
having marked 4 or 5 (meaning that they agreed with the statement) in the groups of male and
female students were calculated.
The next subsection presents the classical correspondence analysis of the data (Greenacre,
1984), used as exploratory method for determining possible gender differences and common
students’ attitude to the program packages.
Correspondence Analysis for Groups of Male and Female Students
The correspondence analysis begins with analyzing the graph of eigenvalues shown in
Figure 2.
Figure 2. Graph of eigenvalues; percentage of inertia in axes: axis 1 – 86.36%;

axis 2 – 6.81%; axis 3 – 4.49%; axis 4 – .91%; axis 5 – .60%, axis
6 – .55%, axis 7 – .26%.
One can see from Figure 2 that the point corresponding to two dimensions is a breakpoint
of the graph. The percentage of inertia explained by the first two axes is 93.17%. Hence, the
two-dimensional map shown in Figure 3 was used. For the further text, see interpretation of the
statements Q1–Q15 in Table 2.

students
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
387
Figure 3. The correspondence map for male (square marks) and female (triangle
marks) samples.
The round marks denote the statements of Question 6; the quality of projection onto the
plane is ≥ .5 for all points. The marks corresponding to the program packages are squares for
the male sample and triangles for the female sample. Unsatisfactory quality of projection (< .5)
appears only for the point corresponding to Mathematica in the male sample; hence, its mark
is not fully colored.
For all the statements Q1–Q15 from Question 6, the quality of projection onto the plane
appeared good: ≥ .52 for the points Q10, Q11, and Q15; ≥ .79 for the points Q4, Q7, and Q14;
≥ .85 for the rest points. For all the program packages (except for Mathematica in the group
of men), the quality of projection appeared good as well: .53 for Mathematica in the female
sample; ≥ .90 for the rest points. The map in Figure 3 explains 93.17% of dispersion of the
points. Hence, the map presents the situation of the points in the original 7-dimensional space
quite exactly. The present research will not include the position of the point corresponding to
Mathematica in the male sample.
One can remark an interesting fact: the positions of points corresponding to Matlab,
Mathcad, and Microsoft Excel, are practically the same in both male and female samples.
Therefore, attitude to these program packages is mainly the same, and hence the outcomes
of education with methods using one or several program packages would have assumedly no
significant gender differences.
Let us proceed to a more detailed analysis of the map. The positions of points for Matlab,
Mathcad, and Microsoft Excel form a triangle (marked with gray lines on the map). Hence, these
program packages oppose to each other in certain parameters. For Mathcad and Matlab, this is
a surprise. Microsoft Excel is more associated with usage on mobile devices (the statements Q5
and Q6) than any other CAS. This is also unexpected, because there is a free mobile version
of Matlab with an interface analogous to the desktop version of the program. Therefore, either
students are not aware of this, or they regard the mobile version as non-convenient. This fact
requires additional data collection and research.
The statements Q2, Q3, Q9 (the left-hand side) and Q5, Q6 (the right-hand side) load the
axis Dim.1. The conclusion was that the students of both genders practically do not distinguish

students
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
388 usage of programs on tablets and smart phones. The statements Q13, Q14 (downside), and Q6
(upside) load the axis Dim.2.
The program packages and the statements on the map Figure 3 form three clusters marked
with black lines. The first cluster consists of the points corresponding to Microsoft Excel in
both groups and the statement Q7. Such situation is natural since the program contains a large
amount of inbuilt functions for convenient visualization of data. However, each CAS possesses
such kind of functions. Hence, there are two possible reasons (or combination of both reasons)
of the phenomenon observed: either the students are not sufficiently trained to use graphical
possibilities of the CAS’s under consideration, or students regard the interface of Microsoft
Excel, which does not require to write a script, as more convenient. The linkage between the
program package and the statement is stronger in the group of women.
The points corresponding to the CAS Mathcad in both groups, and the statements Q1, Q2,
and Q12, form the second cluster. It has turned out that Mathcad is more convenient for solving
tasks in mathematics and physics, and even general engineering tasks, than any other program
package. There are two possible reasons of such perception: a) the education program of a part
of the students contained compulsory subjects including usage of Mathcad; b) peculiarities of
the program interface of Mathcad. The male sample has shown a stronger linkage between the
program package and the statements.
The points of the third cluster correspond to the CAS Matlab for both samples and the
statements Q3, Q9, and Q15. Students regard the CAS Matlab as suitable to solve engineering
tasks including those demanding computation, but the interface of Matlab is difficult or not
convenient. For the last opinion, a possible reason is the lack of knowledge of the program
package in whole. Students’ attitude to Matlab is positive in general, but in order to use it in
education successfully, a stronger training in its interface and language is necessary. The map
indicates no gender differences in apprehension of Matlab.
The CAS Mathematica in the female group stands aside from other program packages
and statements. One of possible causes is small propagation of the program among engineering
students, and small amount of its usage does not allow estimating its convenience.
The statements Q8, Q13, and Q14 equally associate with Microsoft Excel and Mathcad.
As to convenience of usage in seminar rooms without computers (the statement Q10), the
students did not prefer any one of the program packages. Therefore, the research has revealed
that usage of mobile versions of the program packages was not widespread among the students
taking part in the survey.
Analysis of Future Usage of Program Packages
In order to estimate general experience of the program packages, let us proceed to

analyze students’ plans of their future usage of programs and consider male and female samples
side by side with the whole sample. One would expect that the students having used one of the
programs would estimate it to be better than the other ones. The following research indicates
whether it is true.
The CAS Mathematica has appeared worse than the other program packages; it has
received the average mark 2.44 from the male students and 2.51 from the female students (in a
5-points scale). Even those who have used Mathematica estimate its reusability by 3.29 (men)
and 3.15 (women). The CAS Mathematica has appeared unsatisfactory, hence the analysis of
only three other program packages is considered in this section.
Figure 4 presents the average grades of reusability of each program package. The data
was collected from all male and female students, and from users of certain program packages.

students
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
389
Figure 4. Average grades of reusability of each program package by male

(square marks) and female (triangle marks) students for the whole
sample, and for the samples of users of each program.
The estimates calculated from the whole sample are slightly different for male and female
students. The average estimate given by male students is nearly the same for all programs,
whereas female students estimated Matlab worse than the other programs, but average estimates
of Mathcad and MS Excel by female students are higher.
The male Matlab users expectedly considered Matlab as the best for future use, the next
were Mathcad and Excel. Conversely, the female students who had used Matlab estimated
its future usage lower than other programs. Hence, women regarded this program package
as less convenient than the other ones. However, the average estimate of Matlab was quite
high. This phenomenon can be explained by women’s lack of knowledge of this program and
peculiarity of the tasks they had considered during education. It is interesting to note that,
whatever program package had female students used, they considered Matlab as less reusable
than Mathcad and Excel.
The male users of Mathcad and Excel gave almost equal average grades of reusability
to all the three program packages, although both groups estimated Mathcad slightly higher.
Hence, the possibilities of MS Excel in some tasks are not sufficient. Generally, male students
in all groups, except the Excel users, estimate Excel as weaker than other programs. In the
corresponding samples of women, the situation was different: the estimates differ from each
other. The female Mathcad users estimated future reference of Mathcad higher than other
programs, this result was expectable. The situation for the female students having used
Microsoft Excel is analogous: they gave the highest grade to Microsoft Excel.
The Mathematica users of both genders surprisingly estimate future reference of this
program package lower than usage of the other ones. A possible reason is that engineering
students study the CAS Mathematica only in their free will in their independent work, since
their education program contains no compulsory courses including usage of Mathematica.
For quantitative analysis of differences of estimates in the samples under consideration,
Mann–Whitney U-test was applied. There was significant difference at the level 5% between
the estimates given to Mathcad by female and male Mathcad users (Z-statistic = -2.23; p-value
= .0260). Significant differences at the level 10% took place between the estimates of Mathcad

students
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
390 by the Matlab users of different genders (Z-statistic= -1.65; p-value = .0998), between the
estimates of Microsoft Excel between male and female students in the whole sample (Z-statistic
= -1.75; p-value = .0796), and among Microsoft Excel users (Z-statistic = -1.95; p-value =
.0515).
To sum up, in the female students sample the programs Mathcad and Excel are leading,
Matlab is performed a little worse. As noted in the previous subsection, Matlab is associated
with the statement Q15 “difficult / inconvenient interface”. In the male sample, the three
programs got nearly equal grades. Female students estimate future reference of Microsoft Excel
higher than male students do.
Discussion
The students were suggested to assess several program packages with the following
respects: solution of tasks in mathematics, physics, and special subjects; preparing presentations;
usage on mobile devices and in display rooms; and interface friendliness. The last question
asked which program packages the students were going to use in the future.
The main purpose of this research was to answer the following three questions:
1) With respect to Microsoft Excel: Is it the most popular? What is the students’ attitude
to its usage for solution of tasks concerned with teaching STEM-subjects?
2) With respect to the CAS Matlab, Mathcad, and Mathematica: Are there differences in
comprehension of these CAS (by all students or by males/females separately)? Which of the
CAS are more suitable for teaching students in large heterogeneous classes?
3) With respect to the programs adapted for mobile devices: Do the students prefer them
to desktop program packages?
MS Excel appeared the most popular among both male and female students (91%). The
second was Mathcad (used by 88% of male and 81% of female students) and the third was
Matlab (78% and 59%, respectively). The CAS Mathematica was the least popular; only 19%
of male and 28% of female students used it. Almost all students of both genders use Microsoft
Excel since it is widespread (it is presented in the majority of display rooms), it is the most
simple and acceptable software for solution of students’ tasks, and the basis of working with
Microsoft Excel has been taught in many schools (Beare, 1991), (Gorbacheva et al., 2008).
This confirms the conclusion made in (Broley et al., 2018) about significant segmentation of
students with respect to the choice of CAS and levels of mastering them. In the same time,
the situation with Microsoft Excel is the reverse; the results obtained agree with the research
(Mezhennaya, 2019) which has established that Microsoft Excel could be used in teaching
engineering students from the beginning of the University course, without assigning additional
lessons for its learning.
Microsoft Excel is more associated with the usage on mobile devices than any CAS,
although there exists, e.g., a free mobile version of Matlab with an interface analogous to its
desktop version. Therefore, either students are not aware of this, or they regard the mobile
version as non-convenient. This fact requires additional data collection and research.
Thus, a surprising result is established: students of both genders do not choose aps for
their convenience for using on mobile devices but choose them for convenience of task solution.
This situation can be partially explained by the fact that mobile versions of the programs do not
work well enough for solution of complicated engineering tasks.
Therefore, the present research gives a positive answer to the first question concerned
with Microsoft Excel. In the same time, certain difficulties arise in the usage of Microsoft
Excel, e.g., a big amount of routine work or the necessity to learn programming languages
(Visual Basic, Delphi, etc.). The present research did not touch the questions of mastering
programming skills, but, as teachers often see, the majority of students either do not possess

students
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
programming skills, or they are strictly segmented in languages used (Broley et al., 2018). 391
Moreover, the programming languages are not always adapted to solution of mathematical
tasks. In the other hand, there are a lot of languages elaborated for teaching mathematics (e.g.,
Logo (Hoyles & Sutherland, 1989), Scratch (Swanier, Seals, & Billionniere, 2009), etc.), but
their usage in large classes without assigning supplementary lessons for learning them would
lead to much difficulties. Therefore, often it is necessary to penetrate more popular CAS into
education. Besides, this is a necessary factor of students’ competition within their future work
(Buteau et al., 2014).
Proceed to the CAS considered in this research. It has turned out that Mathcad is more
convenient for solving tasks in mathematics and physics, and general engineering tasks as well,
than any other program package. There are two possible reasons: the education program of
a part of the students had contained compulsory subjects including usage of Mathcad; some
peculiarities of the program interface of Mathcad are more convenient.
Students regard Matlab as suitable for solution of engineering tasks including those
demanding computation (Cretchley et al., 2000), but the interface of Matlab is difficult or not
convenient. For the last opinion, a possible reason is the lack of knowledge of the program
package in whole. In order to use Matlab in education successfully, a stronger training in its
interface and language is necessary.
The CAS Mathematica stands aside from other program packages. One of possible
causes is the scarce propagation of this program among engineering students. This does not
allow estimating its convenience.
The research has revealed a significant difference in the percentages of male and female
students using Matlab, certain gender differences in usage of Matlab were detected already by
(Cretchley et al., 2000). The ratio between the percentages of male and female students using
this package is approximately 4 to 3. With respect to CAS Mathematica, the situation is the
opposite. Female students use it more frequently than male students (the ratio of the shares is
about 3 to 2).
It should be remarked that the attitude to Matlab, Mathcad, and Microsoft Excel by
male and female students is practically the same. Hence, the outcomes of education with
methods using one or several program packages would have assumedly no significant gender
differences. This agrees with the results (Ivaniushina et al., 2016), (Tikhomirova, 2017), where
no gender differences in education outcomes were detected either. However, there are possible
gender differences in the cognitive and emotional aspects of apprehension of these programs
(analogous to those detected with respect to tablets in (Davidovitch & Yavich, 2018)), which
were not considered in this research. Hence, it is necessary to carry out an additional research, in
order to find such differences, and to use them effectively in designing new teaching materials,
especially for independent work.
In order to estimate general experience of the program packages, the researchers have
analyzed students’ plans of their future reference. Expectedly, the students having used one of
the programs would estimate it higher than other ones, except CAS Mathematica. A possible
reason is that engineering students study the CAS Mathematica only in their free will, since
their education program contains no compulsory courses including usage of Mathematica.
To sum up, in the female students sample the programs Mathcad and Excel are leading,
Matlab is performed a little worse. Matlab is associated with the statement “it is difficult / has
inconvenient interface”. In the male sample, the three programs got nearly equal grades. Female
students estimate future usage of Microsoft Excel higher than male students do.
So, the second question of the research received the following answer. Popularity of
a program package influences positively on its apprehension by students of both genders.
This is caused by previous experience of some students, by Internet availability of additional
information on the program packages, and by the possibility of other students’ help. Thus, the

students
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
392 choice of the program package to use in composing teaching materials ought to be considered,
which of the program packages had the students learned earlier. If the university education
did not include the courses of program packages, then the teacher has no basis to choose one
of another system. Therefore, it is necessary to elaborate the most flexible teaching materials,
which would be possible to adapt to any platform.
In concern with modern engineering problems and scientific research methods (Buteau
et al., 2014) it is necessary to penetrate CAS since the first half-year term of education. Hereby,
the education program should not be oriented on a certain program package, but on studying
the basic principles of work with such systems. Students should understand that the choice
of a program package for solution is often determined by other factors than the student’s will
(the tasks to solve, equipment available, the instructor’s decision, etc.). Moreover, in the future
engineering practice they have to adapt quickly to any system. The purpose of education is to
train students to use almost any CAS. One of the methods is usage of interactive templates
(Mezhennaya & Pugachev, 2018) prepared by the teachers and used in students’ independent
work. In the future practice, students can either receive analogous templates in their firms or
find suitable solutions in Internet independently.
Another possible strategy of education is endowing the students with the right to choose
a certain program package. Hereby it is necessary to consider the increase of teachers’ work
and the necessity to explain the students the criteria of choice of suitable program packages
(Harrison & Lee, 2018). For example, many students use various on-line calculators providing
sufficient accuracy of solution of simple tasks, e.g., systems of linear equations with few
variables, but such tools are not suitable for calculations requiring much resource.
In the present research, it was not explored what reasons induce students to choose a
certain program package. There are several possible reasons: friends’ advice, a teacher’s
instruction, availability of a license, usage in the work. For example, in (Buteau et al., 2014)
it was found that the choice of a certain program package is often determined by the teacher’s
decision depending on his/her personal preference and on availability of technical possibilities.
The question of students’ choice of program packages requires a new research.
Another interesting problem is the linkage between using the program packages by
junior and major students. In the present research, this question was not explored since there
were too few students having long interruption in education. But since many major students are
employed on jobs requiring usage of certain program packages chosen without the students’
opinion, the differences between the first-year and the major students can be significant. It is
worth to conduct an additional research, in order to establish the basic factors influencing the
choice of program packages to learn.
In the sample of students taking part in the survey, there were 29 foreign students. They
had a different situation depending on previous experience. They had an additional difficulty in
studying programming in Russian, side by side with difficulty of programming itself. Hence it
is necessary to proceed to the research of possible difficulties concerned with teaching foreign
students, and to consider possible difficulties of spreading program packages between Russian-
speaking and foreign students.
In the contemporary conditions, the education process should train students for a variable
work within future engineering practice, first by penetration of student-oriented education.
Hereby the education methods should adapt to both specific purposes of education and certain
student’s interests and skills. For development of such methods, it is possible to use computer-
based teaching techniques, including the CAS-based ones. Moreover, one should consider
positive influence of motivation constraints arising from the usage of program packages
(Dana-Picard, 2007). The present research partly clarifies the question which systems can be
successfully used in this trend.

students
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
Conclusions 393
The present research deals with popularity, perception and usage scenarios of several
program packages often used in education: Matlab, Mathcad, Mathematica, and Microsoft Excel,
among engineering students of Bauman Moscow State Technical University in the autumn term
of the 2018-2019 year. Hereby possible gender differences in the aforementioned respects were
analyzed. It has been found that almost all of the systems considered can be successfully used in
education, but some of them (Matlab and Mathematica) require additional lessons to learn their
interface and peculiarities. Gender differences in perception of the program packages do not
determine the education outcomes. The differences of perception of the programs are mainly
caused by their popularity and the students’ skills of using certain program packages.
The results presented indicate that, in the choice of strategies of computer-assisted
education, the teachers ought to consider the students’ previous experience of mastering certain
program packages, and the education purposes stated. Often sharp segmentation of students
causes the necessity to assign supplementary lessons for learning a certain type of CAS. The
present research has established the peculiarities of apprehension of program packages (in
both male and female groups) which can be used by other researchers and teachers during
elaboration of teaching materials, taking gender structure of the groups of students, their
previous experience, and education purposes into account.
Acknowledgements
The researchers are grateful to Markelov G.E. for his help in collecting the statistical
data, and to all the students of BMSTU who have taken part in the survey.
References
Albano, G., & Dello Iacono, U. J. (2018). GeoGebra in e-learning environments: a possible integration in
mathematics and beyond. Journal of Ambient Intelligence and Humanized Computing, 9 (1), 1-13.
doi: 10.1007/s12652-018-1111-x.
Barba, P. G., Kennedy, G. E., & Ainley, M. D. (2016). The role of students’ motivation and participation
in predicting performance in a MOOC. Journal of Computer Assisted Learning, 32 (3), 218-231.
doi: 10.1111/ jcal.12130.
Barkatsas, T., Gialamas, V., & Orellana, C. (2016). Secondary students’ attitudes toward learning
mathematics with computer algebra systems (CAS). In: Barkatsas T., Bertram A. (Eds.), Global
Learning in the 21st Century. Global Education in the 21st Century Series. SensePublishers,
Rotterdam. doi: 10.1007/978-94-6300-761-0_8.
Beare, R. (1991). A system to exploit the spreadsheet ‘Excel’ for enhancing learning in science. Research
in Science Education, 21 (1), 20-29. doi: 10.1007/BF02360453.
Broley, L., Caron, F., & Saint-Aubin, Y. (2018). Levels of programming in mathematical research
and university mathematics education. International Journal of Research in Undergraduate
Mathematics Education, 4 (1), 38-55. doi: 10.1007/s40753-017-0066-1.
Buteau, C., Jarvis, D. H., & Lavicza, Z. (2014). On the integration of computer algebra systems (CAS) by
Canadian mathematicians: Results of a national survey. Canadian Journal of Science, Mathematics
and Technology Education, 14 (1), 35-57. doi: 10.1080/14926156.2014.874614.
Cretchley, P., Harman, C., Ellerton, N., & Fogarty, G. (2000). MATLAB in early undergraduate
mathematics: An investigation into the effects of scientific software on learning. Mathematics
Education Research Journal, 12 (3), 219-233. doi: 10.1007/BF03217086.
Dana-Picard, T. (2007). Motivating constraints of a pedagogy-embedded computer algebra system.
International Journal of Science and Mathematics Education, 5 (2), 217-235. doi: 10.1007/
s10763-006-9052-9.

students
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
394 Davidovitch, N., & Yavich, R. (2018). The impact of mobile tablet use on students’ perception of learning
processes. Problems of Education in the 21st Century, 76 (1), 29-42.
Durán, M. J., Gallardo, S., Toral, S. L., Martínez-Torres, R., & Barrero, F. J. (2007). A learning
methodology using Matlab/Simulink for undergraduate electrical engineering courses attending
to learner satisfaction outcomes. International Journal of Technology and Design Education, 17
(1), 55-73. doi: 10.1007/s10798-006-9007-z.
Erokhin, S. V., Sadikova, A. R., Zhdankina, J. S., Korzhuev, A. V., & Semenov, S. V. (2018). Moodle
e-learning platform as a resource for improving the quality of technical education. Novosibirsk
State Pedagogical University Bulletin, 6, 138-154. doi: 10.15293/2226-3365.1806.09.
Gorbacheva, A. N., Smirnova, A. N., & Potekhin, N. V. (2008). Solution of tasks for modeling in Microsoft
Excel. Informatics and Education, 3, 34-40.
Greenacre, M. J. (1984). Theory and applications of correspondence analysis. Academic Press, London.
Harrison, T. R., & Lee, H. S. (2018). iPads in the mathematics classroom: Developing criteria for
selecting appropriate learning apps. International Journal of Education in Mathematics, Science
and Technology (IJEMST), 6 (2), 155-172. doi:10.18404/ijemst.408939.
Haspekian, M. (2005). An “Instrumental Approach” to study the integration of a computer tool into
mathematics teaching: the Case of spreadsheets. International Journal of Computers for
Mathematical Learning, 10 (2), 109-141. doi: 10.1007/s10758-005-0395-z.
Hoyles, C., & Sutherland, R. (1989). Logo mathematics in the classroom. New York, NY: Routledge.
Ibrahim, D. (2009). Using the Excel spreadsheet in teaching science subjects. Procedia – Social and
Behavioral Sciences, 1 (1), 309-312. doi: 10.1016/j.sbspro.2009.01.058.
Ivaniushina, V. A., Alexandrov, D. A., & Musabirov, I. L. (2016). The structure of students’ motivation:
Expectancies and values in taking data science course. Educational Studies Moscow, 4, 229–250.
doi: 10.17323/1814-9545-2016-4-229-250.
Jacinto, H., & Carreira, S. (2017). Mathematical problem solving with technology: The Techno-
mathematical fluency of a student-with-GeoGebra. International Journal of Science and
Mathematics Education, 15 (6), 1115-1136. doi: 10.1007/s10763-016-9728-8.
Kalyar, M. N., Ahmad, B., & Kalyar, H. (2018). Does the teacher’s motivation influence the student’s
motivation? The mediating role of a teacher’s professional behavior. Educational Studies Moscow,
3, 91-119. doi: 10.17323/1814-9545-2018-3-91-119.
Malone, K. L., Schunn, C. D., & Schuchardt, A. M. (2018). Improving conceptual understanding and
representation skills through Excel-based modeling. Journal of Science Education and Technology,
27 (1), 30-44. doi: 10.1007/s10956-017-9706-0
Mezhennaya, N. M. (2019). On the perception of the ‘Microsoft Excel’ software program by engineering
students. Science for Education Today, 9 (2), 140-155. doi: 10.15293/2658-6762.1902.10.
Mezhennaya, N. M., & Pugachev, O. V. (2018). On the results of using interactive education methods in
teaching Probability Theory. Problems of Education in the 21st Century, 76 (5), 678-692.
Mezhennaya, N. M., & Pugachev, O. V. (2019). Advantages of using the CAS Mathematica in a study of
supplementary chapters of Probability Theory. European Journal of Contemporary Education, 8
(1), 4-24. doi: 10.13187/ejced.2019.1.4.
Morán-Soto, G., & Benson, L. (2018). Relationship of mathematics self-efficacy and competence with
behaviors and attitudes of engineering students with poor mathematics preparation. International
Journal of Education in Mathematics, Science and Technology (IJEMST), 6 (3), 200-220. doi:
10.18404/ijemst.428165.
Neuwirth, E. (1995). Visualizing formal and structural relationships with spreadsheets. In: diSessa A.
A., Hoyles C., Noss R., Edwards L. D. (eds) Computers and Exploratory Learning. NATO ASI
Series (Series F: Computer and Systems Sciences), vol 146. Springer, Berlin, Heidelberg. doi:
10.1007/978-3-642-57799-4_9.
Ozan, C., Gundoglu, K., Bay, E., & Celkan, H. Y. (2012). A study on the university students’ self-regulated
learning strategies skills and self-efficacy perceptions in terms of different variables. Procedia –
Social and Behavioral Sciences, 46, 1806-1811. doi: 10.1016/j.sbspro.2012.05.383.
Peng, Y., Hong, E., & Mason, E. (2014). Motivational and cognitive test-taking strategies and their
influence on test performance in mathematics. Educational Research and Evaluation, 20 (5), 366-
385. doi: 10.1080/13803611.2014.966115.

students
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
Sanford, J. (2018). Introducing computational thinking through spreadsheets. In: Khine M. (eds) 395
Computational Thinking in the STEM Disciplines. Springer, Cham. doi: 10.1007/978-3-319-
93566-9_6.
Särvari, C. (2005). CAS integration into learning environment. ZDM, 37 (5), 418-423. doi: 10.1007/
s11858-005-0031-3
Swanier, C., Seals, C., & Billionniere, E. (2009). Visual programming: A programming tool for increasing
mathematics achievement. iManager’s Journal of Educational Technology, 6 (2), 1-5. doi:
10.26634/jet.6.2.784.
Tikhomirova, T. (2017). Spatial thinking and memory in Russian high school students with different
levels of mathematical fluency. Procedia – Social and Behavioral Sciences, 237, 1260-1264.
Vasilieva, L. N. (2011). Using Matlab package in the course of studying differential equations. Pedagogical
Informatics, 4, 67-73.
Vlasova, E. A., Popov, V. S., & Pugachev, O. V. (2016). Creating the fund of assessment tools in the
study of linear algebra with using MATLAB. Bulletin of Moscow Region State University. Series:
Physics and Mathematics, 4, 77-85. doi: 10.18384/2310-7251-2016-4-77-85.
Weisstein, E. (2014). Computable data, Mathematics, and digital libraries in Mathematica and Wolfram
Alpha. In: Watt S. M., Davenport J. H., Sexton A. P., Sojka P., Urban J. (Eds.), Intelligent computer
mathematics. Lecture notes in computer science, vol 8543. Springer, Cham. doi: 10.1007/978-3-
319-08434-3_3.
Zeldin, A. L., & Pajares, F. (2000). Against the odds: Self-efficacy beliefs of women in mathematical,
scientific, and technological careers. American Educational Research Journal, 37 (1), 215-246.
doi: 10.3102/00028312037001215
Zhan, Z., Fong, P., Mei, H., & Liang, T. (2015). Effects of gender grouping on students’ group performance,
individual achievements and attitudes in computer-supported collaborative learning. Computers
in Human Behavior, 4, 587-596. doi: 10.1016/j.chb.2015.02.038.
Received: February 27, 2019 Accepted: June 14, 2019
Natalia M. Mezhennaya Candidate of Physics and Mathematics, Associate Professor, Department

(Corresponding author) of Applied Mathematics, Bauman Moscow State Technical University, 2-nd
Baumanskaya Street, 5, Moscow 105005, Russian Federation.
E-mail: natalia.mezhennaya@gmail.com
ORCID iD: 0000-0003-1426-3613
Oleg V. Pugachev Doctor of Physics and Mathematics, Professor, Department of Applied

Mathematics, Bauman Moscow State Technical University, 2-nd Baumanskaya
st., 5, Moscow 105005, Russian Federation.
E-mail: opugachev@yandex.ru
ORCID iD: 0000-0003-2149-5738

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
396
EFFICIENCY OF PUBLIC SPENDING
ON HIGHER EDUCATION: A DATA
ENVELOPMENT ANALYSIS FOR EU-28
Maja Mihaljevic Kosor, Lena Malesevic Perovic, Silva Golem
University of Split, Croatia
E-mail: majam@efst.hr, lena@efst.hr, sgolem@efst.hr
Abstract
One of the main goals of education policy is to enhance educational outcomes. If resources are used
inefficiently, they will fail to maximise those outcomes. Data Envelopment Analysis was used to calculate
technical efficiency of public spending on education for EU-28 using the latest higher education statistics
available. Focusing on European higher education, conceptual and methodological issues related to
the measurement and analysis of efficiency were discussed. The most efficient countries are identified
and also countries for which real efficiency improvements are possible. A novel set of variables is used
to highlight more appropriately the distinctiveness of the higher education sector and the relationship
between input and outputs. The advantage of using Data Envelopment Analysis is that it identifies the
best performing decision, making units and not the averages. This type of information about the efficiency
of public spending on education is of importance to many parties. It can be used to promote ‘yardstick’
competition in the areas of education where the lack of market mechanisms is apparent, guide policy
proposals, and to enhance the monitoring of education.
Key words: efficiency in education, higher education, public spending, data envelopment analysis,
European Union.
Introduction
Higher education (HE) plays an essential role in an economy, by promoting economic

growth, increasing productivity and contributing to social cohesion. Many countries in the
world finance their higher education systems from public funds and the European Union (EU) is
not an exception. In the interest of accountability and in light of the economic crisis of 2008 it is
becoming crucial to evaluate the efficiency with which these funds are being utilised. Efficiency
concerns are even more serious when looking at some of the trends in European HE.
In the EU-28 there has been a steady increase in the number of students in tertiary
education. According to Eurostat (2017) in 2015 there were 19.5 million students in tertiary
programmes and around 4.7 million students graduating from tertiary education. The share of
persons aged 30-34 with completed tertiary education increased, from 23.6% in 2002, to 39.1%
in 2016 in EU-28 (Eurostat, 2017). Germany had the highest number of tertiary education
student in 2015, about 3 million, or 15.2% of EU-28 total. Following Germany is France with
12.4% of EU-28 total, the United Kingdom (11.95%), Spain (10.1%), Italy (9.4%) and Poland
(8.5%). In all EU countries the majority of students were enrolled on Bachelor’s degrees. This
is especially the case for Greece where 84.4% of students were studying for Bachelor’s degrees.
One of the targets of the Europe 2020 strategy is to have at least 40% of persons aged 30-34
with completed higher education by 2020.
Most of the EU countries have a significant share of their HE financed from public
sources. In the EU-28 (excluding Greece and Croatia), public expenditure on tertiary education

Maja MIHALJEVIC KOSOR, Lena MALESEVIC PEROVIC, Silva GOLEM. Efficiency of public spending on higher education: A data
envelopment analysis for EU-28
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
was corresponding to 1.3% of GDP in 2014 (Eurostat, 2017). This ranged from 0.5% of GDP 397
in Luxembourg, to 1.3% in Germany and 2.4% in Denmark in 2014. With above mentioned
increases in the number of people participating in HE and the already stretched out public
resources, the issues of efficiency, effectiveness and accountability are being evoked more often
in discussions and national planning documents.
In many EU Member States there is a growing sentiment that the existing systems of
higher education are not organized efficiently and a large number of empirical studies to date
have attempted to define and measure this (in)efficiency in an HE framework. These studies
used a variety of techniques to identify ‘efficient’ decision-making units (DMU) (students,
HEIs, departments, universities, countries) and compare them with ‘inefficient’ ones.
The efficiency concepts currently dominant in economics originate from engineering
relations where a technical process is considered efficient when the desired mix of outputs is
maximised for a given level of inputs or when inputs are minimised for a desired mix of outputs.
When transposed to the field of economics (of education) it seems there is no general consensus
about how to define and measure efficiency. The foundation of the theory of efficiency and
its measurement was laid out by Farrell (1957) who used three main measures of efficiency
(overall, technical and price efficiency). Technical efficiency was defined as the ability of a
firm to achieve maximum output from a given set of inputs. Fried, Lovell and Schmidt (2008)
provided a more detailed analysis of the efficiency concepts in neoclassical economics.
With the development of non-parametric techniques in the 1970s such as Data
Envelopment Analysis (DEA) (Banker, Charnes, & Cooper, 1984; Charnes, Cooper, & Rhodes,
1978), Stochastic Frontier Analysis (SFA) (Aigner, Lovell, & Schmidt, 1977), and others there
has been a thriving literature on efficiency across various settings, including education. A
recent survey of this general literature focusing on the most widely used method, DEA, can be
found in Emrouznejad and Yang (2017). Surprisingly, only about 3.5% of studies using Data
Envelopment Analysis were dedicated to the higher education issues and even smaller share is
dedicated to cross-country comparisons (noted in Thanassoulis et al., 2016).
Research Problem
Given the importance of higher education and the limited research in this area, the
focus is the analysis of efficiency in the context of higher education. Using the non-parametric
technique of Data Envelopment Analysis and the latest available data, technical efficiency
of public spending on higher education was estimated for EU-28. A novel set of variables
was used to highlight more appropriately the distinctiveness of the HE sector and the link
between input and outputs. The advantage of using DEA is that it identifies the best performing
DMUs and not the averages. In today’s competitive economies, countries cannot afford to have
average education systems. Technical efficiency estimates serve to inform education policy-
makers about the effects of changes in the production of educational services on outputs, hence
they can learn more about the consequences of different procedures and methods in education.
Without such information they would solely rely on past practices and traditional approaches.
As emphasised in Pausits and Pellert (2007), with the increasing size of HE the complexity
of the system increases and HEIs are themselves becoming more performance oriented and
specifying goals and strategies to achieve them. Hence, examining the efficiency of an HE
system is of great importance as a means of obtaining relevant information about the way HE
functions.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
Measuring Efficiency in Higher Education: General Background
The higher education system has important characteristics separating it from other levels
of education. In HE there are multiple stakeholders, multiple objectives and multiple outputs
(Dixit, 2002). Contemporary HEIs are diverse, have multiple inputs and carry out a number of
activities sometimes extending further than the teaching and research work (Cohn & Cooper,
2004). Furthermore, most educational outcomes are not sold at market prices, thus making
it difficult to attach a market value to these outcomes. These distinguishing features of HE
need to be taken into consideration when estimating the production function and measuring
its efficiency. An overview of literature on the efficiency of various levels of education can be
found in Johnes, Portela and Thanassoulis (2017) and a focus on efficiency in HE in Mihaljevic
Kosor (2013).
The term production function can simply be defined as producing the maximum output
feasible with given inputs. In a mathematical form a production function illustrates how a DMU
generates a vector of outputs using a flow of inputs and some available technology. When
estimating efficiency in HE, several distinctive techniques have developed in the literature.
Their main features are briefly discussed, and the main approach is introduced. Three most
widely used approaches in efficiency estimation are the statistical, econometric approach that
mostly uses regression analysis, the Stochastic Frontier Estimation and the DEA.
A statistical approach is often parametric and assesses how DMUs produce educational
outcomes analogous to firms producing outputs. Economic criteria from the neoclassical theory
of the firm (Baumol, Panzar, & Willig, 1982) are applied to model educational production. This
method provides estimates of parameters whose significance can be tested. However, there
are problems with model misspecification and, more importantly, this approach cannot handle
multiple inputs and outputs. That is a serious shortcoming when estimating HE efficiency.
The Stochastic Frontier Analysis was pioneered by Aigner et al. (1977). Here a functional
form is established between the set of explanatory variables and the dependent variable. The
analysis provides parametric estimates of efficiency. The main difference between SFA and
the traditional parametric regression is that the error term in SFA consists of two parts: a
normally distributed error term, and a second term capturing the remaining error (i.e. technical
inefficiency). Given the presence of normally distributed error terms, the tools of statistical
inference can be employed which is seen as an advantage of this approach. However, a particular
distributional form for the error terms needs to be imposed by the researcher that gives rise to
misspecifications in the efficiency measure. Additionally, SFA uses data on costs and prices
according to which may introduce additional measurement errors (Worthington, 2001). Finally,
SFA is not easily extended for multiple input and output settings.
The third approach in estimating efficiency is the DEA. It was originally developed for
efficiency evaluation of ‘not-for-profit entities participating in public programs’ where prices
are not clearly observed (Charnes et al., 1978). It is a non-parametric method which assigns
a set of weights to selected outputs and inputs. Efficiency estimates are then obtained as the
maximum of a ratio of weighted outputs to weighted inputs, subject to certain restrictions such
as monotonicity and convexity. This approach amounts to constructing an efficiency frontier
over the data so that the actual input/output quantities are either on or in the interior of this
frontier. The efficiency frontier outlines the maximum combinations of outputs that can be
produced by a given set of inputs. DEA assigns a score of 1 to a DMU which lies on the frontier.
That unit in comparison with other units shows no evidence of inefficiency. An efficiency score
less than 1 implies that a linear combination of other DMUs from the sample could generate the
same output using less inputs. However, the method does have some downsides.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
Being a nonparametric technique, statistical inference cannot be used to examine the 399
possible bias resulting from measurement error or omitted variables. Furthermore, DEA only
estimates relative efficiency scores but cannot offer insights on the performance of DMUs in
comparison to the global best-practice. Some of the advantages of DEA are that it can handle
multiple inputs and multiple outputs, it requires no assumptions on the functional form linking
inputs to outputs and the DMUs are directly compared to one another. All of this makes it
an appealing choice for measuring the efficiency in HE. Comparing these three different
approaches, DEA stands out as a valuable tool in measuring efficiency in an HE setting as can
be noted from the following research on efficiency in HE.
Sample
Existing studies on efficiency in HE have focused mostly on specific countries and

their HEIs as the main DMUs (some of the exceptions are Johnes 2006a; Barra & Zotti 2016;
Thanassoulis et al. 2017). The UK has a remarkably long tradition in the efficiency analysis
of HE. See, among other studies, Athanassopoulos and Shale (1997), Glass, McKillop and
Hyndman (1995), Johnes (2006), Sarrico, Hogan, Dyson and Athanassopoulos (1997), Sarrico
and Dyson (2000). Most of these studies have shown the efficiency of the UK’s HEIs to be
quite high. Australian HE system and its universities have also been widely examined (see in
Abbott & Doucouliagos, 2003; Worthington & Lee, 2008). Studies on efficiency in HE can
also be found for Italy (Abramo, D’Angelo, & Pugini 2008; Agasisti & Dal Bianco, 2006),
Germany (Kempkes & Pohl, 2010) and more recently for Greece (in Thanassoulis, Kumar,
Petridis, Goniadis, & Georgiou, 2017).
Only a small number of studies analyses HE efficiency for several European countries.
Some of the examples are Joumady and Ris (2005) for 209 HEIs in eight European countries,
Bonaccorsi, Daraio & Simar (2007) for six European countries (Italy, Spain, Portugal, Norway,
Switzerland, and the UK), Agasisti and Johnes (2009) for Italy and England, Agasisti (2011) for
18 countries, and finally, Wolszczak-Derlacz and Parteka (2011) for seven European countries
(Austria, Finland, Germany, Italy, Poland, Switzerland and the UK).
All of the above studies focus either on country specific data or use a small sample of
DMUs in cross-country comparisons. This is understandable given the problem of obtaining
micro data on HE that can then be used for country comparison. The DMUs are mostly
universities or HE staff and not the whole country. The exception is the study by Agasisti (2011)
who used countries as DMUs enabling an assessment of the overall performance of the HE
system and their cross-country comparison. Some of the variables used in that research are the
total resources for HEIs as a percentage of GDP – this is the input variable. And for the outputs,
the author used employment rates for the population aged 25-64 and tertiary graduation rates
(which were later dropped from the model due to missing data). The variables for DEA are for
a selection of European countries in the period from 2000-2003. A similar approach is followed
here.
DEA is used in measuring the efficiency of public spending on higher education.

It identifies optimally performing DMUs and assigns them a score of one (or 100). These
DMUs serve to identify an efficiency frontier against which all other DMUs are compared.
A fundamental step in DEA is the selection of appropriate inputs and outputs (Thanassoulis,
2001). These variables need to satisfy the condition of exclusivity and exhaustiveness i.e.
the inputs and they alone must influence the output levels, and only of the outputs used in
estimation. Furthermore, to have sufficient discriminatory power, the number of DMUs should

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
400 exceed the number of inputs and outputs by more than a few times in the DEA model (Cohn &
Cooper, 2004). The DMU’s used in this research are the 28 countries of the European Union.
The efficiency frontier in DEA will vary conditional on the scale assumptions of the
model. Two scale assumptions are commonly used in DEA: constant returns to scale (CRS) and
variable returns to scale (VRS).
Variable returns to scale assumption allows the production technology to exhibit
increasing, decreasing and constant returns to scale. Given the nature of HE, i.e. that outputs will
not change by the same proportion as inputs, VRS is better suited to examining the efficiency.
Furthermore, an output orientation approach is used which is also common when examining
efficiency in education. The output orientation is a logical choice given the nature of higher
education financing. The objective is to maximise the output production while not exceeding
the actual input level.
Data Analysis
One input and two outputs related to country’s higher education system are used (Table
1). The input used is the public expenditures on tertiary education as a percentage of GDP. This
serves as a proxy for all inputs in public HE and allows easier cross-country comparisons as
discussed in the research by Afonso and St Aubyn (2005, 2006). This indicator is available for
all countries in the sample. However, countries may still have a different education production
technology that is not best expressed through the use of public expenditures. However, given
to data limitations on the more precise indicators of education production in each country this
indicator was used as an input, hence focusing on EU-28 there is a collection of countries with
similar policy objectives in HE.
The two outputs are the share of graduates in HE per 1000 inhabitants and the employment
rates of people aged 20-34 with completed tertiary education. In comparison to the work by
Agasisti (2011) this is a valuable improvement in data availability. The output variable on
employment rates is available for population aged 20-34, which gives a better indication of the
short-term effects of HE. These two indicators serve as a proxy for the quality of HE delivered
and capture, to some, extent, private returns to HE.
Table 1. Variable names and definitions.
Variable Category Description Year

Public expenditures Public expenditures on tertiary education as a
Input 2012
(%GDP)* percentage of GDP
Graduates in HE (per 1000 Graduates in tertiary education per 1000 of

Output 2015
of population) population
Employment rates of young people not in educa-

Employment rates tion and training with completed tertiary education
Output 2016
(%) level one to three years beforehand aged 20 to
34
The data is from Eurostat. For the two outputs the latest data available is used. This
allowed a fresh perspective to be taken on the efficiency of European HE. For the share of
graduates the latest data available was for the year 2015, and for employment rates the data is
for 2016.
For the input variable, public expenditures, the data is for 2012. The motivation is the
following. Public expenditures being spent on HE (including students) in one year are assumed

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
to have a full effect when that cohort of students graduates or looks for employment several 401
years after. Here the same cohort of student is followed, as they progress through their studies
towards graduation and employment. In all EU countries in 2015, the majority of students were
enrolled in Bachelor’s degrees (Eurostat, 2017). The term cohort is used loosely; it actually
follows the whole HE system being funded in one year and its outputs several years later.
This approach is drawing some of its insight from the window analysis technique in DEA
pioneered by Charnes et al. (1997) which measured efficiency changes over time. Given the
characteristics of variables that change very slowly over time, the above mentioned approach
was used. This allows a stronger functional link to be established in the model between the
input and the outputs.
Research Results
The model was estimated using the Performance Improvement Management Software
(PIMDEA). Summary statistics for variables is presented in Table 2.
Table 2. Summary statistics.
Mean Sum St.dev. Variance Min Max Range

Public expenditures (%GDP) 1.26 35.22 0.44 0.2 0.45 2.25 1.8
Graduates in HE
72.5 2030 19.22 369.4 27.4 126.8 99.4
(per 1000 pop)
Employment rates
82.62 2313.3 9.39 88.18 55 97 42
(%)
The highest share of public expenditures on HE was recorded in Denmark (2.5% GDP),
followed closely by Finland (2.13%) and Sweden (2.01%). These are the only countries with
public expenditures on HE over 2% of GDP. The lowest public expenditures are in Luxembourg
(0.45%) and a fairly better situation is in Bulgaria (0.66%), Romania (0.78%), Hungary
(0.82%), Italy (0.83%) and Portugal (0.85%). The latter group is formed of countries with
public expenditures less than 1% of GDP. The average public expenditures on HE in the EU in
2012 are 1.26%.
The greatest number of graduates from tertiary education per 1000 of population is in
Ireland (126.8), followed by Denmark (102.9), France (96.5) and Poland (96). However, if only
the data on the number of graduates from tertiary education is used, Ireland and Denmark are
not the leading countries. The greatest population of tertiary graduates is in France (752,068
graduates), the UK (740,276) and Poland (516,675). The smallest number of graduates per
1000 inhabitants is in Luxembourg.
The employment rates for recent graduates are calculated for the age group 20–34. It
captures those who had successfully completed their highest level of education one to three
years previously. The minimal employment rate is in Greece (55%) and the highest in Malta
(97%), followed by the Netherlands (94.2) and Germany (93.1).
The results of the DEA model are in Table 3.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
402 Table 3. DEA results.
Country Technical efficiency

Austria 95.62
Belgium 93.78
Bulgaria 100
Croatia 85.3
Cyprus 82.53
Czech Republic 95.72
Denmark 96.03
Estonia 82.9
Finland 87.46
France 87.51
Germany 90.14
Greece 59.5
Hungary 100
Ireland 100
Italy 73.39
Latvia 96.78
Lithuania 97.37
Luxembourg 100
Malta 100
Netherlands 98.24
Poland 98.17
Portugal 89.35
Romania 90.37
Slovakia 93.89
Slovenia 88.27
Spain 87.42
Sweden 94.23
United Kingdom 94.61
The average technical efficiency of public expenditures on HE for EU-28 was 91.4%.
Five countries stand out as 100% efficient. These are Bulgaria, Hungary, Ireland, Luxembourg
and Malta. It might be argued that Luxemburg and Malta stand out as efficient because of their
size and that perhaps they should be dropped from the analysis. It was decided not to remove
them from the sample to report a complete picture of efficiency in public spending for EU-28.
The Netherlands, Poland, Latvia, Lithuania and Denmark are close to the efficiency frontier
with an efficiency score over 96%.
The least efficient country was Greece (59.5). Given the recent economic crisis in Greece
this inefficiency in public spending was an expected result. As previously mentioned, to achieve
full efficiency in an output-oriented model, an inefficient unit may focus on an increase in its

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
outputs while the input proportions remain the same. For Greece this would imply an increase 403
in the share of graduates and an improvement in the employment rates.
These results of DEA provided a form of ranking for countries, from best to worst. This
ranking is easier for inefficient countries. However, the countries on the efficiency frontier all
have the same efficiency score equal to 1 (or 100) and it is difficult to establish which countries
stand out as best examples.
In the next section the benchmarks were considered for each of the individual countries.
These are the countries that can potentially serve as role models for inefficient countries. These
results are in Table 4.
Table 4. Benchmarks.
Name Bulgaria Hungary Ireland Luxembourg Malta

(Frequencies) 5 12 22 7 18
Austria 0 0 1 0 1
Belgium 0 0 1 0 1
Bulgaria 1 0 0 0 0
Croatia 1 0 1 1 0
Cyprus 0 1 1 0 1
Czech Republic 0 1 1 0 1
Denmark 0 0 1 0 1
Estonia 0 1 1 0 1
Finland 0 0 1 0 1
France 0 1 1 0 1
Germany 0 1 0 1 1
Greece 0 0 1 0 1
Hungary 0 1 0 0 0
Ireland 0 0 1 0 0
Italy 1 0 1 1 0
Latvia 0 1 1 0 1
Lithuania 0 1 1 0 1
Luxembourg 0 0 0 1 0
Malta 0 0 0 0 1
Netherlands 0 0 1 0 1
Poland 0 1 1 0 1
Portugal 1 0 1 1 0
Romania 0 1 1 1 0
Slovakia 0 1 1 1 0
Slovenia 0 1 1 0 1
Spain 1 0 1 0 0
Sweden 0 0 0 0 1
United Kingdom 0 0 1 0 1

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
404 The efficient countries are in the first row and all of the other countries are compared to
them. Efficient countries are their own “benchmarks” while inefficient countries have several
benchmarks. For example, Romania has Hungary, Ireland and Luxembourg as benchmarks,
while Austria has Ireland and Malta as best practice examples or peers. This means, to become
efficient Austria must use a combination from both Ireland and Malta. Those two countries
have the highest frequencies in the table as they are peers to 22 and 18 countries respectively
(including themselves).
The next issue is how to calculate what combination of efficient peers must an inefficient
country use to become efficient. So, for Austria the question would be, should it attempt to
‘become more’ like Ireland or more like Malta. The answer is in the λ (lambda) weights. For
example, Austria should look up more to Malta (λ=0.77) than to Ireland (λ=0.23). These results
uncovered interesting connections between countries that would merit a more detailed analysis.
That is, however, out of scope, hence these results can be obtained from the authors.
The next goal was the estimation of efficient targets for input and outputs. In Table
6 target input and output levels are suggested for each country. These targets would allow
countries to gain full efficiency. However, some of these target values and improvement options
may not be practical for policy makers in HE. They are presented here only to offer more insight
into the efficiency of public spending on HE and as a source for potential future research in the
area.
Table 5. Efficient targets for input and outputs.
Input Output Output

Public spending Graduates in HE Employment rates
Gain
Value Target Value Target Gain (%) Value Target Gain (%)
(%)
Austria 1.88 1.46 -22.4 75.6 79.06 4.58 90.5 94.64 4.58
Belgium 1.48 1.44 -2.51 79.3 84.56 6.63 87.9 93.73 6.63
Bulgaria 0.66 0.66 0 70.3 70.3 0 78.5 78.5 0
Croatia 0.93 0.93 0 70.4 82.54 17.24 74.7 87.58 17.24
Cyprus 1.17 1.17 0 57.6 69.79 21.17 76.4 92.57 21.17
Czech Republic 1.05 1.05 0 74.3 77.62 4.47 86.3 90.16 4.47
Denmark 2.25 1.38 -38.79 102.9 107.15 4.13 86.4 89.97 4.13
Estonia 1.06 1.06 0 59.4 71.65 20.62 75.5 91.07 20.62
Finland 2.13 1.41 -33.74 83.4 95.36 14.34 80.4 91.93 14.34
France 1.27 1.27 0 96.5 110.27 14.27 77.3 88.33 14.27
Germany 1.37 1.37 0 55.9 62.02 10.94 86.4 95.85 10.94
Greece 1.48 1.42 -4.03 54.9 92.28 68.08 55 92.44 68.08
Hungary 0.82 0.82 0 55.3 55.3 0 90.5 90.5 0
Ireland 1.32 1.32 0 126.8 126.8 0 86.7 86.7 0
Italy 0.83 0.83 0 58.2 79.31 36.26 61.3 83.53 36.26
Latvia 1.01 1.01 0 63.2 65.3 3.33 88.4 91.34 3.33
Lithuania 1.4 1.4 0 80 82.16 2.7 91.1 93.56 2.7
Luxembourg 0.45 0.45 0 27.4 27.4 0 89 89 0
Malta 1.5 1.5 0 64.9 64.9 0 97 97 0
Netherlands 1.7 1.48 -12.9 70.3 71.56 1.8 94.2 95.89 1.8

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
Poland 1.15 1.15 0 96 97.79 1.86 87 88.62 1.86 405
Portugal 0.85 0.85 0 66.8 74.76 11.92 77.8 87.07 11.92

Romania 0.78 0.78 0 52.7 58.31 10.65 80.7 89.3 10.65
Slovakia 0.95 0.95 0 78.3 83.4 6.51 82.5 87.87 6.51
Slovenia 1.23 1.23 0 77.5 87.8 13.29 80.2 90.86 13.29
Spain 1.02 1.02 0 88.4 101.12 14.39 72.3 82.97 14.76
Sweden 2.01 1.5 -25.37 59 64.9 10 91.4 97 6.13
United King-
1.47 1.43 -2.83 84.7 89.52 5.69 87.9 92.9 5.69
dom
In a VRS, output-oriented model the objective was to maximize the output production
(tertiary graduates and employment rates) while not exceeding the actual input level (public
spending). DEA calculated for each country the amount and category that needs to change in
order for country to reach efficiency frontier. For inefficient countries an increase in outputs
is suggested. In some cases, a decrease in public spending is also an option. All the efficient
countries have the same targets and values for input and outputs and gain equal to zero. Countries
that were far from the efficiency frontier (Greece, Italy, Cyprus, Estonia, Croatia) have to make
substantial changes in their outputs in order to become efficient. For example, Greece will need
to increase graduates and employment rates by 68 percentage points and Italy by 36 percentage
points. Countries which were close to the frontier (e.g. Netherlands, Poland, Lithuania, Latvia)
have a smaller magnitude of changes suggested. Only 1.9 percentage point increase in outputs
is suggested for Poland, and a 2.7 percentage points increase in outputs for Lithuania.
Discussion
Although there is a growing research on efficiency of public spending on HE there are

not many comparable approaches in estimating it. The basis for this analysis, as clarified in the
section on methodology, was the empirical work found in Agasisti (2011) who used countries
of the EU as the decision making units. The obtained results are similar to the ones obtained in
that research. However, it should be noted that the variables used in DEA are not the same i.e.
there is a slight difference in the outputs used and the time period. Furthermore, the estimation
by Agasisti was performed for 18 countries of the EU. Therefore, any similarities should be
discussed with caution as the DEA is very sensitive to the selection of inputs and outputs
(Thanasoulis et al., 2016).
In general, the technical efficiency of public expenditures on HE is high (91.4%), however
there are noticeable differences among countries as in the research by Agasisti (2011). The least
efficient country is Greece. This is in line with the expectations given the external context
i.e. the economic situation and the financial crisis, including Greece in the model reduced the
overall efficiency score when compared to the results in Agasisti (2011).
When efficient targets for inputs and outputs are considered a decrease in public spending
is suggested for eight countries (Austria, Belgium, Denmark, Finland, Greece, Netherlands,
Sweden and the UK). A detailed formulation for the calculation of these target values can be
found in Thanassoulis and Dyson (1992). An interesting example was Denmark; with a cut in
public spending of almost 40 percent and an increase of 4 percentage points in employment rates
and share of graduates, it could achieve full efficiency. Bearing in mind the limitations of DEA,
in the time of economic crisis this might present an interesting issue for further investigation.
As Monk (1992) argues, the production function can be viewed as a model which connects
theoretically and mathematically outcomes, inputs, and the processes that convert the latter

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
406 into the former in educational institutions, thus, production functions can be important as a
means of identifying ways of improving both technical and allocative efficiencies. With more
information on performance and efficiency, policymakers can be in a better position to develop
effective HE strategies. Nevertheless, the key responsibility for generating reforms in HE lies
with individual Member States.
Conclusions
From the overview of the research in this area there is still a lot of room for improvement.
There are often data limitations preventing researchers from making functional cross-country
comparisons. There are also well-documented limitations of methods being used to calculate
efficiency i.e. DEA is good at estimating relative efficiency of DMUs, but it converges slowly
to total efficiency, it is also very sensitive to changes in the data, and the hypotheses testing
is still not available. Although great care was taken in the selection of variables, and current
literature was consulted, measuring the efficiency of public spending on HE is still a challenging
undertaking. Results in this research need to be interpreted with caution.
Using DEA and the latest data available, this research compared the efficiency of public
spending for 28 European countries. The average efficiency in spending is high, although there
are stark differences among countries in their efficiency scores. Five most efficient countries
were identified (Bulgaria, Hungary, Ireland, Luxembourg and Malta) and also countries for
which real efficiency improvements were possible. Benchmark countries and target values also
revealed new approaches to deal with the inefficiencies.
Efficiency analysis is valuable in informing policy makers and providing a better
understanding of the education system. Although there are severe problems in estimating
efficiency in an HE system, it remains important to develop reliable estimates, especially
considering the emphasis currently given to issues of accountability, quality and costs. European
HE system needs to contribute to Europe’s prosperity. Finding the most productive benchmarks
and eliminating waste in resources dedicated to HE represents a good starting point.
Acknowledgements
This work was fully supported by the Croatian Science Foundation under Grant UIP-
2013-11- 9558.
Note
The authors would also like to thank the participants of the 15th International Conference
on Data Envelopment Analysis (held in Prague), where an earlier version of this research was
presented and its abstract published.
References
Abbott, M., & Doucouliagos, C. (2003). The efficiency of Australian universities: A data envelopment
analysis. Economics of Education Review, 22 (1), 89–97. doi: 10.1016/S0272-7757(01)00068-1.
Abramo, G., D’Angelo, C. A., & Pugini, F. (2008). The measurement of Italian universities’ research
productivity by means of non-parametric-bibliometric methodology. Scientometrics, 76 (2), 225–
244. doi: 10.1007/s11192-007-1942-2.
Afonso, A., & St. Aubyn, M. (2005). Non-parametric approaches to education and health efficiency in
OECD Countries. Journal of Applied Economics, 8 (2), 227-246. doi:10.1080/15140326.2005.1
2040626.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
Afonso, A., & St. Aubyn, M. (2006). Cross-country efficiency of secondary education provision: A semi- 407
parametric analysis with non-discretionary inputs. Economic Modelling, 22 (3), 476-491. doi:
10.1016/j.econmod.2006.02.003.
Agasisti, T. (2014). The efficiency of public spending on education: An empirical comparison of EU
countries. European Journal of Education, 49, 543–557. doi:10.1111/ejed.12069.
Agasisti, T., & Johnes, G. (2009). Beyond frontiers: Comparing the efficiency of higher education
decision-making units across more than one country. Education Economics, 17 (1), 59-79. doi:
10.1080/09645290701523291.
Agasisti, T., & DalBianco, A. (2006). Data envelopment analysis to the Italian university system:
Theoretical issues and political implications. International Journal of Business Performance, 8
(4), 344–367. doi: 10.1504/IJBPM.2006.009613.
Agasisti, T. (2011). Performances and spending efficiency in higher education: A European
comparison through non‐parametric approaches. Education Economics, 19 (2), 199-224. doi:
10.1080/09645290903094174.
Aigner, D., Lovell K, & Schmidt, P. (1977). Formulation and estimation of stochastic frontier production
function models. Journal of Econometrics, 6 (1), 21-37. doi: 10.1016/0304-4076(77)90052-5.
Athanassopoulos, A. D., & Shale, E. (1997). Assessing the comparative efficiency of higher education
institutions in the UK by means of data envelopment analysis. Education Economics, 5 (2), 117-
134. doi: 10.1080/09645299700000011.
Banker, R.D., Charnes, A., & Cooper, W. (1984). Models for estimation of technical and scale inefficiencies
in data envelopment analysis. Management Science, 30, 1078-1092. doi: 10.1287/mnsc.30.9.1078.
Baumol, W., Panzar, J., & Willig, R. (1982). Contestable markets and the theory of industry structure.
New York: Harcourt Brace Jovanovich.
Bonaccorsi, A., Daraio, C., & Simar, L. (2007). Efficiency and productivity in European universities:
Exploring trade-offs in the strategic profile, Chap. 5. In A. Bonaccorsi & C. Daraio (Eds.).
Universities and strategic knowledge creation: Specialization and performance in Europe,
Cheltenham/Northampton, MA: Edward Elgar Publishing, 144–206.
Charnes, A., Cooper, W. W., & Rhodes, E. (1978). Measuring the efficiency of decision making units.
European Journal of Operational Research, 2, 429-444. doi: 10.1016/0377-2217(78)90138-8.
Cohn, E., & Cooper, S. (2004). Multiproduct cost functions for universities: Economies of scale and
scope, in Johnes, G., & Johnes, J. (Eds.), International Handbook on the Economics of Education,
Cheltenham: Edward Elgar.
Barra, C., & Zotti, R. (2016). Managerial efficiency in higher education using individual versus aggregate
level data. Does the choice of decision making units count? Managerial and Decision Economics,
37 (2), 106–126. doi: 10.1002/mde.2697.
Dixit, A. (2002). Incentives and organizations in the public sector: An interpretative review. Journal of
Human Resources, 3 (4), 696-727. doi: 10.2307/3069614.
Emrouznejad, A., & Yang, G. (2018). A survey and analysis of the first 40 years of scholarly literature in
DEA: 1978–2016. Socio-Economic Planning Sciences, 61, 4-8. doi: 10.1016/j.seps.2017.01.008.
Eurostat (2017). Tertiary Education Statistics. Retrieved 15/06/2018, from http://ec.europa.eu/eurostat/
statistics-explained/index.php/ Tertiary_education_statistics.
Farrell, M. (1957). The measurement of productive efficiency. Journal of the Royal Statistical Society
(Series A), 120 (3), 253-281. doi: 10.2307/2343100.
Fried, H.; Lovell, K. & Schmidt S. (2008). Measurement of productive efficiency and productivity growth.
New York: Oxford University Press.
Glass, J. C., McKillop, D. G., & Hyndman, N. S. (1995). Efficiency in the provision of university teaching
and research: An empirical analysis of UK universities. Journal of Applied Econometrics, 10,
61–72. doi: 10.1002/jae.3950100106.
Johnes, J. (2006a). Measuring teaching efficiency in higher education: An application of data envelopment
analysis to economics graduates from UK universities 1993. European Journal of Operational
Research, 174 (1), 443-456. doi: 10.1016/j.ejor.2005.02.044.
Johnes, J. (2006b). Data envelopment analysis and its application to the measurement of efficiency in higher
education. Economics of Education Review, 25, 273–88. doi: 10.1016/j.econedurev.2005.02.005.
Johnes, J. (2006c). Measuring the efficiency of universities: A comparison of multilevel modelling and
data envelopment analysis. Bulletin of Economic Research, 58 (2), 75–104. doi: 10.1111/j.0307-
3378.2006.00238.x.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
408 Johnes, J., Portela, M., & Thanassoulis, E. (2017). Efficiency in education. Journal of the Operational
Research Society, 68 (4), 331–338. doi: 10.1057/s41274-016-0109-z.
Joumady, O., & Ris, C. (2005). Performance in European higher education: A non-parametric production
frontier approach. Education Economics, 13 (4), 189–205. doi: 10.1080/09645290500031215.
Kempkes, G., & Pohl, C. (2010). The efficiency of German universities: Some evidence from nonparametric
and parametric methods. Applied Economics, 42, 2063–2079. doi: 10.1080/00036840701765361.
Mihaljevic Kosor, M. (2013). Efficiency measurement in higher education: Concepts, methods and
perspective, Procedia - Social and Behavioral Science, 106, 1031-1038. doi: 10.1016/j.
sbspro.2013.12.117.
Monk, D. (1992). Education productivity research: An update and assessment of its role in
education finance reform. Educational Evaluation and Policy Analysis, 14 (4), 307-332. doi:
10.3102/01623737014004307.
Pausits, A., & Pellert, A. (Eds.) (2007). Higher education management and development in Central,
Southern and Eastern Europe. Germany: Waxmann Verlag.
Sarrico, C. S., Hogan, S.M., Dyson, R. G., & Athanassopoulos, A. D. (1997). Data envelopment analysis
and university selection. Journal of the Operational Research Society, 48, 1163–77. doi: 10.1057/
palgrave.jors.2600475.
Sarrico, C. S., & Dyson, R. G. (2000). Using data envelopment analysis for planning in UK universities
– An institutional perspective. Journal of the Operational Research Society, 51, 789–800. doi:
10.1057/palgrave.jors.2600949.
Thanassoulis, E., & Dyson R.G. (1992). Estimating preferred target input-output levels using data
envelopment analysis. European Journal of Operational Research, 56 (1), 80-97. doi:
10.1016/0377-2217(92)90294-J.
Thanassoulis, E., Kumar, P., Petridis, K., Goniadis, I., & Georgiou, A. (2017). Evaluating higher education
teaching performance using combined analytic hierarchy process and data envelopment analysis.
Journal of the Operational Research Society, 68 (4), 431–445. doi: 10.1057/s41274-016-0165-4.
Thanassoulis, E., De Witte, K., Johnes, J., Johnes, G., Karagiannis, G., & Portela, C.S. (2016). Data
envelopment analysis. New York: Springer Science and Business Media.
Thanassoulis, E. (2001). Introduction to the theory and application of data envelopment analysis: A
foundation text with integrated software. US: Springer. doi: 10.1007/978-1-4615-1407-7.
Wolszczak-Derlacz, J., & Parteka, A. (2011). Efficiency of European public higher education institutions:
A two-stage multicountry approach. Scientometrics, 89 (3), 887–917. doi: 10.1007/s11192-011-
0484-9.
Worthington, A.C. (2001). An empirical survey of frontier efficiency measurement techniques in
education. Education Economics, 9 (3), 245-268. doi: 10.1080/09645290110086126.
Worthington, A.C., & Lee, B.L. (2008). Efficiency, technology and productivity change in Australian
universities, 1998–2003. Economics of Education Review, 27 (3), 285–298. doi: 10.1016/j.
econedurev.2006.09.012.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
409
Maja Mihaljevic Kosor PhD, Assistant Professor, University of Split, Faculty of Economics, Split, Cvite Fiskovica
(Corresponding author) 5, Split, Croatia.
E-mail: majam@efst.hr
Website: http://www.efst.unist.hr/o-fakultetu/fakultet/djelatnici/stranice-djelatnika/detalji/
majam
Lena Malesevic Perovic PhD, Associate Professor, University of Split, Faculty of Economics, Split, Cvite Fiskovica
5, Split, Croatia.
E-mail: lena@efst.hr
lena
Silvia Golem PhD, Assistant Professor, University of Split, Faculty of Economics, Split, Cvite Fiskovica
5, Split, Croatia.
E-mail: sgolem@efst.hr
sgolem

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
410
EXPLORING THE STUDENT AND SOCIAL
ACCOUNTABILITY OF THE LIFE SCIENCES
CURRICULUM: A CASE OF HIV/AIDS
Lindelani Mnguni
University of South Africa, South Africa
E-mail: mngunle@unisa.ac.za
Abstract
Science is generally expected to respond to students and societal needs by adopting student and social
accountability principles. Therefore, school science curricula are revised regularly to address emerging
socio-economic, political, and scientific issues. Similarly, the National Curriculum and Assessment Policy
Statement of South Africa was introduced to foster a student and social accountable education that adopts
relevant curriculum ideologies. Partly, it attempts to equip students with relevant skills and knowledge
related to emerging societal challenges such as HIV/AIDS. The aim of the present research, therefore,
was to determine the extent to which the curriculum is student and social accountable concerning HIV/
AIDS. The current research explored the extent to which HIV/AIDS knowledge was presented in the
CAPS Life Sciences curriculum and selected textbooks, and how curriculum ideologies informed this
presentation of HIV/AIDS knowledge. Data were collected from the Life Sciences CAPS document and
selected textbooks. Results revealed several HIV/AIDS-related topics, which were taught in Life Sciences.
While the literature suggests that the citizenship-centered ideology is most relevant for student and
social accountability, the present study found that the Life Sciences curriculum adopted a predominantly
discipline-centered ideology. It is concluded that Life Sciences may not provide students with HIV/AIDS-
related skills and knowledge required in a student and socially accountable curriculum.
Keywords: curriculum ideologies, HIV/AIDS education, life sciences, school science curriculum.
Introduction
Countries around the world regularly revise their education policies and school curricula
to respond to emerging socio-economic, political, and scientific issues (Hoeg & Bencze, 2017;
Pietarinen, Pyhältö, & Soini, 2017). Similarly, the dawn of democracy in South Africa has
seen school curriculum revisions which are aimed at aligning the education system with the
democratic constitution of the country to ensure that the curriculum is student accountable and
socially accountable. To this end, at least three major curriculum reforms have taken place,
including the adoption and revision of Curriculum 2005, National Curriculum Statement (NCS)
and the current Curriculum and Assessment Policy Statement (CAPS). The primary contention
behind these curriculum reforms can be traced back to Hebert Spencer’s question, “what
knowledge is of most worth,” particularly in the 21st century, and should, therefore, be included
in the curriculum (Spencer, 1896, p. 21). Dowden (2013, p. 2) suggests that the curriculum
should respond to “the needs of the individual student, the demands of wider society, and the
vested interests of subject areas” (Dowden, 2013, p. 2).
Consequently, some countries use the school science curriculum to respond to various
emergent socio-scientific issues. Christenson and Chang Rundgren (2015), for example,
proposed that curriculum could be used to facilitate students' argumentation skills using
content related to genetically modified organisms. Similarly, Atabey and Topcu (2017) used a
science curriculum to teach primary school learners about global warming. In some instances,

Lindelani MNGUNI. Exploring the student and social accountability of the life sciences curriculum: A case of HIV/AIDS
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
the school science curriculum is used to promote social justice, citizenship, and democracy 411
(Nuangchalerm & El Islami, 2018). Within the South Africa context, one of the significant
socio-scientific issues which has impacted significantly on the socio-economic, political and
scientific landscape is HIV/AIDS (Kharsany & Karim, 2016), which has led to the death of
thousands of citizens over the last three decades (Statistics South Africa, 2018). To this end,
HIV/AIDS is one of the significant social threats to South Africa in the 21st century. It is,
therefore, imperative that the curriculum should respond to HIV/AIDS by providing learners
with cutting-edge knowledge and skills required to deal with the socio-economic challenges
caused by HIV/AIDS. The extent to which this is the case currently has been addressed by
various scholars who suggest that the curriculum may not be playing a significant role in the
fight against HIV/AIDS (e.g., Mnguni & Abrie, 2012; Mnguni, Abrie, & Ebersohn, 2016). The
underlying cause of this misalignment between the curriculum and social needs has not been
extensively explored. The present research, therefore, examined the extent to which the school
curriculum is student and socially accountable by determining the curriculum ideologies that
foreground the presentation of HIV/AIDS knowledge.
Student and social accountability can be defined as the commitment to ensuring that
education including teaching, learning, and research are directed toward addressing existing
socio-economic and educational needs of students and societies (Lindgren & Karle, 2011).
A student and social accountable curriculum, therefore, would be one where the curriculum
ideology is student and citizenship-centered. A curriculum ideology refers to the underlying
values of the curriculum as reflected on the overarching purpose of education, the characterization
of the student and the teacher and their respective roles during teaching and learning as well as
the nature, function, and purpose of knowledge, instructional process, and assessment (Schiro,
2013).
While curriculum reform intentions may include student and social accountability, the
extent to which teaching and learning in school science is student and social accountability
remains to be explored in great detail, both in South Africa and other countries. For example,
Bird (2014) suggests that science should go beyond responsible conduct of research to include
macroethics, which provides for social responsibility of science and scientists as an ethical
standard. Macroethics is based on the “expectation that scientists will pay attention to the health,
safety, and welfare of the public and the environment” as reflected in the Uppsala Code of Ethics
for Scientists (Bird, 2014, p. 169). The extent to which school science responds to macroethics,
however, remains to be understood. Therefore, in light of the HIV/AIDS challenge in South
Africa, the present study sought to research the extent to which HIV/AIDS is addressed in the
school science curriculum, as a preliminary effort to understanding science’s responsiveness to
social issues.
Problem Statement and Rationale
HIV/AIDS was of interest in the present research because official reports suggest that in
2018, the prevalence of HIV in South Africa was 13.06% compared to 10.74% in 2008 (Statistics
South Africa, 2018). It is also reported that among women aged 15 to 49, approximately 23%
are living with HIV. Furthermore, 19% of adults aged between 15-49 are HIV positive. These
figures suggest that HIV/AIDS is one of the significant social threats for South Africa, which
must be addressed, amongst other means, through a student and socially accountable education.
However, the extent to which the current CAPS science curriculum can respond to the HIV/
AIDS challenges faced by the youth remains to be researched in great detail. It is on this basis
that the present research explores the ideology that informs the integration of HIV/AIDS in the
science curriculum.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
412 The concerns about the extent to which the curriculum is student and socially accountable
spiked recently when students engaged in what is locally known as #FeesMustFall protests. In
these protests students from universities and schools across the country called for rigorous
nation-wide decolonization and Africanisation of the curriculum across the entire education
system (Le Grange, 2016; Luescher, Loader, & Mugume, 2017). Such a curriculum, they
argued, would address students and society’s immediate socio-economic needs, including social
empowerment concerning everyday issues such as health, safety, and job creation. Subsequently,
education authorities and the government are currently engaged in various projects that seek to
explore the modalities related to possible curriculum reforms.
In light of the above discourse; therefore, the current researcher posits that the extent to
which the current South African school curriculum is student and socially accountable requires
urgent attention. Additionally, there is a need to explore the extent to which subject-specific
curricula address the needs of students and societies concerning everyday socio-economic
dynamics. Perhaps more specific to the present research is the need to explore the extent to
which the science curricula are student and socially accountable concerning socio-scientific
issues such as HIV/AIDS.
Findings of this research could necessitate a global effort in addressing student and
social accountability in science. Additionally, findings could contribute to discourses among
researchers, curriculum designers, and society in general about the role of science education in
social reform. Such discussions could lead to renewed interest in designing and implementing
curriculum and instructional design strategies for the empowerment of students and societies in
response to emerging global and local socio-scientific issues.
The Aim of the Present Research
The present research, therefore, sought to determine the extent to which the Life Science
is student and socially accountable. This was done by exploring the extent to which HIV/
AIDS knowledge is presented in the CAPS Life Sciences curriculum and textbooks, and how
curriculum ideologies inform this presentation of HIV/AIDS knowledge. The research question
framing this research asks: To what extent is the Life Sciences student and socially accountable
as reflected in the curriculum ideologies that inform the presentation of HIV/AIDS knowledge
in the CAPS curriculum and related textbooks?
Theoretical Framework
The present research adopted Mnguni’s (2018a,b) curriculum ideologies as a framework

for exploring the extent to which the Life Sciences curriculum is student and socially
accountable (table 1). As stated earlier, a curriculum ideology refers to the underlying values
of the curriculum as reflected on the overarching purpose of education, the characterization of
the student and the teacher and their respective roles during teaching and learning as well as the
nature, function and purpose of knowledge, instructional process and assessment (Schiro, 2013).
Mnguni (2018a,b) and Schiro (2013) suggest that there are at least four curriculum ideologies
that inform school curricula. These curriculum ideologies are the discipline-centered ideology,
service-centered ideology, student-centered ideology, and citizenship-centered ideology. As
suggested by scholars (e.g., Good, 1959; Schiro, 2013; Tanner & Tanner, 1987; Waks, 2003), a
curriculum should specify the subject content matter and its purpose, the instructional process,
the role of teachers, the role of students as well as the assessment process. These are therefore
uniquely defined in each of the four curriculum ideologies. To this end, the discipline-centered
ideology primary objective is to enhance the development of the discipline by transmitting
discipline-specific ontology and epistemology to students with the view to initiate them into

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
the discipline as certified members through the attainment of relevant qualifications (Cotti & 413
Schiro, 2004; Mnguni, 2018b; Schiro, 2013). The service-centered ideology seeks to prepare
students for their roles in service delivery for society by helping them develop relevant practical
skills and acquire relevant knowledge (Schiro, 2013). The citizenship-centered ideology seeks
to bring about social transformation and empowerment of citizens beginning with students,
by teaching social reconstructionist epistemology and ontology (Cotti & Schiro, 2004). It
attempts to provide students with knowledge and skills required to identify, transform, and/
or reconstruct social ills, norms, and values to enhance social empowerment (Kliebard, 2004).
The student-centered ideology adopts a view that places students at the center of teaching and
learning by supporting student development concerning his/her individual and social needs.
In this ideology, teachers are tasked with nurturing student development and facilitate social
and individual learning (Schiro, 2013). Researchers generally agree that the student-centered
and citizenship-centered ideologies are most suitable for a student and socially accountable
curriculum (Cotti & Schiro, 2004; Mnguni, 2013; Schiro, 2013).
The four curriculum ideologies, therefore, were adopted in the current study to determine
the extent to which the Life Sciences curriculum is student and socially accountable in relation
to HIV/AIDS knowledge. Research methods followed in this regard are discussed below.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
414 Table 1. Curriculum ideologies and their characteristics (adapted from Mnguni,
2018a,b).
Components of Discipline-centered Service-centered Student-centered Citizenship-centered

the curriculum ideology ideology ideology ideology
Purpose for knowl- Actualizing one- Interpreting and recon-
Understanding reality Performing tasks
edge self structing society
Individuals' cre-
Objective reality as ex- Normative objective Individuals' interpretation
Nature of knowl- ative meaning in
plained by academic reality as socially un- of society's past, present,
edge response to expe-
disciplines derstood and future
rience
Understanding so- Understanding
Mastering discipline- cial principles and oneself and ac- Understanding social prin-
Purpose and nature specific knowledge for acquiring knowledge quiring and/or con- ciples and acquiring and/or
of instruction the advancement of for providing prede- structing knowl- constructing knowledge for
the discipline termined services to edge for individual social transformation
society growth
Students viewed rela- Students viewed
Students viewed Students viewed relation
tion to standardized relation to standard-
as individuals as to standardized norms as
norms as they pas- ized norms as they
Role of the student they actively con- they actively learn emerg-
sively absorb pre- actively learn prede-
during instruction struct knowledge ing knowledge which
existing knowledge termined knowledge
which transforms transforms their mindsets
which transforms which transforms
their mindsets and behavior
mindsets their behavior
Accurate 'represen- Learning supervisor Visionary and colleague
tor' of the discipline who follows pro- Growth facilitator who adapts the curriculum
who transmits didactic grammed instruction who adapts the according to social con-
knowledge to stimu- to stimulate unifor- curriculum accord- cerns to effectively transfer
Role of the teacher
late uniformity by di- mity by directly im- ing to students' a social vision by stimulat-
during instruction
rectly implementing plementing the cur- needs to stimulate ing uniformity to prepare
the curriculum to ad- riculum to prepare growth and diver- students for their roles as
vance students in the students to perform sity change agents
discipline tasks.
Used at the end of Used during in-
Used at the end of instruction to certify struction to diag-
Used during instruction to
instruction to rank that students have nose students’
measure student progress
Purpose and nature students for a future in acquired skills as abilities and to
concerning ability using
of assessment the discipline through determined through facilitate growth
informal subjective diag-
norm-referenced ob- criterion-referenced through informal
nosis
jective assessment objective assess- subjective diag-
ment nosis
General Background
The realist research paradigm was adopted in the present research as a lens through which
reality is viewed. Realism allows for the use of mixed methods as it integrates elements of
different research paradigms (Denzin & Lincoln, 1994; Krauss, 2005). For example, similar to
positivism, realism assumes that reality is independent of the researcher (Healy & Perry, 2000;
Krauss, 2005). However, similar to critical theory and constructivism, realism also accepts
that reality is virtual in that it is context specific and may be socially constructed shaped by
social, economic, ethnic, political, cultural and gender values crystallized over time (Healy &

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
Perry, 2000). Because of these ontological and epistemological views, realism accommodates a 415
mixed-methods approach to research as is the case in the present study.
Sampling and Data Collection
As a starting point, the Life Sciences CAPS document (Department of Basic Education,
2011), which is the official curriculum document mandated by the South African government
Department of Basic Education was purposively sampled and analyzed as a way of identifying
topics in which HIV/AIDS knowledge is taught. By identifying these topics, the researcher had
a structured approach to identifying section of the textbooks that were analyzed.
A purposive sampling approach was then used to sample three Life Sciences textbooks
for document analysis. Life Sciences, previously known as Biology, is an optional school
science subject which is taught in Grades 10 to 12. This subject was purposively selected in
the research because it is the only South African basic education subject where students learn
biology content related to HIV/AIDS.
The three textbooks analyzed were selected randomly from the list of textbooks accredited
by the Department of Basic Education for use in teaching and learning Life Sciences. These are:
• Focus on Life Sciences (Clitheroe, Doidge, Marsden, van Aarde, Ashwell,
Buckley, & Dilley, 2008).
• Shuters Life Sciences Grade 11 Students Book (Ayerst, Langley, Majozi, Metherell,
Raciborska, & Smith, 2008).
• Solutions for All Life Sciences (McKay, Webb, Marchant, Freedman, Simenson,
de Fontaine, & van der Merwe, 2012).
Data Analysis
The analysis of the CAPS document and textbooks was done using an instrument adapted
from Wolff and Mnguni, (2015). The instrument is made up of two sections. The first section has
semi-structured items through which the researcher identified, classified, and interpreted text
and visual models that represent HIV/AIDS-related knowledge. This section of the instrument
focused on how the textbooks describe:
• the nature and characteristics of HIV;

• the nature and attributes of AIDS;
• mechanisms of HIV infection;
• treatment of AIDS;
• strategies for the prevention of HIV infection; and,
• skills that students must develop concerning HIV/AIDS.
The second part of the instrument has semi-structured items which were used to determine
the curriculum ideology that foregrounds the presentation of the HIV/AIDS knowledge in the
textbooks. The curriculum ideology foregrounding the CAPS document was not studied in the
present research as this has already been done in other studies (e.g., Mnguni, 2013; 2018). In
the current research, therefore, the instrument was used to determine the:
• purposes of teaching HIV/AIDS content;
• role of the students and teachers in teaching and learning about HIV/AIDS;
• instructional strategies recommended for teaching about HIV/AIDS; and,
• assessment strategies and the purpose of assessment related to HIV/AIDS.
Document analysis in the present research, therefore, meant the researcher analyses the
textbooks to formulate responses to the above items inductively using verbatim and narrated

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
416 extracts from the textbooks. A keyword search using AtlasTi was used to identify these text
extracts. Emerging responses were then quantified and classified into the different ideologies as
described in table 1.
Validity and Credibility
The use of a standard instrument for analyzing all three textbooks improved the
consistency, credibility, and validity of the results. This was also supported by the fact that
the instrument in its original form had been used in previous research (e.g., Wolff & Mnguni,
2015). However, in the present study, the instrument was also piloted on a different textbook
which was not part of this study. This was done to ensure that the researcher is familiar with the
instrument’s content and can use it effectively. Additionally, a panel of three other experts was
asked to determine the face and content validity of the instrument before its use. The same panel
also moderated the results of the study as generated by the researcher. Through these measures,
the present researcher is confident that the results are credible and valid.
Research Results
Content Knowledge Related to HIV/AIDS in the CAPS Document
Results show that Life Sciences has four knowledge strands (table 2; Department of
Basic Education, 2011, p. 10). HIV/AIDS-related content is taught in all these strands. It is
however explicitly taught only in Grades 11 and 12 (table 2). The term “HIV” is mentioned
twice in the CAPS document (Department of Basic Education, 2011) under the “Biodiversity
and Classification of Microorganisms” topic in Grade 11 (Department of Basic Education, 2011,
p. 39) and under “Evolution by Natural Selection” in Grade 12 (p. 61). The CAPS document
refers to HIV in the Grade 11 curriculum concerning diseases caused by viruses. However, in
Grade 12, HIV is mentioned concerning the evolution of bacteria and viruses, including where
the curriculum discusses the development of HIV resistance to anti-retroviral drugs. The term
“AIDS” is mentioned only once in the CAPS document under "Biodiversity and Classification
of Microorganisms" topic in Grade 11 (Department of Basic Education, 2011, p. 39). Within
the CAPS document, there is no evidence that HIV/AIDS knowledge is taught to foster the
construction of knowledge for individual or societal behavioral change.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
Table 2. A summary of topics taught in Life Sciences (Adapted from the 417
Department of Basic Education, 2011).
Life at molecular,
Life processes in Diversity, change, and Environmental
Strands cellular, and tissue
plants and animals continuity studies
level
Chemistry of life
Support and transport
Inorganic compounds
systems in plants
Organic compounds
Grade Support systems in Biodiversity and classification* Biosphere to
Cell - unit of life*
10 animals History of life on Earth ecosystems
Cell division (mitosis)
The transport system
Plant and animal
in mammals*
tissues
Energy transforma-
tions to support life:
Biodiversity - classification of
photosynthesis Population ecology*
microorganisms**
Grade Animal nutrition Human impact on the
Biodiversity - plants
11** Energy transforma- environment: current
Reproduction - plants
tions: respiration crises
Biodiversity - animals
Gas exchange
Excretion
Reproduction in
invertebrates
DNA code of Life*
Human reproduction* Darwinism and Natural Selec- Human impact on the
Grade RNA and protein
Nervous system tion** environment: current
12** synthesis*
Senses Human evolution crises Grade 11
Meiosis
Endocrine system
Homeostasis
** indicate instances where HIV or AIDS are mentioned explicitly in the CAPS document
* Indicates cases where textbooks or teachers could integrate HIV or AIDS during lessons.
Skills Related to HIV/AIDS in the CAPS Document.
In addition to teaching content knowledge reflected in the CAPS document and textbooks,
Life Sciences also seeks to help learners develop various skills within its three broad subject
specific aims. These skills could be used in acquiring, constructing and applying HIV/AIDS
knowledge in everyday life. Results show that skills developed in Life Sciences are classified as
cognitive skills, science process skills and skills for the application of knowledge in everyday
life (Department of Basic Education, 2011, p. 13-18), as summarized in Table 3.
The CAPS document suggests that the cognitive skills are developed to assist students
to “acquire knowledge” and “understand and make connections between ideas and concepts
to make meaning of life sciences” (Department of Basic Education 2011, p. 13-14). This
view suggests that there is already existing knowledge that students must “acquire” and make
meaning of rather than “construct” on their own. This is typical of discipline-centered ideology
where knowledge is regarded as an “objective reality as interpreted by academic disciplines”
and students must “master such discipline-specific knowledge” (Table 1). The service-centered
ideology is also reflected in that students are expected to “follow instructions by adhering
to safety rules when handling apparatus and making observations (Table 3). This is because
the service-centered ideology is concerned with training students to “performing tasks”
where teachers are viewed as “supervisors who follow programmed instruction to stimulate

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
418 uniformity” (see Table 1). The student-centered and social reconstruction ideologies were
observed in that students develop the ability to “apply knowledge of life sciences in new and
unfamiliar contexts, by using information in a new way; and applying knowledge to new and
unfamiliar contexts” (Table 3). This suggests that students will be able to use knowledge as
“personal creative meaning in response to experience” which is a characteristic of the student-
centered ideology (see table 1). This is also applicable to the citizenship-centered ideology in
that students learn the ability to “interpret society's past, present and future” by “acquiring and/
or constructing knowledge for social transformation” (Table 1).
Table 3. HIV/AIDS-related skills reflected in the CAPS document (adapted from

the Department of Basic Education, 2011).
Type of skills Skills Related curriculum ideologies
Acquire knowledge including accessing information by selecting Discipline-centered ideology

key ideas, recalling and describing phenomena
Create meaningful connections between ideas and concepts by

Discipline-centered ideology
building a conceptual framework of ideas by organizing knowledge
Service-centered ideology
and recognizing patterns
Cognitive skills
Apply knowledge in new contexts by using and applying knowl- Service-centered ideology
edge to novel contexts Student-centered ideology
Citizenship-centered ideology
Analyze, evaluate and synthesize scientific knowledge Service-centered ideology
Student-centered ideology
Follow instructions and adhere to guidelines.
Handle equipment and/or apparatus
Make and record observations
Science pro-
cess skills Record information/data Discipline-centered ideology
Measure using appropriate instruments and procedures
Interpret information Service-centered ideology
Design and/or plan investigations or experiments Service-centered ideology

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
419
Understanding the history, importance, and relevance of scientific Discipline-centered ideology
discoveries Service-centered ideology
Skills for the Understand the relationship between indigenous knowledge and
application of scientific knowledge
knowledge
Appreciate the value and application of Life Sciences knowledge
in the industry and everyday life, and career opportunities in Life
Sciences
Content Knowledge Related to HIV/AIDS in the Textbooks
Textbook analysis, however, revealed that HIV/AIDS-related content could be taught in

other topics as shown in the CAPS document (Table 2). For example, teachers could use the
structure of the cell (in Grade 10), immunology (in Grade 11 under Biodiversity and classification
of microorganisms), cell division (in Grade10 and 12) and DNA code of Life, RNA and protein
synthesis (in Grade 12) to teach about HIV target cells, its binding mechanisms and entry to
the target host cell as well as its reproduction and multiplication (Figure 1). It was, however,
noteworthy that textbooks are not consistent in their presentation of HIV/AIDS-related content.
For example, the term HIV is mentioned 19 times in the Focus on Life Sciences and only nine
(9) times in Shuters Life Sciences. Similarly, AIDS is mentioned 12 times in Focus on Life
Sciences and only six (6) times in Shuters Life Sciences. Some topics which are identified
in the literature (Audesirk, Audesirk, & Byers, 2004; Dimmock, Easton & Leppard, 2007) as
significant for learning about HIV/AIDS were not integrated into the CAPS document and in
the textbooks (Figure 1).
Figure 1. The presentation of HIV/AIDS-related concepts and their appearance

frequency in grade 11 textbooks.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
420 The manner in which HIV/AIDS-related content is presented suggests that Life Sciences
generally follows a discipline-centered ideology where knowledge is presented as an objective
reality that is factual as discovered by the authorities in the disciplines. The purpose of learning
this HIV/AIDS is, therefore, to help learners acquire this factual knowledge (see table 1).
However, the textbook analysis shows that other ideologies, such as service-centered and
social-reconstruction ideologies are also reflected. Results show, for example, that a real-life
case study is used to represent and explain patterns of the prevalence of HIV/AIDS in South
Africa (e.g., Focus on Life Sciences). Information related to the characteristics of HIV/AIDS
which could be used by learners to “understand reality”, and “interpret… society” is presented
in a manner typical in the discipline-centered and citizenship-centered ideologies respectively
(see table 1). Furthermore, the use of knowledge in this regard as “normative objective reality
as socially interpreted” to “understand social principles and acquiring knowledge for providing
predetermined services to society” and for “social transformation” is typical of service-centered
and citizenship-centered ideologies (see table 1). These observations suggest that concerning
teaching about HIV/AIDS the curriculum reflects the discipline-centered ideology while the
textbooks reflect various ideologies.
It was not possible to classify the concepts reflected in the CAPS document into the
different ideologies as the CAPS document only presents topics and does not offer the context in
which they are to be taught. However data shows that based on Schiro’s (2013) characterization
of the curriculum ideologies (table 1) and the frequency of appearance of HIV/AIDS-related
concepts, in Shuters Life Sciences, the discipline-centered ideology was found to be the most
depicted ideology in the HIV/AIDS-related concepts (63%) compared to citizenship-centered
(21%), service-centered (11%) and student-centered (5%) ideologies. The same pattern was also
found in Focus on Life Sciences where the discipline-centered ideology is the most reflected
ideology in the HIV/AIDS-related concepts (52%) compared to citizenship-centered (31%),
service-centered (10%) and student-centered (7%) ideologies.
Discussion
Previous research has shown that the Life Sciences curriculum adopts various ideologies
(e.g., Mnguni, 2013). Before the present research what had not been studied however are the
curriculum ideologies reflected in the science textbook, in general, and in Life Sciences in
particular. Furthermore, the curriculum ideology informing content knowledge in the subject
had also not been explored. To this end, the present research examined the extent to which HIV/
AIDS knowledge is presented in the Life Sciences and the curriculum ideology that inform this
presentation of HIV/AIDS knowledge.
Similar to Mnguni (2013, 2018a), the present research has found that the Life Sciences
textbooks integrate the four curriculum ideologies to inform the nature of content knowledge
presented and how that knowledge is presented. The present research also shows that HIV/
AIDS-related content is presented predominantly from a discipline-centered ideology. This is
important to note given the fact that previous research has suggested that in the citizenship-
centered ideology students acquire knowledge and skills that enable them to actively identify
and solving social ills (Schiro, 2013). In the context of the present research, such social ills
may include risk behavior related to HIV/AIDS. Consequently, it would be ideal that a student
and socially accountable curriculum would present HIV/AIDS-related content in line with the
principles of the student-centered and citizenship-centered ideologies. The present research,
however, has found that this is not the case in the Life Sciences curriculum and textbooks.
Previous research has also shown that HIV/AIDS knowledge taught within school
curricula does not seem to affect behaviors (Anderson & Beutel, 2007). Mnguni and Abrie (2012)
suggest that this is probably because such knowledge lacks depth and is presented in a mode that

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
is not relevant to students. Other scholars also argue that HIV/AIDS knowledge is presented 421
in the school curriculum as academic knowledge rather than functional knowledge (Wolff &
Mnguni, 2015). What these studies have not shown are the underlying reasons that foreground
how HIV/AIDS knowledge is presented in the curriculum. For example, the extent to which
the curriculum promotes the adoption of student and socially accountable content knowledge
has, before the present research, not been explored. The current study, however, has found that
both the student-centered and citizenship-centered ideologies are not the primary ideologies
that foreground the integration of HIV/AIDS knowledge. Instead, it is the discipline-centered
ideology that foregrounds this content. The researcher, therefore, posits that the textbooks and
teachers will most likely not be able to present functional HIV/AIDS knowledge which could
lead to behavioral changes among students. This is because the curriculum document is meant
to guide textbook authors and instructional designers in identifying content knowledge that
must be taught in the subject.
The high prevalence of HIV/AIDS in South Africa suggests that the schooling system
may not be able to provide students with sufficient skills for safe behavior adequately. This
may be because the majority of students rely on school-based biology knowledge to make
decisions related to HIV/AIDS (Mnguni & Abrie, 2012). This highlights the need to ensure that
students develop skills required to use scientific knowledge in making informed decisions that
support safe behavioral practices. According to Schiro (2013), the student-centered ideology
is most suitable for this purpose as it allows students to understand themselves by acquiring
and/or constructing knowledge for individual growth which transforms their mindsets and
ultimately behaviors. Interestingly, the present research has found that the student-centered
ideology is least reflected on HIV/AIDS-related knowledge and skills. As such, it is plausible
to suggest that student may not be developing the relevant and necessary skills for constructing
knowledge and decision making. Consequently, students continue to adopt risk behaviors. This
is in line with Mnguni, Abrie and Ebersohn’s (2016) assertion that the availability of HIV/AIDS
knowledge is not always related to self-reported safe behavioral preferences.
Conclusions
Critical to the present research, therefore, is that the Life Sciences curriculum recommends
vital knowledge and skills related to HIV/AIDS which is presented in the textbooks. This
knowledge is regarded as relevant because it is in line with socio-political needs of the country
as reflected in the South African constitution. It was however found that in spite of presenting
some relevant HIV/AIDS knowledge, the subject may not be student and socially accountable.
This is because the present research has found that HIV/AIDS-related content is presented
predominantly from a discipline-centered ideology. The student-centered and citizenship-
centered ideologies are not used to foreground HIV/AIDS knowledge in the Life Sciences
curriculum and textbooks. Consequently, it is unlikely that the presented functional HIV/
AIDS knowledge could lead to behavioral changes among students. In light of the findings, the
researcher posits that Life Sciences may not be macroethical as it does not address a critical
socio-scientific issue that is affecting the global society generally and the South African society
in particular.
The researcher, therefore, believes that further research is required globally to determine
the social responsiveness of school science education. The actual role of school science in
empowering the youth and their societies requires thorough investigation. Policy makers and
curriculum designers should also reflect on criteria used to accredit textbooks, mainly reflecting
on the extent to which textbooks reflect the social responsiveness intended in curriculum
statements. Strategies for ensuring that textbooks and curricula are aligned to students' emerging
socio-scientific needs should also be explored. Ultimately, based on the findings of the present

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
422 research, the researcher recommends that further research be conducted to determine the extent
to which school science is student and socially accountable.
Acknowledgments
This research was funded by the South African National Research Foundation (NRF,
GUN 99210).
References
Anderson, K. G., & Beutel, A. M. (2007). HIV/AIDS prevention knowledge among youth in Cape Town,
South Africa. Journal of Social Sciences, 3(3), 143–151.
Atabey, N., & Topcu, M. S. (2017). The development of a socioscientific issues-based curriculum
unit for middle school students: Global warming issue. International Journal of Education in
Mathematics, Science and Technology, 5(3), 153-170.
Audesirk, T., Audesirk, G., & Byers, B. E. (2004). Life on Earth, 4th ed. Upper Saddle River, N. J.:
Pearson Prentice Hall.
Ayerst, P., Langley, R., Majozi, P., Metherell, A., Raciborska, C., & Smith, D. (2008). Shuters life sciences
grade 11 learners book. Cape Town: Shuter & Shooter.
Bird, S. J. (2014). Socially responsible science is more than "good science." Journal of Microbiology &
Biology Education, 15(2), 169.
Chi, M. T. H., Feltovich, P. J., & Glaser, R. (1981). Categorization and representation of physics problems
by experts and novices. Cognitive Science, 5(2), 121–152.
Christenson, N., & Chang Rundgren, S. N. (2015). A framework for teachers’ assessment of socio-scientific
argumentation: an example using the GMO issue. Journal of Biological Education, 49(2), 204-
212.
Clitheroe, F., Doidge, M., Marsden, S., van Aarde, I., Ashwell, A., Buckley, L., & Dilley, L. (2008). Focus
on Life Sciences. Cape Town, South Africa: Maskew Miller Longman.
Cotti, R., & Schiro, M. (2004). Connecting teacher beliefs to the use of children’s literature in teaching of
mathematics. Journal of Mathematics Teacher Education 7, 329–356.
Department of Basic Education. (2011). Life sciences. Curriculum and assessment policy statement grade
10-12. Retrieved from http://www.thutong.doe.gov.za.
Dimmock, N. J., Easton, A. J., & Leppard, K. N. (2007). Introduction to modern virology, 6th ed. New
Jersey: Blackwell Publishing.
Dowden, T. (2013). Understanding the curriculum. In P. Hudson (Ed). Learning to teach in the primary
school (pp. 37-51). New York, NY: Cambridge University Press.
Fraser, W. J. (1993). The selection and ordering (arrangement) of learning content. In W. J. Fraser, C.
P. Loubser, & M. P. van Rooy (Eds). Didactics for the undergraduate student (2nd ed). Durban:
Butterworths.
Green-Thompson, L. P. (2013). Are we socially accountable educators? African Journal of Health
Professions Education, 5(1), 2.
Hoeg, D. G., & Bencze, J. L. (2017). Values underpinning STEM education in the USA: An analysis of the
next generation science standards. Science Education, 101(2), 278–301. doi: 10.1002/sce.21260.
Kharsany, A. B., & Karim, Q. A. (2016). HIV infection and AIDS in sub-Saharan Africa: Current status,
challenges, and opportunities. The Open AIDS Journal, 10, 34.
Krauss, S. E. (2005). Research paradigms and meaning-making: A primer. The Qualitative Report, 10(4),
758 – 770. Retrieved from http://www.nova.edu/ssss/QR/QR10-4/krauss.pdf.
Le Grange, L. (2016). Decolonising the university curriculum: Leading article. South African Journal of
Higher Education, 30(2), 1-12.
Lindgren, S., & Karle, H. (2011) Social accountability of education, aspects on global accreditation.
Medical Teacher, 33(8), 667-672.
Luescher, T., Loader, L., & Mugume, T. (2017). # FeesMustFall: An Internet-age student movement in
South Africa and the case of the University of the Free State. Politikon, 44(2), 231-245.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
Mnguni, L. (2013). The curriculum ideology of the South African secondary school Biology. South 423
African Journal of Education, 33(2), 1-11.
Mnguni, L. (2018a). Citizenship education and the curriculum ideologies of Natural Sciences and Life
Sciences curricula in South Africa. Curriculum Perspectives, 38(2), 97–106. doi: 10.1007/s41297-
018-0044-z.
Mnguni, L. (2018b). The curriculum ideology recommended by novice teachers for life sciences in South
Africa, 14(7), 3099–3108.
Mnguni, L., & Abrie, M. (2012). HIV/AIDS content knowledge and presentation strategies in biology for
effective use in everyday life. Journal of Biological Education, 46(4), 226-233.
Mnguni, L., Abrie, M., & Ebersöhn, L. (2016). The relationship between scientific knowledge and
behaviour: an HIV/AIDS case. Journal of Biological Education, 50(2), 147-159.
Nuangchalerm, P., & El Islami, R. A. Z. (2018). Context of science on environmental conservation:
Comparative study between Thai and Indonesian novice science teacher students. Jurnal
Penelitian dan Pembelajaran IPA, 4(1), 60-67.
Pietarinen, J., Pyhältö, K., & Soini, T. (2017). Large-scale curriculum reform in Finland–exploring
the interrelation between implementation strategy, the function of the reform, and curriculum
coherence. The Curriculum Journal, 28(1), 22-40. doi: 10.1080/09585176.2016.1179205.
Porter, R. E., Fusarelli, L. D., & Fusarelli, B. C. (2015). Implementing the Common Core: How educators
interpret curriculum reform. Educational Policy, 29(1), 111-139.
Schiro, M. S. (2013). Curriculum theory: Conflicting visions and enduring concerns. 2nd Edition. London:
Sage.
Spencer, H. (1896). Education: Intellectual, moral and physical. New York, NY: Appleton.
Statistics South Africa. (2018). Mid-year population estimates. Pretoria, South Africa: Stats SA.
Tanner, D., & Tanner, L. (1987). The history of the school curriculum. New York: Macmillan.
Veness, D. (2010). “As simple as possible”: the bones of curriculum design. In C. H. Steel, M. J. Keppell,
P. Gerbic & S. Housego (Eds.), Curriculum, technology & transformation for an unknown
future. Proceedings Ascilite Sydney 2010 (pp.1002-1006). Retrieved from http://ascilite.org.au/
conferences/sydney10/procs/Veness-concise.pdf.
Waks, L. J. (2003). How globalization can cause fundamental curriculum change: An American
perspective. Journal of Educational Change, 4, 383–418.
Wolff, E., & Mnguni, L. (2015). The integration of HIV and AIDS as a socio-scientific issue in the
life sciences curriculum. African Journal of Research in Mathematics, Science and Technology
Education, 19(3), 213-224.
Lindelani Mnguni PhD, Associate Professor, Department of Science & Technology Education, University
of South Africa, 7-15 AJH vd Walt Building, Muckleneuk Campus, UNISA Pretoria, South
Africa.
E-mail: mngunle@unisa.ac.za
ORCID : http://orcid.org/0000-0002-0361-0002

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
424
THE OPINIONS OF PRE-PRIMARY
EDUCATION TEACHERS ON THE
ADVANTAGES AND DISADVANTAGES
OF HETEROGENEOUS AGE GROUPS
Ivana Rochovska
Catholic University in Ruzomberok, Slovakia
E-mail: ikrupova@gmail.com
Zlatica Huľová
Matej Bel University in Banská Bystrica, Slovakia
E-mail: zlatica.hulova@umb.sk
Vladimír Klein
Catholic University in Ružomberok, Slovakia
E-mail: vladimir.klein@ku.sk
Miroslava Gašparová
Matej Bel University in Banská Bystrica, Slovakia
E-mail: miroslava.gasparova@umb.sk
Abstract
The research focuses on the issue of mixed-aged children in nursery schools. The aim of the research
was to find out the opinions of the nursery school teachers on the advantages and disadvantages of
a heterogeneous age groups based on their practical experiences. In order to find out the opinions of
the respondents, a questionnaire was constructed and used. Based on the results, it can be said that
243 respondents consider an advantage to be the better social and emotional development of children,
regardless of whether the respondent prefers to work in a homogeneous or heterogeneous age group.
However, they did not agree with the disadvantages of heterogeneous age groups. The respondents
teaching homogeneous age groups have generally identified far more disadvantages, and the greatest
disadvantage being that children in a heterogeneous age group have worse conditions for cognitive
development and do not have the opportunity to compete with each other. It has also been confirmed that
those respondents with more years of work experience prefer a more heterogeneous age composition of
groups than respondents with fewer years of work experience.
Keywords: mixed-age groups, nursery school, nursery school teacher, pre-primary education.
Introduction
The pre-school age is crucial to the development of every person’s personality. In this
period, the so-called window of opportunities is open for the acquisition of different abilities;
the foundations for the development of intelligence and psychomotor skills are built and a
social relationship with other people is emerging. It is important that all aspects of a child’s
personality develop in a balanced way, so that an emphasis is not only on the cognitive aspect, at
the expense of the socio-emotional and psychomotor aspects. For this reason, an important role

Ivana ROCHOVSKA, Zlatica HUĽOVÁ, Vladimír KLEIN, Miroslava GAŠPAROVÁ. The opinions of pre-primary education teachers on
the advantages and disadvantages of heterogeneous age groups
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
in the development of the child’s personality is also played by the environment of the nursery 425
school and the organisation of the education, in terms of the homogeneity of the groups.
Theoretical Starting Point of the Studied Issue
Peter Gray, an American psychologist, who studied the self-education of hunter-gatherer

children in primitive tribes from an anthropological point of view, concluded that child education
is more effective when children are in mixed-age groups. The researcher recommended that
mixed-age groups were also applied in the current education system (Gray, 2008). Concerning
the issue of age-homogeneity, the researcher stated: “One of the most bizarre and, from my point
of view, the most harmful aspects of our treatment of children, is our interest in segregating
children into separate age groups. We do it not only in school, but also in extra-curricular
environments, thus depriving children of a valuable component of their natural means of self-
education” (Gray, 2013, p. 58).
In this research, the concept of a heterogeneous age group (HeG) designates nursery
school group (class) with children of different ages from 3 to 7 years old, and the concept of a
homogeneous age group (HoG) designates group with children of the same age category, e.g.
3-4 years old, 5-6 years old, etc.
The division of children into a HeG in school classes has its origins in the reform
education of the 20th century (Montessori, 2004; Kovalik & Olsen, 1993; Petersen & Walberg,
1979). The emergence of many alternative schools was a response to traditional education, in
an attempt to create educational institutions, which would provide a better quality of education.
The emerging alternative schools have been distinguished from classical ones by several
features, for example, goals, content, didactic methods and formats, including the formation of
heterogeneous age groups.
From classical alternative schools, M. Montessori (2004) can be mentioned, where the
children’s relationships are not marked by rivalry and competition; on the contrary, in a HeG,
they strengthen mutual relationships through free cooperation. The older children spontaneously
share their knowledge and skills with the younger ones; the younger children find help and
support in the older ones if needed. Their coherence is also supported. The advantage is that
older children, who want to help or explain something to the younger ones, have to structure
their knowledge and try to communicate with them in a more understandable manner, thereby
expanding their own comprehension of skills, while strengthening self-confidence (Montessori,
1972).
Another classic alternative school is the Jenaplan School, which is characterised by a
family atmosphere; it prepares the children a naturally stimulating environment, so that they
would be able to develop freely (Petersen, 1979). Children are divided into the so-called tribal
groups combining two or three years, thus creating natural groups on the basis of voluntary and
free grouping.
Although many of the classical alternative schools do not explicitly insist on creating
a HeG, nonetheless their principles provide the right conditions. E.g. Waldorf School, Dalton
Plan or Freinet Pedagogy (1993) emphasised collaboration in groups; they prefer to create
social positive relationships and mutual help, more than competitivity (Clanfield & Sivell,
1990), which creates an opportunity for a HeG.
From the modern alternative schools, the model of Integrated Thematic Teaching,
according to Kovalik (Kovalik & Olsen, 1993) can be referred, which promoted teaching in a
HeG (Gejdoš & Graham, 2016). Among its basic principles, the open school included, learning
from one another, cooperation and communication, so instead of the traditional child divisions
by age, was preferred to create a HeG. A HeG is typical for cooperative teaching, in which
children of different abilities, sexes, ethnic backgrounds and social groups worked together
(Johnson & Johnson, 2018), according to the options of the school and their age.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
426 From other alternative concepts supporting free play and a HeG, one can mention free
schools, for example, the model of anti-authoritarian education by Neill (1995). In 1921, he
founded a school that up until the present day operates on the same principles. It is attended by
hundreds of learners of different nationalities, aged 5-18. Sudbury Valley schools around the
world are working on similar principles (Greenberg, 1992).
At present, the trend of pre-primary education is to create an inclusive environment
(Chepel, Aubakirova, & Kulevtsova, 2016), where any diversity, whether in terms of nationality,
race, language, social environment or disability, is not only accepted, but even welcomed. A
HeG can be especially of benefit to the effective work of an inclusive school.
A heterogeneous age class allows the child a wider range of social connections. Every
child, when he is three or four years old, tries out the role of a freshman during the attendance
of a nursery school and lets the older children advise and guide him. Later, when he reaches 5
or 6 years, he has the opportunity to experience the role of a younger child’s guide, protector
and supporter. The child also has the opportunity to observe more behavioural patterns than in
a HoG.
The creation of a HeG also has its disadvantages, especially in a classical education
movement, where it is necessary to pay attention to the fulfilment of the prescribed content
and performance standards. Teachers have a more demanding preparation and organisation of
teaching.
Researched Issue
The researched issue started out from verified hypotheses that the respondents - nursery
school teachers who taught in homogeneous age groups - would still choose to teach a
homogeneous age group. Likewise, teachers who taught in heterogeneous age groups, would
rather choose to teach in a homogeneous age group (Huľová, Rochovská, & Klein, 2018). The
respondents’ opinions are, therefore, very different from those of most experts, so the topic of
age homogeneity groups in nursery schools can be considered current. The opinions of experts
on the importance and benefits of heterogeneous age groups in nursery schools are consistent
with another validated hypothesis in the aforementioned research. The respondents teaching
heterogeneous age groups see more benefits of such divisions compared to the respondents
teaching homogeneous age groups. Also, in this research, it was found that younger respondents
preferred homogeneous age groups more than the older ones. The age of the teachers is generally
related to the number of years of their teaching experience. Older teachers with more years
of experience have more work experience and can more appreciate the benefits of bringing
children into heterogeneous age groups (Huľová, Rochovská, & Klein, 2018).
For these reasons, the research questions were: 1. Whether there is a statistically
significant difference among the respondents favouring a HeG and the respondents favouring
a HoG in the opinions related to benefits of a HeG “children have better conditions for social
development” and “children have better conditions for emotional development”; 2. Whether
there is a statistically significant difference among the respondents favouring a HeG and the
respondents favouring a HoG in the opinions related to disadvantages of a HeG “children
have worse conditions for cognitive development” and “children do not have the possibility to
compete with each other”; 3. Whether there is a statistically significant difference between the
respondents with a higher number of years of work experience and those with a fewer number
of years of work experience in terms of preference for a HeG.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
The Aim and Hypotheses of the Research 427
The main aim of the research was to find out the opinions of the nursery school teachers
on the advantages and disadvantages of a HeG. From the main aim of the research, partial goals
came about:
1. To find out what advantages the teachers see from teaching in a HeG.
2. To find out what disadvantages the teachers see from teaching in a HeG.
3. Compare the advantages and disadvantages of a HeG, in the opinions of those
teachers who prefer a HoG.
4. Compare the advantages and disadvantages of a HeG, in the opinions of those
teachers who prefer a HeG.
5. Compare HeG preferences among the teachers, according to their years of work
experience.
Based on the research aim and on the basis of a research with similar problematics
(Huľová, Rochovská, & Klein, 2018), the hypotheses were formulated:
1. Among the respondents favouring a HeG and the respondents favouring a HoG,
there is a statistically significant difference in the number of marked benefits of
a HeG, with the respondents favouring a HeG considering “children have better
conditions for social development” and “children have better conditions for
emotional development”.
2. Among the respondents favouring a HeG and the respondents favouring a HoG,
there is a statistically significant difference in the number of marked benefits of
a HeG, with the respondents favouring a HoG considering “children have worse
conditions for cognitive development” and “children do not have the possibility to
compete with each other”.
3. Between the respondents with a higher number of years of work experience and those
with a fewer number of years of work experience, there is a statistically significant
difference in terms of a HeG, in favour of those respondents with more years of work
experience.
General Background
This research was carried out between November 2017 and December 2018. It was part
of the larger project entitled “Advantages and disadvantages of heterogeneous age groups
from the perspective of nursery school teachers”. Some findings from this project, regarding
teachers’ preference of age-homogeneous or heterogeneous classes have already been published
(Huľová, Rochovská, & Klein, 2018). In this larger research project, a questionnaire, a semi-
structured interview and a direct unstructured observation were used as research methods to
find out the opinions of the respondents about the issue. In the non-structured interview, the
respondents freely expressed their opinion about the issue. The researchers also conducted an
observation of the educational activities at four nursery schools, which divide children into
HoG and HeG.
A questionnaire was used to find out the opinions of the respondents. The questionnaire
contained 8 questions; 5 questions were evaluated for the purposes of this research. These were
items featuring the choice of response with different options offered, while the respondents had
the option to give another answer for each item.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
428 The respondents indicated whether they are teaching in a class with a HoG or a HeG, and
whether they would choose to teach in a HoG or a HeG, if it was within their power. In the other
two items, they have chosen from several options, the advantages and disadvantages that they
see in HeG teaching. The advantages and disadvantages were formulated on the basis of the
theoretical starting points of the given issue (Evangelou, 1989; Gray, 2008; Greenberg, 1992;
Salcedo, 2018). They are listed in the legends of Figures 1 and 2. In addition, the respondents
could also mention other advantages and disadvantages that they did not find in the options
offered. At the end, the respondents reported the number of years of their teaching work
experience.
The questionnaire was designed by researchers for the purposes of this research, so its
expertise and pilot testing were necessary. Five experts in the area of pre-school education have
commented on each item in order to assess its constructive and content validity. Each item was
assessed on a scale of 5 (relevant) - 0 (irrelevant). The result was expressed as a percentage.
(90%).
Due to the smaller scale of the research tool, it is necessary to realise its lower reliability,
however, the benefit is the good homogeneity of the research tool, because it measures one
attribute, and the items are close to each other due to their content (all are related to age group
homogeneity within a nursery school).
A constructed questionnaire was pilot tested on a sample of 20 kindergarten teachers. After
filling out the questionnaire, they were commenting on whether the instructions concerning the
answering of the question could be well understood, whether the questions were linguistically
correct, easy to understand and logically organised, whether some questions were redundant
and whether the questionnaire was sufficiently motivating for nursery school teachers. Based
on the answers by the respondents in the pilot testing, serious adjustments were not required.
Subsequently, a pre-survey was carried out with the aim to eliminate minor mistakes, which
have been overlooked in the pilot testing, as well as to test the statistical processing of the
results.
The questionnaires were administered personally, electronically and online through
social groups.
Sample Selection
The respondents took part in the research on a voluntary basis, which they have confirmed
in the informed consent at the beginning of the questionnaire (Huľová & Šukolová, 2016).
The research sample was assured by the selection available. 234 nursery school teachers were
involved in the research. Of these, 95 teach in HoG classes and 139 teach in HeG classes. In
terms of the years of work experience, the research sample shows the characteristics listed in
Tables 1 and 2 with the majority of respondents having around 5 or around 30 years of work
experience. In terms of the preference of a HoG and a HeG, the research sample shows the
characteristics listed in Table 3.
Table 1. The length of the teaching practice of the respondents (%).
Work
experience up to 5 6-10 11-15 16-20 21-25 26-30 31-35 above 36
(years)
% resp. 28.21 10.26 9.40 2.56 8.55 12.82 17.09 11.11

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
Table 2. The length of the teaching practice of the respondents (Descriptive 429
statistics).
M SD x̃ xmin xmax
Work experience
(years) 18.72 13.59 19.5 0 42
Table 3. The preference of homogeneous and heterogeneous age groups among

the respondents.
Work experience
Teach HoG Teach HeG Total
(number of years)
Prefer HoG 93 80 173

Prefer HeG 0 57 57
Pther answers 2 2 4
Total 95 139 234
Legend: Teach HoG - respondents teaching in a class with a homogeneous age composition of children;
Teach HeG - respondents teaching in a class with a heterogeneous age composition of children; Prefer HoG
- respondents preferring teaching in classes with a homogeneous age composition of children; Prefer HeG -
respondents preferring teaching in classes with a heterogeneous age composition of children.
Data Analysis
The respondents’ answers to the selected questions were analysed. For a statistical
verification of the hypotheses, Pearson’s chi-squared test was used, as it is a comparison of the
two groups within the nominal variable.
Research Results
The Benefits of Teaching in Heterogeneous Age Groups
In Figure 1, the answers of the respondents, concerning the marked benefits of a HeG in
nursery school classes, are presented.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
430 25,00
20,00
15,00
Prefer HoG
10,00 Prefer HeG
5,00
0,00
a) b) c) d) e) f) g) h) i)
Figure 1. Benefits of heterogeneous age classes in nursery schools [in %].
Legend: Prefer HoG - respondents preferring teaching in classes with a homogeneous age composition
of children [in %]; Prefer HeG - respondents preferring teaching in classes with a heterogeneous age
composition of children [in %]; a) the class more resembles a family environment; b) children have
better conditions for social development; c) children have better conditions for emotional development;
d) children have better conditions for cognitive development; e) deregulation of competition between
children; f) the teacher can compare the progress made by the child over a longer period of time; g) the
child will develop a deeper relationship with the teacher; h) a longer and better cooperation with the
parents of the child; i) other.
Figure 1 shows that teachers preferring teaching in the HoG classes, most commonly
referred as a benefit the fact that children have better conditions for social development (b), that
the classroom more resembles a family environment (a) and that children have better conditions
for emotional development (c). Teachers who prefer teaching in classes with a HeG of children
most often referred to the same benefits, but in a different order - children have better conditions
for social development (b), children have better conditions for emotional development (c) and
the class more resembles a family environment (a).
From the other benefits, the respondents mentioned, for example, that the younger
children learn from the older ones through observation and imitation; the children learn faster
from the older children without the intervention of a teacher and there is the possibility to place
siblings in one class.
In Table 4, the benefits of a HeG from the point of view of the respondents, broken down
by the preferences of the advantages b and c are presented.
Table 4. Benefits of heterogeneous age groups from the point of view of the
respondents.
Observed frequency Expected frequency

Prefer Prefer Prefer Prefer
HoG HeG HoG HeG
Benefits b and c 150 104 Benefits b and c 151.41 102.59
Other benefits 278 186 Other benefits 276.59 187.41
The respondents favouring a HeG and the respondents favouring a HoG agreed in their
opinions that in a HeG the children have better conditions for social and emotional development.
The hypothesis H1 was rejected (Table 6).

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
Disadvantages of Teaching in Heterogeneous Age Groups 431
In Figure 2, the answers of the respondents concerning the identified disadvantages of a

HeG in the classes of nursery schools are presented.
30,00
25,00
20,00
15,00 Prefer HoG

Prefer HeG
10,00
5,00
0,00
a) b) c) d) e) f) g) h) i)
Figure 2. The disadvantages of heterogeneous age classes in nursery schools [in %].
Legend: Prefer HoG - respondents preferring teaching in classes with a homogeneous age composition of
children [in %]; Prefer HeG - respondents preferring teaching in classes with a heterogeneous age composition
of children [in %]; a) the child is in contact with a smaller number of children of the same age; b) the children
have worse conditions for social development; c) the children have worse conditions for emotional development;
d) the children have worse conditions for cognitive development; e) the children do not have the opportunity
to compete with each other; f) the child is assessed by only one teacher for several years; g) the child is in
contact with only one/two teachers over several years; h) the parents of the child are in contact with only one/
two teachers over several years; i) other.
Figure 2 shows that teachers preferring a HoG most often referred as a disadvantage
the fact that a child is in contact with a lower number of children of the same age (a), the
children have worse conditions for cognitive development (d) and the children do not have an
opportunity to compete with each other. Respondents favouring a HeG have identified far fewer
disadvantages. They indicated in particular the disadvantage that the child is in contact with a
smaller number of children of the same age (a) and the child is in contact with only one/two
teachers over several years (g).
Other disadvantages reported by the respondents were, for example, more difficultly
securing spatial conditions; more demanding teacher preparation; the copying of older children
in a negative sense; disturbing the activity of the older children by the younger ones and a
marginalisation of the younger children to the detriment of the older ones.
Table 5 shows the disadvantages of a HeG from the perspective of the respondents,
broken down according to the disadvantages of d and e.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
432 Table 5. Disadvantages of heterogeneous age groups from the perspective of

the respondents.

Prefer Prefer Prefer Prefer
HoG HeG HoG HeG
Disadvantages of d and e 86 6 Disadvantages of d and e 70.12 21.88
Other disadvantages 164 72 Other disadvantages 179.88 56.12
There was a statistically significant difference between the respondents. Respondents

favouring a HeG have identified less disadvantages of d and e than the respondents favouring a
HoG. The H2 hypothesis was accepted (Table 6).
Table 6. Statistical processing of the research data.
Chi-square criterion Degrees

Hypothesis Significance (p) Acceptance/rejection
(χ2) of freedom (df)
1 0.05 1 .823 Is rejected
2 21.013 1 .001 Is accepted
3 9.432 1 .002 Is accepted
Preferences of Homogeneous and Heterogeneous Age Groups in Terms of the Length of

Teaching Work Experience
In Figure 3, the HoG and the HeG preference among the respondents are referred.
30,00
25,00
20,00
15,00 Prefer HoG

Prefer HeG
10,00
5,00
0,00
a) b) c) d) e) f) g) h) i)
Figure 3. HeG and HoG preference among the respondents in terms of the
number of years of teaching work experience.
Legend: Prefer HoG - respondents preferring teaching in classes with a homogeneous age composition
of children [in %]; Prefer HeG - respondents preferring teaching in the classes with a heterogeneous age
composition of children [in %].

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
It is clear from Figure 3 that those respondents with less than 5 years of work experience 433
had a strong preference for a HoG. Among the respondents with over 21 years of work
experience, there was a preference for a HeG.
Table 7 shows the observed and expected frequencies.
Table 7. HeG and HoG preference among the respondents in terms of the number
of years of teaching work experience.
Prefer HoG Prefer HeG Prefer HoG Prefer HeG
Work experience Work experience

up to 20 years 98 20 up to 20 years 87.74 30.26
Work experience Work experience

over 21 years 76 40 over 21 years 86.26 29.74
There was a statistically significant difference between the variables. Respondents

with more years of work experience preferred a HeG more than those with less years of work
experience. The H3 hypothesis was accepted (Table 6).
Discussion
The aim of the present research was to find out the opinions of the nursery school
teachers on the advantages and disadvantages of HeG. This research provided more details
about the teachers’ opinions related to the advantages “children have better conditions for social
development”, “children have better conditions for emotional development”, and disadvantages
“children have worse conditions for cognitive development”, and “children do not have the
possibility to compete with each other”.
The results of the research carried out among 243 nursery school teachers revealed that
the respondents consider the benefits of a HeG to be the better social and emotional development
of the children. Those respondents favouring either a HeG or a HoG in nursery schools both
agreed with this opinion. Similarly, in the research study on age homogeneity in groups of
nursery school classes (Huľová, Rochovská, & Klein, 2018), the group of respondents teaching
a HeG found the best benefit to be the better social and emotional development of the children.
These opinions were in line with the opinions of the experts quoted in the theoretical starting
point of the issue, who emphasised the development of the personality of the child in the
nursery school (not just the preference of the cognitive side) and the promotion of prosocial
and collaborative behaviour. The age difference between the children allows them to experience
more diverse contacts, more social roles, acquire more strategies, and gain more opportunities
for natural imitation and the development of speech and communication.
The respondents also agreed that in a HeG, the child was with fewer children of the
same age, which was a disadvantage for him. Therefore, when creating a HeG, it is appropriate,
within the scope of the nursery school, to provide a balanced representation of a sufficient
number of children from each age category. It is also possible to regularly include activities in
which the group of children of the same age, especially the oldest age group, would cooperate
in. On the other hand, as noted by P. Gray and J. Feldman (2004), it was generally well known
that children of the same age had similar abilities to children of different ages, but this was not
always true. In a HeG, a child who was ahead of, or was behind his or her peers in a certain area,
could find equal partners among the older or younger children.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
434 The group of respondents favouring a HoG marked significantly more often that
disadvantages of a HeG were that the children had worse conditions for cognitive development.
These opinions contradict the results of the research examining the relations between classroom
age variability on children’s vocabulary development for 2,743 children between the ages of
two years, nine months, and six years, 11 months in Denmark. Findings indicated a significant
nonlinear relationship between the range of child age within a classroom and children’s
vocabulary development, such that classrooms with a maximum age range of 24 months were
associated with the greatest gains in vocabulary growth (Justice, Logan, Pustell, Bleses, &
Højen, 2018).
The group of respondents favouring a HoG marked significantly more often that
disadvantages of a HeG were that the children were not able to compete with each other. It is
important to realise whether pre-primary education is perceived primarily as preparation for
admission into primary school, where worse conditions for cognitive development are perceived
only in a negative sense, or whether an emphasis is placed on the overall development of the
personality of the child, where, alongside the cognitive aspect, the socio-emotional and psycho-
motoric side is also well developed. Concerning competition in pre-primary education, it is
necessary to emphasise that it is more appropriate to create a collaborative environment for the
child’s social, emotional and cognitive development.
The research results also showed that the respondents with more years of work experience
prefer a HeG more, when compared to the respondents with less years of work experience.
Similarly, in a research on the age homogeneity of a group in nursery school classes (Huľová,
Rochovská, & Klein, 2018), it was confirmed that younger respondents preferred homogeneous
age groups more than the older ones did.
Respondents with long-term practical experience are likely to have a greater insight and
are aware of the benefits that work in a HeG brings, despite the more demanding organisation
of work and methodical preparation. In the similar research, R. Smith and E. Engeli (2015)
detected a correlation between the teachers’ attitude towards mixed-age teaching and the
frequency of working with individualized learning sequences in the classroom.
On the basis of the above, it is possible to recommend that the issue of the age homogeneity
of a group in the study programs of the pre-gradual preparation of nursery school teachers
should be extended to a greater extent, at both a theoretical and a practical level.
Conclusions
The findings of this research have important implications for the nursery school teachers’
pregradual education. In has been found that the nursery school teachers considered an advantage
to be the better social and emotional development of children in heterogeneous age group. The
respondents teaching homogeneous age groups considered the greatest disadvantage being that
children in a heterogeneous age group have worse conditions for cognitive development and do
not have the opportunity to compete with each other. It has been confirmed that those respondents
with more years of work experience preferred a more heterogeneous age composition of groups
than respondents with fewer years of work experience.
In the research, it is declared that it is necessary to take into account the recommendations
of experts on the subject and in the context of the undergraduate training of nursery school
teachers, to familiarise them with the advantages and disadvantages of heterogeneous age
groups and to give them the opportunity to experience work in such groups. Teachers should
be mindful of the overall development of the personality of a child and to not restrict pre-
primary education to just preparation for entry into elementary school. The cognitive aspect
of a child’s personality should not be overstated at the expense of the socio-emotional and
psychomotor aspect. On the other hand, it cannot be said for sure that this side develops worse

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
in a heterogeneous age group, because when older children explain something to the younger 435
ones, they themselves strengthen and organise their knowledge. Heterogeneous age groups also
allow nursery schools to break down competition among children and, on the contrary, support
mutual help and cooperation.
The results of the research are that we should continue to study the issue and to extend the
research also to the primary stage of elementary school, where the pro-social and cooperative
behaviour of learners would be developed to the detriment of the competitivity and mutual
comparison of learners, whether with the norm or amongst each other.
Acknowledgements
The research is part of the project, GAPF CU No. 2/41/2018 “Advantages and
disadvantages of heterogeneous age groups from the perspective of nursery school teachers”.
References
Clanfield, D., & Sivell, J. (1990). Cooperative learning & social change: Selected writings of Célestin
Freinet. Toronto, Ontario: Our Schools/Our Selves.
Evangelou, D. (1989). Mixed-age groups in early childhood education. Illinois: ERIC Clearinghouse on
Elementary and Early Childhood Education.
Freinet, C. (1993). Education through work: A model for child-centered learning. Lewiston, New York:
Edwin Mellen Press.
Gejdoš, M., & Graham, L. (2016). Using computer programs in teaching in secondary schools. Dublin,
Ireland: International Scientific Board of Catholic Researchers and Teachers in Ireland.
Gray, P., & Feldman, J. (2004). Playing in the zone of proximal development: Qualities of self-directed
age mixing between adolescents and young children at a democratic school. American Journal of
Education, 110, 108–145.
Gray, P. (2008). The value of age-mixed play. Education Week, April 16, 2008. Retrieved 10/10/2017,
from evolution.binghamton.edu/evos/wp-content/uploads/2008/11/ValueAge-MixedPlay.doc.
Gray, P. (2013). Free to learn: Why unleashing the instinct to play will make our children happier, more
self-reliant, and better students for life. Philadelphia: Basic Books.
Greenberg, D. (1992). The Sudbury Valley School experience. 3rd edition. Framingham: Sudbury Valley
School Press.
Huľová, Z., Rochovská, I., & Klein, V. (2018). The issue of age homogeneity in groups from the
kindergarten teachers’ perspective. The New Educational Review, 53, 203-215.
Huľová, Z., & Šukolová, D. (2017) Slovak Primary education teachers’ views about and attitudes towards
contents of work and Technical education. The New Educational Review, 47, 165-175.
Chepel, T., Aubakirova, S., & Kulevtsova, T. (2016). The study of teachers’ attitudes towards inclusive
education practice: The case of Russia. The New Educational Review, 45, 235-246.
Johnson, R. T., & Johnson, D.W. (2018). An overview of cooperative learning. Retrieved 15/03/2018,
from http://www.co-operation.org/what-is-cooperative-learning/
Justice, L. M., Logan, J. A., Pustell, K., Bleses, D., & Højen, A. (2018, January 9). Does mixing age
groups in early childhood education settings support children’s language development? Applied
Developmental Science. Retrieved from https://doi.org/10.1080/10888691.2017.1386100.
Kovalik, S., & Olsen, K. D. (1993). ITI, the Model: Integrated thematic instruction. 2nd edition. United
States: Susan Kovalik & Associates.
Montessori, M. (1972). The secret of childhood. New York: Ballantine Books.
Montessori, M. (2004). The discovery of the child. Revised and Enlarged Edition of the Montessori
Method. Delhi: Aakar Books.
Neil, A. S. (1995). Summerhill school. A new view of childhood. New York: St. Martin’s Press.
Petersen, P. L., & Walberg, H. J. (1979). Research on teaching: Concepts, findings and implications.
Berkeley: Mc Cutchan.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
436 Salcedo, M. (2018). Strategies for working with mixed-age groups in early childhood education. Retrieved
from https://freespiritpublishingblog.com/2018/07/26/strategies-for-working-with-mixed-age-
groups-in-early-childhood-education/.
Smith, R., & Engeli, E. (2015). An empirical model of mixed-age teaching. International Journal of
Educational Research, 74, 136-145.
Questionnaire. Retrieved 15/03/2018. Retrieved from https://docs.google.com/forms/d/
e/1FAIpQLScEUezpWjX5d-Z_cZKjk6c8lP2J9m_b3NTYlsddGuiQRxARwg/viewform.
Ivana Rochovska PhD, Associate Professor, Catholic University in Ruzomberok, Bottova 15,
(Corresponding author) 054 01 Levoča, Slovakia.
E-mail: ikrupova@gmail.com
ORCID ID 0000-0001-9346-7993
Researcher ID: R-5004-2017
Zlatica Huľová PhD., Associate Professor, Matej Bel University in Banská Bystrica,
Faculty of Education, Ružová 13, 974 11 Banská Bystrica, Slovakia.
E-mail: zlatica.hulova@umb.sk
ORCID ID 0000-0001-7116-7754
Vladimír Klein PhD., Assistant Professor, Catholic University in Ružomberok, Faculty of Edu-
cation, Juraj Páleš Institute in Levoča, Bottova 15, 054 01 Levoča, Slovakia.
E-mail: vladimir.klein@ku.sk
Miroslava Gašparová PhD., Lecturer, Matej Bel University in Banská Bystrica, Faculty of Education,
Ružová 13, 974 11 Banská Bystrica, Slovakia.
E-mail: miroslava.gasparova@umb.sk
ORCID ID 0000-0001-9032-0478

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
LEARNING EXPERIENCE AS A FACTOR 437
OF MOTIVATION IN LOWER-SECONDARY
SCHOOL IN FOREIGN
LANGUAGE LEARNING
Eva Stranovská, Anikó Ficzere, Silvia Hvozdíková, Beáta Hockicková
Constantine the Philosopher University, Slovakia
E-mail: estranovska@ukf.sk, aniko.ficzere@ukf.sk, shvozdikova@ukf.sk,
bhockickova@ukf.sk
Abstract
This research focusses on the analysis of learning experience. Together with the other factors of
motivational structure: Intrinsic motivation, Personal goals, Attractivity of foreign language community,
and Worries about learning, it creates motivational profile of foreign language (FL) learners. The analysis
is based on the theoretical concept of foreign language motivational self-system by Dörnyei.
The research aim was to find out the perception of learning experience and the level of its dependence on
the category of the Associations with the expression “German” in the selected categories. Moreover, it
was to find out the differences in perception of learning experience, motivational structure, and the length
of foreign language studying. The research sample consisted of German FL learners from six different
lower-secondary schools within Slovakia. Motivational Structure Questionnaire was used as a research
tool to obtain the research data. The data were processed by the selected quantitative and qualitative
research methods. The research results supported ambivalence of learning experience.
The statistical significance was found in the two categories: the category of Negative emotions related to
the category negative perception of educational situation, and the category of German speaking countries
related to the higher level of motivation in the factor of learning experience. Furthermore, significant
relations were found between the factor of learning experience and the factors of the ideal self of the
motivational structure and the length of studying a foreign language. The results in the qualitative part of
the research opened up a possible intervention steps towards foreign language motivation reinforcement.
Keywords: foreign language, learning experience, motivation in education, qualitative research
methods, quantitative research methods.
Introduction
Motivation is the primary predictor of educational process and is an inevitable element

of positive educational achievements. When teachers and learners share the same educational
interests, they may create an authentic product, achieve success in educational process and come
to the point of positive learning experience. The factor of positive learning experience plays
an essential role in the process of motivation in education. Prominently because of positive
perception of educational process, relationship to teacher and other learners, social dynamics,
atmosphere, educational content, experience of success or unsuccess and other variables create
experiential continuum of learners and their learning experience.
Many scientific studies emphasized learning experience in the process of education
in the seventies of the 20th century. Kolb (1970, in Sarmány-Schuller, 1992) highlighted
the premise that the process of education is also derived from learner`s experience and is
formed within four-stages. Oravcová (2000), Stranovská (2009), Kaliská (2013) and others

Eva STRANOVSKÁ, Anikó FICZERE, Silvia HVOZDÍKOVÁ, Beáta HOCKICKOVÁ. Learning experience as a factor of motivation in
lower-secondary school in foreign language learning
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
438 have verified the theory in the recent studies. The main emphasis of the studies was self-
regulation of learning, development of the personality of the learner where learners regulate
their own learning processes. Learning experience has an original position in the process of
foreign language learning because there are many uncertain ambiguous situations involved
in the process. It is not possible to learn them by heart or expect exactly how they happen.
There are many situational factors involved, for example, foreign language attractivity, foreign
language cultural perception, experience with a foreign language culture, different cultural
patterns, including cultural background of the mother tongue. The variable has not been studied
in relation to the motivational structure into such extent. The factor of motivational structure
has been studied mainly in relation to intrinsic and extrinsic motivation (Deci, & Ryan, 1993;
Dörnyei, 2001; Gardner, 1985; Riemer, 2003). The studies emphasising intrinsic and extrinsic
motivation in relation to success in language education were led by Janíková (2011), Dunowski
(2015). Currently, the studies have emphasised motivation and self-efficacy, self-perception
(Dörnyei, 2014; Riemer, 2010) in relation to classroom management (Butt, Ahmad, & Ghazala,
2010) and others. The direction of motivational models’ research has been redirected to a wider
range of dimensions. Currently, they focus on learning experience in foreign language learning.
The motivational theories intend to capture complexity of motivational structure.
The emphasis of this research aimed at learning experience in relation to motivational
structure of learners, specifically, intrinsic motivation, personal goals, attractivity of FL
community, worries about learning. It was a multidimensional research encompassing subjective
feeling of success of learning German language, perception of personal relationship to teacher,
perception of knowledge by others. The aim of the research was to analyse learning experience
dimension and its impact on the perception of meaning of the expression German. Moreover,
it was to find correlations between learning experience, motivational structure, and years spent
by learning German language.
Motivational Theories and Learning Experience in Relation to

Foreign Language Education
Several experts in language education claimed the shift of perception of the motivation
concept in education in the 90s, (see Dörnyei, 2014; Kirchner, 2004; Riemer, 2003). The
importance of motivation in education has risen. The experts agreed on the essential role of
motivation as the most accessible determinant in the process of language acquisition, language
learning and teaching of foreign language. The amount of qualitative models of research has
risen. They pointed out the importance of interpersonal variability in the dimension. Several
studies confirmed the differences in the perception of motivation between the individuals as well
as in relation to the length of study. The studies confirmed the decreased levels of motivation
in the later years of studying the language (Juhásová, 2016; Riemer, 2003). In order to reflect
the dynamic character of motivation, it was essential to provide new models of motivation. The
new theoretical constructs integrated earlier knowledge as well as explanation of the situational
impact, and wider individual differences in motivation.
The social and educational theory of motivation by R. C. Gardner dominated in the 70s.
Based on the attitude towards language and language community there are two elementary
types of motivation (Masgoret, & Gardner, 2003). Integrative orientation is demonstrated by
the interest about the culture, the country, the people, and the community of the target language.
In opposition, there is an instrumental motivation demonstrated by the practical use of achieved
knowledge. Gardner`s theory points out social dimension of foreign language acquisition,
a learner compares himself/herself with the language community and culture (Kirchner, 2004).
Deci and Ryan (1993) defined motivation as a continuum. The beginning and the final two
points are the intrinsic and the extrinsic motivation. The intrinsic motivation is demonstrated by

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
curiosity, spontaneity and interest. The external motivation is summonned by external tasks, the 439
challenges from the external world. It is called self-determined motivation theory.
Earlier theories determined motivation as a subjective category. They highlighted the
importance of one dimension and put the two points in opposition, out of which only one
alternative was acceptable, wanted, for example, integrative against instrumental, intrinsic
against extrinsic motivation. They explained the positive aspect of motivation as an internal
conviction having nothing in common with the conditions of external social environment. The
external agents were perceived as less important aspects of motivation (Deci, & Ryan, 1993;
Kirchner, 2004; Masgoret, & Gardner, 2003). Also today a very often approach to the motivation
in relation to teaching and learning processes is to stress it as one of the key tasks of the teacher
professional performance (Gadušová, & Hašková, 2016, 2017; Hašková, & Dovalová, 2017;
Štúr, 2018).
In the current research studies motivation is defined as an attribution theory. Attributes
are cognitive processes. Learners use them to explain their success/unsuccess. Kirchner (2004)
considered the following as the most important attributes in relation to motivation:
- Searching for reasons of (un)success in themselves or in the others
- Considering the reasons as stable or changeable
- Accepting responsibility for the changes or redirecting responsibility to other people.
Based on The Theory of Attribution by Kirchner (Kirchner, 2004) it is possible to expect
higher levels of willingness to direct his/her own education when learners perceive the reasons
of their success mostly in the internal factors.
Considering methodology the research was rooted in the precise concept developed
by Dörnyei (2014). The model of foreign language FL motivational self-system emphasises
dynamic character of motivation integrating a variety of earlier motivational theories. Dörnyei
presupposes existence of possible selves, which play essential roles in the educational process.
The possible selves are the lively images about oneself in the future. According to Dörnyei
(2014) the ideal FL self ought-to-self play the key role in the system of selves. The ideal self
includes the set of goals. The concept of ought-to-self responses to the images is about the
expectations and responsibilities.
Dörnyei (2014) defined the term learning experience as the third dimension of
motivational self-system. It was defined as a “learner´s situation-specific motives related to
the immediate learning environment and experience”(Dörnyei, 2014, p. 521). The dimension
of learning experience includes the previous experience gained from the language acquisition
process and reflects social situation in the classroom during the process. Together with the
cognitive, intrapersonal attributes, it is tightly related to social level of learning, specifically,
the relationships learner-teacher, learner-learner, and the social dynamics in the classroom.
Furthermore, one of the essential aspects of learning experience is the experience of success in
foreign language education. The model directed attention to the notion of aims and supported
teachers to encourage their students to develop systematic, long-term and positive self-
representations in relation to foreign language learning. The current theory intends to explain
complexity of motivational structure of an individual and is placed in the system of external
educational environment agents.
Stranovská (2011) defined important determinants of motivation, for example, the
elements of social environment, social interaction in the classroom, the teacher-learner
relationship, and the factors of further social character, such as traditions, culture, and habits.
Lalinská (2014), Lalinská, Scheer (2014), Kondrla, Králik (2016) and Králik, Lenovsky,
Pavlikova (2018) emphasised the importance of further social character, specifically, culture in
the process of motivation. The essential elements of the motivational self-system by Dörnyei
(2014) are the previous FL experience taken from the social world.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
440 Problem of Research and Research Focus
Although there is a number of research studies dealing with the importance of the motifs
of the ideal and ought-to self, the research studies covering the issues of learning experience are
less apparent. Many times more factors unexpectedly enter the situation - also called ambiguity
- to the process of foreign language learning. Apparently, it is difficult to anticipate their
characteristics, content or to train the responses precisely. Probably, internal processing of such
ambiguous situations is reflected in the specific learning experience of an individual influence
on his/her overall motivation and his/her perception of the studied language and its culture.
That is why this research specifies the difference in the perception of learning experience in
relation to the meaning of the word German in the selected research categories. In this research
the relation between the categories of learning experience, motivational structure and the length
of studying a foreign language was examined. This research is based on the findings of the
current theories and examines motivation in a wider social context.
The following hypotheses were formulated with the examined categories: 1. Learning
experience, 2. Dimensions of motivational structure:
H1: A positive relation is assumed between learning experience and the selected
dimensions of motivational structure.
H2: A negative relation is assumed between learning experience and length of studying
German language.
H3: Perception of learning experience varies in relation to the meaning of the expression
German and depends on the categories Foreign language, Complexity, German speaking
countries, and Negative feelings.
General Background
This research was carried out at lower-secondary schools in Slovakia during the school
years of 2017, 2018. Although, the prevailing approach to the research was qualitative, the
quantitative analysis was also applied.
The motivational structure was operationalised as follows:

1. Intrinsic motivation –into what extent the learners are willing to learn German
language only to become wiser, know more.
2. Personal goals – into what extent learners consider German language knowledge
as important, as a part of future vocational development, as a part of preparation for
working abroad.
3. Attractivity of FL community – learners manifest their interest about FL community,
intend to develop relationships with the people from foreign community, considering
the German countries as the important subjects in the world matter.
4. Worries about learning – learners worry about their negative academic achievements,
about their parents` expectations.
5. Learning experience – the factors related to the experience of learning FL in the
classroom.
The central variable of learning experience consisted of the following items:
1. Extent of subjective feeling of success when using German language in the classroom,
2. Personal teacher-learner relationship perception,
3. How (the others) the classmates accept FL knowledge of the selected learners.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
Research Sample 441
The sample consisted of 84 Slovak learners studying German as a foreign language at the
lower-secondary schools (Table 1). The respondents were selected randomly from the lower-
secondary schools in the region of Nitra, Slovakia. Six schools were selected for this research,
each school was represented by one class of eighth graders; together there were 84 learners. The
respondents were 14-15 years old German language students. They had been studying German
language during the time span of 2 to 4 years. The proficiency level of the learners speaking
German as a foreign language was A1 and A2 (as represented in the frame of CEFR). The
selection of the sample was limited to the region of Nitra in the south-west of the country. The
sample had been monitored during the period of the years 2017 – 2018 as a part of the Research
Project (see the Acknowledgements). Except German, other foreign languages (French and
English) were also examined. The number of German as foreign language students was limited
by the number of the students available within the selected schools.
The research did not have any impact on the assessment of the individual learners, neither
the learners` identities were recorded in any form.
Table 1. Description of the research sample
Gender 43 female adolescents, 41 male adolescents

1st school: 14 learners, 2nd school: 11 learners, 3rd school: 18 learners,
Number of learners of selected schools
4th school: 13 learners, 5th school: 15 learners, 6th school: 13 learners
Length of study of German language 44 learners: 2 years, 29 learners: 3 years, 11 learners: 4 years
Instruments and Procedure
The motivational structure of learners and their associations to German language as

a foreign language were measured by the Questionnaire of Motivational Structure designed
by Dörnyei and Csizér (2002), adapted for the purpose of the research. It consisted of 12 items
including the dimensions of motivational structure, specified in the section General background.
The three items of the questionnaire measured the dimension learning experience. 7-point
Likert scale was used to get the responses. There was also one open question used asking the
respondents to express their own associations with the expression German.
The responses to the open question were coded by the categorical system, defined on
the bases of the empirical material designed specifically for the purpose of the research. The
categories for the content analysis were the following:
1. Foreign language – learners claimed German as a foreign language
2. Complexity – subjective impression of complexity of several areas of German
language system,
3. German speaking countries – name a German speaking country,
4. Negative feelings – implicit or explicit negatively verbalised feelings.
Data Analysis
The descriptive statistics was used for the purpose of the statistical processing and
the analysis of the collected data. The focus was laid on the description of basic statistical
indicators (mean, percentual mean, minimum, maximum, median, standard deviation).
Descriptive analysis was further applied in the three selected questions of the Motivational
Structure Questionnaire measuring the dimension of learning experience. In order to examine

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
442 the correlation between learning experience, the dimensions of motivational structure, and the
years of studying, Spearman correlation was used.
Considering the open question four categories of associations were tested. The categories
were as follows: Foreign language, Complexity, German speaking countries, Negative feelings.
When the tested categories occurred in the responses for 20 % of the respondents the variance
analysis ANOVA was used to test the difference in learning experience. The respondents were
divided into three categories based on the occurrence of the selected associations: 1) The group
of the respondents without the tested associations, 2) The respondents with one association in
the given category, 3) The respondents with multi-occurrence of the association in the category.
The difference of the average in the learning experience dimension was tested by the variance
analysis.
Research Results
Table 2 displays the results of descriptive analysis of the motivational structure

dimension in the group of the selected respondents. The arithmetic mean of the dimensions
of Attractivity of FL community, the Personal goals, and the Worries appeared to be the most
frequent motivational factors in the foreign language learning (the average was over 65 %). The
factors of Intrinsic motivation and Learning experience appeared as less motivating dimensions.
The factor of Learning experience achieved the lowest possible percentage values (the average
was 51.5 %).
Table 2. Descriptive results of the motivational structure dimensions.
Variable IM PG ALC W LE
Mean 8.429 9.19 9.119 9.131 10.81
Mean% 60.2 65.6 65.1 65.2 51.5
Median 8,5 9.5 9 9 11
Standard deviation 3.697 3.209 2.926 3.505 4.306
Minimum 2 2 2 2 3
Maximum 14 14 14 14 20
Explanations: Motivational structure dimensions IM – Intrinsic motivation, PG – personal goals, AlC –
attractivity of language community, W – worries about learning, LE – learning experience.
The items of the dimension learning experience are displayed in Table 3. Positive
learning experience occurred in the lowest possible extent in the question Q2 – acceptance of
language knowledge by the classmates (arithmetical mean 36.6%), the highest in the question
Q7 – personal relationship to the teacher perception and Q12 – the extent of subjective feeling
of success in German language in the classroom (arithmetical means 59.7% a 58.2%). The
opinions of the respondents were divergent, answering to the question Q7 – personal relationship
to the teacher perception (standard deviation 2.528)
Table 3. Descriptive results of the items of the dimension learning experience.
Variable Q2 Q7 Q12
Mean 2.560 4.179 4.071
Mean% 36.6 59.7 58.2
Median 2 4 4
Standard deviation 1.623 2.528 1.925
Minimum 1 1 1
Maximum 7 7 7

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
Learning experience dimension: Q2 – acceptance of the knowledge by the classmates, Q7 – perception of the 443
relationship to the teacher, Q12 – the feeling of success in German language.
In order to test the research hypotheses, the Spearman correlation was used. It examined
the relations between learning experience and the motivational structure. The same method was
applied to test the relations between the dimension of learning experience and the length of FL
study. The results are presented in Table 4. Analysis supported the hypothesis about the positive
relations between the factor learning experience and the dimensions of motivational structures,
specifically, the factor of intrinsic motivation (p = .001***), attractivity of FL community (p =
.042*), and personal goals (p = .0001***). The correlation was not confirmed in the case of the
dimension Worries. Moreover, the results of the analysis confirmed negative relation between
learning experience and the dimension years of studying the language (p = .022*).
Table 4. The results of correlations (Spearman rho) between the dimension of

learning experience and the other motivational structure dimension
specifically, the years of studying the language.
Study
IM ALC PG W
years
Correlation Coefficient -.250 .351 .223 .411 .016

LE
Significance (2-tailed) .022* .001*** .042* .0001*** .882
The values of statistical significance p = .05 (*), p = .01 (**), p = .001 (***)
IM – intrinsic motivation, PG – personal goals, ALC – attractivity of language community, W – worries
about learning, LE – learning experience
The motivational dimension of learning experience was tested in relation to word

association of the word German. The variance analysis ANOVA was used to test the interaction
between the selected groups of answers and the dimension of learning experience. The analysis
was processed in the following categories: Negative feelings, Language complexity, Foreign
language, German-speaking countries. Considering the categories language complexity and
foreign language, no statistically significant difference in the motivational dimension of
learning experience was found. However, there were statistically significant differences found
considering the other two categories as presented in the selected groups of the respondents:
negative feelings and German speaking countries as displayed in the Table 5. Although negative
feelings were related to the negative perception of educational situation as the variance analysis
showed, the category German speaking countries were closely related to the higher extent of
motivational dimension of learning experience.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
444 Table 5. Difference of the dimension of learning experience in relation to the

associations of the word German.
Negative feelings
Variable Occur-rence Occur-rence
Occur-rence ≤ 2 p/Lev. Anova_F p
Mean 0 1
LE 11.78 8.05 10 .798 6.464 .002**
Complexity
Occur-rence ≤ 2 p/Lev. Anova_F p-value
Mean 0 1
LE 10.92 10 11.13 .2405 .251 .779
Foreign Language
Mean 0 1
LE 10.60 11.33 10.52 .2346 .723 .488
German Speaking Countries

Mean 0 1
LE 10.12 13.71 12 .191 4.538 .014*
Statistical significance at the level p = .05 (*), p = .01 (**), p = .001 (***)
LE – learning experience, p/Lev. – significance of Levene`s test
Discussion
The research purpose was to examine motivational structure and its relation to the variable
Learning experience of the lower secondary school learners. The idea of early motivation in
foreign language education was supported through the research results. In order to motivate
students to learn a foreign language, it is essential to get them motivated in early years of
studying the language. Therefore, this research focused on the relations between the dimension
of learning experience and the other selected motivational structure dimensions, specifically,
the years of foreign language study. Moreover, this research conducted qualitative analysis of
learning experience in relation to associations with the word German in the selected categories.
Firstly, motivational structure of learners was descriptively analysed. The dimensions of
Attractivity of language community, Personal goals, and Worries were demonstrated as the
most frequent motivational factors in foreign language learning. Based on the findings, the
motivational structure of German language learners was mostly extrinsic.
Apparently, it was related to the age and the fact that the respondents were beginners
perceiving more the external character of the language and the grammatical structure of the
language. The external motivation is considerably important as well as the internal motivation.
Deci and Ryan (1993) pointed out the dynamic character of external motivation. Integrated
external motivation may become the initial entry into the internal motivation in the later years
of foreign language learning. Descriptive analysis of the dimension of learning experience
demonstrated almost no importance in the dimension of acceptance of knowledge by the
classmates. The relationship to the teacher as well as the feelings of success in the process of
learning German language was perceived as average and above average (59.7% and 58.2%).
Apparently, the personality and the role of a teacher and the feelings of success when learning
German language were confirmed as vital in the educational environment of the lower secondary
schools.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
Moreover, the research results showed the relations of the dimension of learning 445
experience and the selected dimensions of motivational structure and the years of foreign
language studying. The results show that the dimension of learning experience is related to
intrinsic motivation, attractivity of language community and personal goals. Based on the
correlation analysis, it was supposed that perception of learning experience would be an integral
element of the motivational structure tightly related to the dimensions of ideal self, specifically,
the images tied to knowledge and the abilities learners desire to have. Although some of the
results presented lack of intrinsic motivation in the later years of studying, extrinsic motivation
was also absent over the process of studying. Moreover, the results of Riemer (2003, 2010) and
Juhásová (2016) also emphasised the importance of learning experience and the ideal self in
the process of foreign language learning. They claimed that both influence the further stages of
foreign language development. They focused on learning experience as a significant dimension
in motivational structure of an individual. The research results showed the same significance
of learning experience as the results by Riemer (2003, 2010). Apparently, learning experience
of a lower secondary school learner (A1 level) is a predictor of their motivational structure.
Moreover, the relation between learning experience and the length of study was examined,
too. The longer learners study, the less they are motivated to learn the foreign language. Such
significant results were found together with Riemer (2003, 2010) and Juhásová (2016). They
showed the decrease of extent of overall motivation to learning in case of learning experiences
in relation to the length of study. A certain negative role is also played by the specific structure
of German language. Apparently, it takes longer time from 2 -4 years until the structure of
German language is acquired which may cause a decrease of motivation of learners.
Qualitative analysis of the dimension of learning experience in relation to the expression
German was carried out in the categories of 1) Complexity, 2) Foreign language, 3) Negative
feelings, and 4) German speaking countries. The results manifested the difference between the
perception of learning experience in relation to the associations with the expression German.
Although, the category German speaking countries was related to the extent of motivation,
specifically, to the positive perception of learning experience, the category Negative feelings
was related to negative perception of the dimension of learning experience. Probably, the
relations between education and reality of German speaking countries increased a positive
perception and motivation to learning of German language. The category Negative feelings was
related to the dimension of learning experience in case of low level of motivation. Apparently,
the respondents perceived German language more as a school subject, than a real need to learn
to communicate. They had experienced no real touch with the language outside the school
environment; neither had they experienced any touch with the foreign language community.
Together with Dörnyei (2014) the results of this research demonstrated the difference between
the ideal self and ought-to-self. Although the ideal self is focused on the self-determined
achievement of objectives, ought-to-self relates only to its attributions of the expectations of
the others and the tasks set by the others. The intentions of the group of learners having the
dominant ought-to-self are to have it over, learn, without any specific goal setting. Having
intentions in the process of learning appears to be one of the essential factors tightly related to
the motivational structure and learning experience of learners. As Dörnyei (2014) suggested
motivation relates to positive, long-lasting, and systematic self-determination of individuals. As
the results of this research demonstrated, positive self-determination is related to motivation to
learn German language supporting positive learning experience.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
446 Conclusions
The research results demonstrated learning experience as the essential indicator

of motivation to learning foreign languages. Learners considered attractivity of language
community, personal goals, and intrinsic motivation as important in the context of foreign
language learning. The main contribution of this research was to examine the field of learning
motivation in relation to the dimension of learning experience and the other dimensions of
motivational structure, specifically, the ideal self in the process of learning German as a foreign
language. Considerably, this research brings novelty to the research field of learning experience
- as a factor of motivational structure of a foreign language learner - showing that motivation
to learn a foreign language in the later years of study does not explicitly relate only to the
intrinsic or extrinsic motivation. Many times it is also related to the overall learning experience.
Apparently, this dimension of positive learning experience is a predictor of motivational
structure of learners-beginners of German language. The results also showed a negative relation
between learning experience and the length of FL learning.
Moreover, the research results also supported the hypothesis of difference of perception
of learning experience in relation to the perception of the expression German. Although, the
category of German speaking countries was related with higher extent of motivation, the
expression of negative feelings occurred in relation to the negative perception of learning
experience.
Generally, the results of this research contribute to the research in education. The
results may also encourage teachers to consider the need for positive learning experience in
education. Although strong intrinsic motivation is a positive phenomenon in education, it is
necessary to encourage it through reinforcement of external goals and development of positive
learning experience in the classroom and overall positive self-determination. The relations
between a positive learning experience and the category German speaking countries points out
the importance of real-life experience with the target language community and their culture.
Such a need of the learners should be reflected in the educational settings.
The current research opened up new research questions. Research of motivation in
foreign language learning is a complex phenomenon. Further research should specify those
essential predictors of motivation, which are necessary for teachers to develop in the process of
education. One of the further research aims, is also to widen qualitative analysis of motivational
structure as perceived in the expression German in the further relevant categories. The limitation
of this research is the number of the samples therefore, it is necessary to verify the results of this
research also in the other regions of the country, as well as the other age groups of the learners.
Acknowledgements
This work was supported by the Slovak Research and Development Agency under the
contract No. APVV-17-0071 and National Agency for Research VEGA 1/0062/19.
References
Butt, A. T., Ahmad, T., & Ghazala, R. (2010). School and classroom management. Lahore: Majeed Book
Depot, Urdu Bazar.
Deci, E. L., & Ryan, R.M. (1993). Die Selbstbestimmungstheorie der Motivation und ihre Bedetung
für die Pädagogik [The self-determination theory of motivation and its relevance to education].
Zeitschrift für Pädagogik, 39(2), 223–238.
Dörnyei, Z. (2001). Motivational strategies in the language classroom. Cambridge: Cambridge University
Press. doi: 10.1017/CBO9780511667343

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
Dörnyei, Z. (2014). Motivation in second language learning. In M. Celce-Murcia, D. M. Brinton & M. A. 447
Snow (Eds.), Teaching English as second or foreign language (pp. 518–531). Retrieved from http://
www.zoltandornyei.co.uk/uploads/2014-dornyei-cengage.pdf.
Dörnyei, Z., & Csizér, K. (2002). Some dynamics of language attitudes and motivation: Results of a
longitudinal nationwide survey. Applied Linguistics, 23(4), 421–462. Retrieved from http://www.
zoltandornyei.co.uk/uploads/2002-dornyei-csizer-al.pdf.
Dunowski, E. (2015). Motivation im fachbezogenen DaF-Unterricht bei tschechischen Studierenden: erste
Ergebnisse einer qualitativen Studie [Motivation in subject-related lessons of German as a second
language of Czech students: First results of a qualitative study]. In V. Janíková, & H. Andrášová
(Eds.), Deutsch ohne Grenzen: Didaktik Deutsch als Fremdsprache (pp. 108–123). Brno: Tribun.
Gadušová, Z., & Hašková, A. (2016). Evaluation of teachers’ competences as a part of school leadership.
In H. Zhang (Ed.), Lecture Notes in Management Science: Vol. 50 (pp. 7–12). Singapore: Singapore
Management and Sports Science Institute. doi: 10.5729/lnms.vol50.7
Gadušová, Z., & Hašková, A. (2017). Results of an on-line survey on competence profiles of primary
and secondary school teachers. In W. Fok, & V. Wenting Li (Eds.), Teaching and Learning with
Technology (pp. 271–281). Singapore: World Scientific Publishing.
Gardner, R. C. (1985). Social psychology and second language learning: The role of attitudes and
motivation. London: Edward Arnold.
Hašková, A., & Dovalová, D. (2017). How to evaluate teachers` professional competence. In L. G. Chova,
A. L. Martínez, & I. C. Torres (Eds.), ICERI2017 Proceedings. 10th International Conference of
Education, Research and Innovation (pp. 7147–7150). doi: 10.21125/iceri.2017.1910
Janíková, V. (2011). Výuka cizích jazyků [Teaching foreign languages]. Praha: Grada.
Juhásová, J. (2016). Sprachbewusstsein und ihr Einsatz von Jugendlichen, die Deutsch als Tertiärsprache
in der Slowakei lernen. [Language awareness and ist use by young people learning German as L3
in Slovakia] Zeitschrift für Interkulturellen Fremdsprachenunterricht, 21(2), 27–41. Retrieved from
https://tujournals.ulb.tu-darmstadt.de/index.php/zif/.
Kaliská, L. (2013). Vyučovanie zamerané na učebné štýly. [Learning Styles in Education.] Banská
Bystrica: PF UMB.
Kirchner, K. (2004). Motivation beim Fremdsprachenerwerb: Eine qualitative Pilotsudie zur Motivation
schwedischer Deutschlerner [Motivation in second language acquisition: A qualitative pilot study to
motivation of Swedish learners of German]. Zeitschrift für Interkulturellen Fremdsprachenunterricht,
9(2). Retrieved from https://tujournals.ulb.tu-darmstadt.de/index.php/zif/.
Kondrla, P., & Králik, R. (2016). Authentic being and moral conscience. European Journal of Science
and Theology, 12(4), 155–164.
Králik, R., Lenovsky, L., & Pavlikova, M. (2018). A few comments on identity and culture of one ethnic
minority in Central Europe. European Journal of Science and Theology, 14(6), 63–76.
Lalinská, M. (2014). Between tradition and present: Implementation of folk art and culture into the foreign
language education of high school students. In S. Pokrivčáková (Ed.), LLCE2014: Proceedings (pp.
176–187), Nitra: SlovakEdu.
Lalinská, M., & Scheerová, J. (2014). Kulturologický fenomén ľudového umenia vo vyučovaní
francúzskeho jazyka [Culturological phenomenon in folk arts in French language education].
Disputationes scientificae, XIV(4), 35–47.
Masgoret, A. M., & Gardner, R.C. (2003). Attitudes, motivation and second language learning: A meta-
analysis of studies conducted by Gardner and associates. Language Learning, 53(1), 123–163. doi:
10.1111/1467-9922.00227
Oravcová, J. (2000). Psychologické aspekty vizuálnej komunikácie [Psychological aspects of visual
communication]. Banská Bystrica: UMB.
Riemer, C. (2003). „Englisch war für mich nur ein Teil meines Stundenplans“ – Motivation zum
Englischlernen in Zeiten der Globalisierung [„English was for me only part of my timetable“
– motivation to learn English in the times of globalisation]. Zeitschrift für Interkulturellen
Fremdsprachenunterricht, 8(2-3), 72–96.
Riemer, C. (2010). Motivation. In H. Barkowski, & H.-J. Krumm (Eds.), Fachlexikon Deutsch als Fremd-
und Zweitsprache. Tübingen, Basel: Francke.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
448 Sarmány-Schuller, I. (1992). Kognitívne štýly – súčasný stav a perspektívy [Cognitive Styles – The
Current Situation and Perspectives]. In Z. Ruiselová, & I. Ruisel (Eds.), Praktická inteligencia I.
Teoretické a metodologické problémy (pp. 89–106). Bratislava: ÚEP.
Stranovská, E. (2009). Mosty, prieniky a perspektívy efektívneho štúdia I. [Bridges, meeting points and
perspectives of effective learning I.]. Nitra: ASPA.
Stranovská, E. (2011). Psycholingvistika: Determinanty osvojovania si a učenia sa cudzieho jazyka
a kultúry. [Psycholinguistics: Determinants of foreign language and culture acquisition]. Brno:
MSD.
Štúr, M. (2018). Metaphor as a criterium of shifts in understanding language and communication. World
Literature Studies, 10(3), 68–85.
Eva Stranovská PhD, Associate Professor, Constantine the Philosopher University, Nitra,
(Corresponding author) Slovakia.
E-mail: estranovska@ukf.sk
Anikó Ficzere PhD. Student, Constantine the Philosopher University in Nitra, Štefánikova
67, 949 74 Nitra, Slovakia.
E-mail: aniko.ficzere@ukf.sk
Silvia Hvozdíková PhD, Constantine the Philosopher University in Nitra, Štefánikova 67, 949 74
Nitra, Slovakia.
E-mail: shozdikova@ukf.sk
Beáta Hockicková PhD, Constantine the Philosopher University in Nitra, Štefánikova 67, 949 74
Nitra, Slovakia.
E-mail: bhockickova@ukf.sk

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
449
GUIDE FOR AUTHORS
ISSN 1822-7864 /print/ ISSN 2538-7111 /Online/
General Information
The publication language is English. All authors must take care of the language revision on their
own. The language must be clear and accurate. The work should be written in an impersonal style.
The editor reserves the right to send the manuscript to be reviewed. If English is a second language
for the author, please consider having the manuscript proof read and edited before submitting.
Format of Manuscripts
Title
The main research idea/problem should be reflected in the title. The title should show the nature
of the research/study. It is recommended to avoid the title of the question form. The recommended
length for a title is no more than 12 words (APA, 2.01, p. 23).
The title of the paper 14-point, bold with capital letters, align left; titles in the text (chapters) 12-point
bold, not numbered; sub-titles (subchapters) 12-point, italic, not numbered. There must be 1 empty
line before and after a title or a subtitle. The text chapters must be separated by 1 empty row. The
title should be concise and informative. Avoid abbreviations.
Author names and affiliations
Full names/surnames should be provided. Please indicate affiliations of the author(s). All the e-mails
should be indicated below.
Abstract
The first page of the manuscript must begin with the title of the paper and an abstract which should
be about 150 - 250 words. For the papers reporting original research, state in brief: the primary/
main aim (the research questions addressed or any hypothesis tested); the research design; the
methods and procedures employed; the number of participants; the main outcomes and results; the
conclusions drawn from these data and results, including their implications for further research or
application/practice. An abstract represents briefly a content of a text. Do not cite references in the
abstract. The abstract should grip the reader’s attention.
Keywords
Please provide 3 to 5 keywords in alphabetical order. Note that a keyword does not have to be made
of only one word. At least one of these should indicate the topic area and one should indicate the
methodology of the research.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
450 The structure of the manuscript (recommended)

The body of the text of the manuscript must generally have the following parts:
• abstract + keywords;
• introduction (it can be divided into some subchapters if needed);
• methodology of research (it is recommended to divide /rubricate);
• results of research;
• discussion;
• conclusions and/or implications;
• acknowledgements (if any);
• references (in APA style, 6thEd);
• appendix (if any).
P.S. The structure can be different if the paper is only theoretical qualitative research.
The length of the manuscript
The manuscript should be not shorter than 6 pages (including references, tables and figures).
Manuscripts should be typed on A4, in Times New Roman 12-point font size, single-spaced, 2.5
cm with all the margins, word-document (Word 6.0 or later) format.
Acronyms
Acronyms should be defined the first time they appear.
Tables and figures
Tables and figures should be valuable, relevant, and visually attractive. Tables (made in Word or
another software) and figures must be referred to in the text and numbered in the order of their
appearance. Each table and figure should have a complete, descriptive title; and each table column
an appropriate heading. The texts in tables and figures should be 11-point (in some cases 10) and
their width should be 12 cm at maximum. The figures should be in format .jpg (unless done in
Word or Excel); resolution 1200 dpi. Figures, tables (black and white colour, without ground-
colour) and captions should be inserted within the manuscript at their appropriate locations. All the
graphics (figures) must be editable. The font size should be Times New Roman for all figures and
tables. Figures should be carefully explained in the text and cited in numerical order.
It is the author’s responsibility to obtain permission to reproduce illustrations, tables, etc. from
other publications.
Discussion
Focus the discussion to the two important questions: What is already known about your topic?
What your research adds? It is recommended that the aim and major findings of the research
should be reminded. And then the similarities and differences of the findings with previous
research studies should be exemplified in detail.
Conclusions
This part is not a summary. Bring out the significance of your research. Show how you’ve brought
closure to the research problem, and point out remaining gaps in knowledge by suggesting issues
for further research. The main research outcome should be clearly seen.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 77, No. 3, 2019
References 451
The title „References“ must be used. APA style for writing references in the text and in the
reference list must be used. References in the text should be presented in parentheses (Knox, 1988;
Martin, 1995). If necessary, the page can be indicated: (Martin, 1995, p. 48). The list of references
should be presented after the text.
The author should make sure that there is a strict one-to-one correspondence between the names
and years in the text and those on the list. All the references should be listed in alphabeti¬cal order
by author’s name.
For the sake of accuracy, references should always be as updated as possible, also in the interest of
the readers and researchers on the topic the paper is dealing with.
Notes
Note 1: Manuscripts in which references are not in the APAstyle will be returned without review.
Note 2: References to online sources should include the type of medium (such as “serial online”
or “monograph online”), the date of that specific reference (if applicable), the uniform resource
locator (URL), and the date that the source was accessed. A source accessed online should al-ways
be referenced accordingly, even if it is also published in printed form.
Note 3: All papers must meet the criteria of originality and scientific quality. Obviously, they must
also comply with style and format requirements. The paper will not be subject to further review,
if the manuscript is NOT WITHIN THE SCOPE and/or there is POOR USAGE OF LANGUAGE
(all manuscripts must be written in clear and grammatical English).
Note 4: Submitted papers will be assessed based on their novelty, technical quality, potential
impact, and clarity of writing.
Note 5: All papers are checked by CrossCheck system.
Editorial Board
Updated: July 2018
Website: http://www.scientiasocialis.lt/pec/

Problems of Education in the 21st Century,
Vol. 77, No. 3, 2019
Compiler Vincentas Lamanauskas

Designer Jurgina Jankauskienė
Paste-up artist Loreta Šimutytė-Balčiūnienė
English language proofreader Ilona Ratkevičienė
21 June 2019. Publishing in Quires 8,875. Edition 200.
Publisher Scientia Socialis Ltd. in cooperation with SMC „Scientia Educologica“

Donelaicio Street 29, Siauliai, Lithuania
E-mail: gu@gu.puslapiai.lt
Phone: +370 687 95668

Printing „Šiaulių spaustuvė“ Ltd.

9A P. Lukšio Street
LT-76207 Šiauliai, Lithuania
Phone: +370 41 500 333.
Fax: +370 41 500 336
E-mail: info@dailu.lt

Problems of Education in The 21st Century, Vol. 77, No. 3, 2019

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Problems of Education in The 21st Century, Vol. 77, No. 3, 2019

Uploaded by

Copyright:

Available Formats

Problems of Education in the 21st Century, Vol. 77, No.

Address: Scientia Socialis, Ltd.

EDUCATION AS A HUMAN RIGHT AGAINST HATE SPEECH AND INTOLERANCE

CHARACTERISTICS OF INDEPENDENT SCHOOLS DIRECTED AT STUDENTS IN

DEVELOPING OF FACTOR STRUCTURE FOR LEARNING STRATEGIES OF ESTONIAN

CLASSIFICATION OF ENGINEERING STUDENTS' SELF-EFFICACY TOWARDS

THE FORMATIVE RESEARCH OF ECUADORIAN UNIVERSITY TEACHING STAFF

ON PERCEPTION OF COMPUTER ALGEBRA SYSTEMS AND MICROSOFT

EFFICIENCY OF PUBLIC SPENDING ON HIGHER EDUCATION:

EXPLORING THE STUDENT AND SOCIAL ACCOUNTABILITY OF THE LIFE

THE OPINIONS OF PRE-PRIMARY EDUCATION TEACHERS ON THE ADVANTAGES

LEARNING EXPERIENCE AS A FACTOR OF MOTIVATION IN LOWER-SECONDARY

INSTRUCTIONS FOR AUTHORS ............................................................................. 449

ISSN 1822-7864 (Print) ISSN 2538-7111 (Online)

https://doi.org/10.33225/pec/19.77.314 ISSN 1822-7864 (Print) ISSN 2538-7111 (Online)

ISSN 1822-7864 (Print) ISSN 2538-7111 (Online) https://doi.org/10.33225/pec/19.77.314

United Nations – Sustainable Development Goals – 2030 Agenda. https://www.un.org/

https://doi.org/10.33225/pec/19.77.314 ISSN 1822-7864 (Print) ISSN 2538-7111 (Online)

CHARACTERISTICS OF INDEPENDENT 317

SCHOOLS DIRECTED AT STUDENTS IN

ISSN 1822-7864 (Print) ISSN 2538-7111 (Online) https://doi.org/10.33225/pec/19.77.317

The Emergence of Independent Schools in Sweden

https://doi.org/10.33225/pec/19.77.317 ISSN 1822-7864 (Print) ISSN 2538-7111 (Online)

Independent School Support for Students in Difficulties

ISSN 1822-7864 (Print) ISSN 2538-7111 (Online) https://doi.org/10.33225/pec/19.77.317

https://doi.org/10.33225/pec/19.77.317 ISSN 1822-7864 (Print) ISSN 2538-7111 (Online)

Conditions and Prerequisites for Learning and Development

Schooling directed to students in need of support is characterised by complexity. To

ISSN 1822-7864 (Print) ISSN 2538-7111 (Online) https://doi.org/10.33225/pec/19.77.317

A qualitative approach characterized the design of the research based on a content

https://doi.org/10.33225/pec/19.77.317 ISSN 1822-7864 (Print) ISSN 2538-7111 (Online)

ISSN 1822-7864 (Print) ISSN 2538-7111 (Online) https://doi.org/10.33225/pec/19.77.317

Figure 1. Data analyses.

Reliability and Ethical Aspects

https://doi.org/10.33225/pec/19.77.317 ISSN 1822-7864 (Print) ISSN 2538-7111 (Online)

Research Results 325

Target Group for the Schools

According to the classification of Swedish municipalities produced by the Swedish

School Organisation, Content and Implementation

ISSN 1822-7864 (Print) ISSN 2538-7111 (Online) https://doi.org/10.33225/pec/19.77.317

326 1. Schooling only (9 schools)

Figure 2. Characteristics of the independent schools in terms of organisation of

Access to Staff and Competencies

https://doi.org/10.33225/pec/19.77.317 ISSN 1822-7864 (Print) ISSN 2538-7111 (Online)

Student Number and Ages 327

Creating Relations and Cooperation

Ways of Working and Specific Methods

ISSN 1822-7864 (Print) ISSN 2538-7111 (Online) https://doi.org/10.33225/pec/19.77.317

328 Governing and Guiding Documents

Formulating Goals and Visions

Table 1. Descriptions of goals and visions.

Thematization of descriptions of goals and con-

The students should achieve the best study results as

https://doi.org/10.33225/pec/19.77.317 ISSN 1822-7864 (Print) ISSN 2538-7111 (Online)

A Follow up Research of Websites from Independent Schools

ISSN 1822-7864 (Print) ISSN 2538-7111 (Online) https://doi.org/10.33225/pec/19.77.317

Follow Up and Evaluation

https://doi.org/10.33225/pec/19.77.317 ISSN 1822-7864 (Print) ISSN 2538-7111 (Online)

Creating Relations and Organising Teaching

In the presentations from the independent schools, the importance of well-functioning

Limitations and Further Research

ISSN 1822-7864 (Print) ISSN 2538-7111 (Online) https://doi.org/10.33225/pec/19.77.317

An underlying neuropsychiatric and psychological paradigm seems to exist emphasising