Bibliometric Analysis of Multimodality Based on Multimodal Learning

ed
Mochamad Bayu Firmansyah1, Anak Agung Ayu Dian Andriyani2, Ria Arista Asih3*

1School of Psychology and Pedagogy, Universitas PGRI Wiranegara, Indonesia

Email: firmansyahbayu970@gmail.com

iew
2Facultyof Foreign Language, Universitas Mahasaraswati, Indonesia
Email: agungdianjepang@unmas.ac.id

3*Schoolof Pedagogy, Universitas Muhammadiyah Malang, Indonesia

ev
Email: aristaria86@umm.ac.id

Abstract
Multimodality is potentially valuable for the educational process, despite its lacking of extensive

r
studies and development. This study aims to investigate the multimodal learning publication trend
in higher education as well as axiological considerations for future research growth. The Google

er
Scholar database was used as the data source and VOSViewer was employed for data processing
in this bibliometric study. The results show the gap in transdisciplinary studies employing
modalities, namely, language, images, music, sound, and movement despite collaboration across
pe
various fields, such as industry, medicine, and transportation. Few collaborative networks between
institutions limited findings that are unique to specific institutions and particular countries. It is
suggested that axiological principles are utilised when integrating the industrial, medicinal, and
transportation fields in universities enabling cross-country research collaborations, multimodal
learning, and academic integration.
ot

Keywords: Bibliometric Analysis, Multimodality, Multimodal Learning, Research trend,

VosViewer
tn

Introduction
Research on multimodal learning aims to integrate, be dynamic, and be innovative in
response to changing learning needs (Firmansyah, 2018; Mondada, 2016; Nina Nørgaard, 2009).
Multimodal learning research has been published since 1994 as recorded by Google Scholar. The
rin

production of knowledge and meaning through interaction and the integration of text coherence is
represented by multimodality in social semiotics. According to the way language is understood,
multimodality is social semiotic.
Language study focuses primarily on discourse, where language acts as a depiction of
ep

social semiotics. These aspects of multimodality are methods, while multimodality is the medium
(Firmansyah et al., 2020; Lee, 2014; Lim-Fei, 2013). Language, images, music, sound, and
movement are all common modality kinds used in multimodal educational experiences to create
and retrieve knowledge (Hermawan, 2013). In the framework of higher education, multimodal
Pr

learning makes a significant contribution, particularly in the development of 21st-century

competencies and learning based on regional wisdom (Baharudin, 2010; Nesi et al., 2010; Ryu &

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4298216
 Boggs, 2016). Efforts to increase 21st-century capabilities must focus on learning systems and

ed
competencies with media that serve as a reference in the process of continuous learning (Coccetta,
2018).
Serious efforts and care are required from the system supplier and system implementers,
namely lecturers and students, to improve competency and multimodal learning media. These

iew
initiatives are linked together and can serve as a systematic runway. Learning results in experiential
learning in the context of adult learners (andragogy), which is an adaptation to the capacity to
understand and process data, technology, and human interaction based on concrete experiences
from various modalities (language, images, music, sound, and movement), which are then regarded
and transformed into new ideas and concept development.
The information that pupils have found, however, is not the best for enhancing their

ev
multimodal competence. This is due to several factors, including the following: students have not
prioritized real learning in relevant contexts; they have not been optimal in formal and non-formal
learning processes; they have not been optimal in motivating themselves in learning so that the
context of new interactions is neglected, and they have not eventually realised that learning needs

r
to be directed to construct experience (Firmansyah & Rokhmawan, 2018; Firmansyah et al., 2020).
As part of a student’s multimodal competence, it is important to instil in them from the beginning

er
the capacity to comprehend cultural signs in a variety of ways of producing meaning that involve
the use of language, speech, and visuals from a variety of online resources in a variety of real-
world communications.
pe
In the present era, students need to be multimodally competent. It is essential to develop a
student competence orientation that focuses on learning and innovation abilities, life and career
skills, as well as the fields of technology, media, and information. This is in line with 21st-century
learning principles, which place a significant emphasis on students’ capacity to conceive problems,
gather knowledge from multiple sources, critically think, and collaborate to solve them. Among
the 21st-century learning are: (a) Critical-Thinking and Problem-Solving Skills: Skilled of thinking
ot

critically, laterally, and methodically, particularly when addressing problems; (b) Communication
and collaboration skills, including the capacity to cooperate and communicate effectively with a
variety of partners; (c) Critical-Thinking and Problem-Solving Skills, including the capacity to
tn

think laterally and systematically, particularly when tackling problems; (d) Communication and
collaboration skills, including the capacity to cooperate and communicate effectively with a variety
of partners; (e) Possessing the capacity to create and innovate (creativity and innovation skills),
and being able to use their creativity to make a variety of ground-breaking discoveries; (f)
rin

Information and communication technologies proficiency, such as the ability to use technology to
enhance daily tasks and performance; (g) Contextual Learning Skills, which include the ability to
conduct autonomous learning activities in an environment that is relevant to one’s own growth,
and (h) Information and media literacy abilities, including the capacity to comprehend and apply
ep

a range of communication mediums in order to communicate a range of ideas and conduct

cooperative activities and interactions with such a variety of parties.
Understanding cultural signs in different ways of conveying information and making
meaning utilizing methods and media incorporating numerous modalities is known as multimodal
Pr

competency (language, images, music, sounds, movements) from many websites in multiple real
communications (Lim, 2011; Pinar, 2019). Through social semiotics, a consequence of social

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4298216
 processes, language usage is connected to facts. Language, visuals, music, sound, and movement

ed
are all used concurrently and in sync with one another in this situation. Language incorporates
modalities (language, pictures, music, sounds, and gestures) that are framed in discourse to enable
the interchange of meaning in interpersonal situations. This system of meaning information at a
concrete level is the context of multimodality incompetence. To increase multimodal competency

iew
(language, picture, music, sound, and movement) in the learning system, online activities with a
focus on participation in personally and professionally relevant material are used (Cheng & Liu,
2014). Additionally, the reason for using it varies, ranging from utilitarian uses to intellectual,
informative, social, and entertainment ones.
Research on multimodal learning is being developed in the developing world to address
demands for education in personal, social, and cultural contexts. A systematic study on the growth

ev
of multimodal learning shows a pattern of integration with business, health, and transportation
systems (Calantonea et al., 2002; Chi et al., 2009; Ebrahimi et al., 2016; Pichler et al., 2010; Pinar,
2019). Several academics, including (Calantonea et al., 2002; Chi et al., 2009; Ebrahimi et al.,
2016; Pichler et al., 2010; Pinar, 2019) have adjusted multimodality in learning to meet the

r
objectives of the business, the medical community, and transportation systems to obtain the desired
results. However, a further in-depth study is still required to fully understand the focus of
multimodal learning during the past ten years.
er
Worldwide and in Indonesia, multimodal learning research is still expanding. Studies on
multimodality that concentrate on the domains of business, health, and transportation are
pe
numerous, which is a sign of the growth of multimodal learning research that is integrated with
varied demands. Many professional institutions that are the research topics might be viewed as
sources for this research. The trajectory of multimodality research must thus be mapped, with a
particular emphasis on learning at Indonesian and international institutions. An endeavour to
develop research in learning in higher education is a tremendous opportunity for multimodal
learning research. Numerous researchers investigate multimodality utilizing a notion modified in
ot

response to development requirements, particularly in terms of industry (Pichler et al., 2010; Pinar,
2019; Salimova et al., 2019), health (Chi et al., 2009), related to transportation (Bayoudh et al.,
2022; Hung et al., 2013). It may or may not be pertinent to multimodal learning in higher
tn

education. (Calantonea et al., 2002; Chi et al., 2009; Ebrahimi et al., 2016; Pichler et al., 2010;
Pinar, 2019). However, although it is still uncommon, modality research has also been done in the
field of education. Examples of this study include (Firmansyah, 2021b; Freyn & Ed, 2017;
Mondada, 2016; Nouri, 2019). The subjects and patterns that are the focus of multimodal research
rin

have not, however, been attempted to map in any of these studies. Hence, this present study
employs a bibliometric technique to examine the multimodal research trend.
The results of this study are anticipated to explain the integrity and research landscape for
multimodal learning in higher education, as well as the potential for research collaboration
ep

between researchers from different universities around the world and their cultural backgrounds,
as well as the possibility that multimodal learning in higher education has a uniqueness that is
relevant and unique. The objective is to increase the global community’s accessibility to research
on multimodal learning in higher education, to grow professionally, to adapt to the demands of
Pr

multimodal learning in higher education, and to continue to be appealing.

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4298216
 Bibliometric analysis is fundamental to developing multimodal learning in higher

ed
education. This study demonstrates the global trend of multimodal learning research development
and its consequences for the growth of multimodality theory in general and multimodal learning
research in universities in particular. Through an examination of the diction choices made in
university-based multimodal learning research, this study will also shed light on the impact factors

iew
of academic publications in a variety of multimodality-related fields as well as emerging trends in
the use of multimodal approaches and media in learning.

Methods
The bibliometric analysis used in this study. Finding bibliographic data as research data is done
using a quantitative technique called the bibliometric analysis approach (Alves et al., 2022;

ev
Philosophy et al., 2020; Rusydiana, 2019; Zamroni et al., 2022). Performance analysis in science
mapping is possible because of the bibliometric analysis assessment domain.
Software programs are used in the bibliometric investigation (Alves et al., 2022; Kurniasih
et al., 2020) that is, the usage of terms that denote the study’s variables. By measuring the diction

r
utilised, keyword analysis is based on accuracy analysis to examine the content, patterns, and
trends of the document collection (Berry et al., 1997). The number of times the research’s

er
keywords appear in the article under consideration is calculated using the diction analysis
approach. The article title or abstract, where a common diction analysis map may be found,
contains keyword maps ranked by correctness, significance, or distinctiveness. As a result, the
pe
researcher believes that the research’s findings should be seen and investigated as a reference for
the notion. Researchers can identify author standardization with the use of software and thesaurus.
The bibliometric analysis approach analyzes research in scientific journals by looking at
impact factors and citations, collaboration trends, distribution strategies, and reports on research
productivity (Alves et al., 2022). To acquire the description, this study divides the exposure to data
into four categories: publishing sources, publication impact factors, keywords utilized, research
ot

networks, and researcher productivity.

Google Scholar was utilized to gather data for this study, which included papers on the
growth trend of multimodality research in institutions. Google Scholar is regarded as a search
tn

engine that can accommodate more study findings and scientific publications from academics,
practitioners, and researchers on the subject of multimodal learning in universities. To gather data,
publish and perish software was used to search Google Scholar publications for articles containing
the terms multimodality, multimodal learning, and multimodality in the context of learning in
rin

universities. The data is displayed as annual publishing totals. Microsoft Excel offers a journal
containing articles on the evolution of college administration and research on multimodal learning
in universities, authors, origins of writers, and themes. The growth of multimodal learning research
is examined using the VosViewer program.
ep

Results and Discussion

Results
Reference of a source. This bibliometric study employs the Google Scholar database, which was
Pr

accessed on October 1, 2021, as the source of publication. The Google Scholar database was
chosen since it contained the bulk of research articles, particularly those that dealt with the topic

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4298216
 of multimodal learning in higher education. With the terms multimodality, multimodal learning,

ed
and multimodality in the context of college learning, the search is conducted using Harzing's
publish or perish program. Table 1 displays the findings of the distribution of research for each
year.
No Year Article Percentage

iew
1 1994 10 2,40
2 1996 2 0,48
3 2001 1 0,24
4 2002 1 0,24
5 2003 1 0,24
6 2004 4 0,96

ev
7 2005 4 0,96
8 2006 4 0,96
9 2007 3 0,72
10 2008 5 1,20
11 2009 5 1,20

r
12 2010 8 1,92
13 2011 11 2,64
14
15
16
17
2012
2013
2014
2015
er 9
6
14
16
2,16
1,44
3,36
3,84
pe
18 2016 27 6,47
19 2017 15 3,60
20 2018 44 10,55
21 2019 30 7,19
22 2020 71 17,03
ot

23 2021 69 16,55
24 2022 57 13,67
tn

Total 417 100

Table 1 Distribution of the Number of Multimodal Learning Research in the Google

Scholar Database Per Year
rin

According to Table 1, the Google Scholar database had 417 papers and publications of research
findings on multimodal learning from 1994 to 2022 in a variety of scientific journals. Figure 1
depicts the distribution of research in the 417 journals with the largest volume of publications in
2020. With 71 articles representing 17.03% of the total distribution of articles, the year 2020 had
ep

the most publications. Even though publishing fell between 2020 and 2022, the overall number of
papers published is still rising.
Pr

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4298216
 6
The Journal of …, Taylor & Francis, 7

ed
7
Journal of Applied …, jastt.org, 79
6
Journal of neuro …, Springer, 27
11
Journal of Neural Engineering, iopscience.iop.org,5
21
Journal for STEM Education Research, Springer,6
7
Elt Journal, academic.oup.com, 11
4
International Journal of …, Elsevier, 35
42

iew
IEEE Journal on …, ieeexplore.ieee.org, 9
15
Journal of Educational …, journals.sagepub.com,5
4
… Assisted Learning, Wiley Online Library,6
70
ISLLAC : Journal of Intensive Studies on Language,...6
6
… Assisted Learning, Wiley Online Library, 9
6
JALT CALL Journal, oro.open.ac.uk,10
146
British Journal of …, Wiley Online Library, 17
8
Journal of Learning …, ERIC, 7
8
Journal of Learning Analytics, learning-analytics. 17
12
… Assisted Learning, Wiley Online Library, 27

ev
23
British... 21
34
Applied Linguistics Journal
77
23 51
Journal of Learning...
0 50 100 150 200

r
Most Relevance Source (Article per Journal)

er
Figure 1 Distribution of multimodal learning research documents from various sources
Based on Figure 1, it is known that, with 146 articles, the Journal of Medical Systems has published
the most research on multimodal learning. This sum symbolizes the publication of multimodal
pe
learning research findings in a transdisciplinary context. This demonstrates that past approaches
to multimodal learning research are utilized to determine medical activities using data from various
modality types, including language, pictures, gestures, and sounds.
The Average Citation
ot

200 1
190
180 0.9
170
160 0.8
tn

150 146
140 0.7
130
120 0.6
110
100 0.5
90
rin

80 79 0.4
70 70
60 60 0.3
50
40 42 0.2
30 32
25
ep

20 20 0.1
10 7 3 10 3 4 2 6 7 4 12 4
0 0 0 1 1 0
1994
1996
2001
2002
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
Pr

Figure 2 Average Article Quotation

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4298216
 Source impact. The number of citations that journals and articles receive as well as the H-Index

ed
of scientific journals is used to determine the influence of published source media. The average
number of articles cited annually and the H-index of scholarly publications are both covered in
this section. Figure 2 displays the annual average of article citations for all published scientific
works over a period of years.

iew
According to Figure 2, the greatest citation average was in 2020, with 146 citations in each article,
while the lowest average, 0.04 citations, was in 1994. This is acceptable considering that there
weren't many publications published during that period, making it difficult to read and unlikely to
be cited. Figure 3 shows the impact on the media of 417 journals with the highest H-index in the
period from 1994 to 2022.

ev
6
The Journal of …, Taylor & Francis, 7
7
Journal of Applied …, jastt.org, 79
6
Journal of neuro …, Springer, 27
11
Journal of Neural Engineering, iopscience.iop.org,5
21
Journal for STEM Education Research, Springer,6

r
7
Elt Journal, academic.oup.com, 11
4
International Journal of …, Elsevier, 35
42
IEEE Journal on …, ieeexplore.ieee.org, 9
15
Journal of Educational …, journals.sagepub.com,5
… Assisted Learning, Wiley Online Library,6
ISLLAC : Journal of Intensive Studies on Language,...6
… Assisted Learning, Wiley Online Library, 9
JALT CALL Journal, oro.open.ac.uk,10
4

British Journal of …, Wiley Online Library, 17

6
6
er
8
70

146
pe
Journal of Learning …, ERIC, 7
8
Journal of Learning Analytics, learning-analytics. 17
12
… Assisted Learning, Wiley Online Library, 27
23
British... 21
34
77
Applied Linguistics Journal
23 51
Journal of Learning...
0 50 100 150 200
ot

Source Impact (H-Index)

tn

Figure 3 Average H-index of Article citations

According to Figure 3, the Journal of Medical Systems, which has an H-index of 146, has the most
influence, followed by the Journal of Applied Science and Technology Trends, which has an H-
index of 79. a multimodal user interfaces journal with a 70 H-Index. Because access to other
sources, including conferences and repositories, cannot be determined by their H-index, the impact
rin

of scarce resources is only considered based on journal articles.

Most keywords for research. To determine the trends of factors and search terms that university
management researchers have studied using VosViewer software. To find out the trend of variables
and keywords that often appear and are researched by university management researchers using
ep

VosViewer software.

No Keywords Cluster Frequent Percentage

1 Language 10 18 17
Pr

2 Multimodal Deep Learning 6 17 18

3 Multimodal Approach 10 20 17

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4298216
 4 Multimodal Analysis 13 31 27

ed
5 Student 15 51 49
6 Article 14 20 14
7 Information 14 31 22
8 Machine Learning 8 26 23
9 Algorithm 13 31 22

iew
10 Data 20 79 58
11 Video 7 10 9

Table 2 Most keywords by Multimodal Learning Researchers

The most popular search terms used by researchers studying multimodal learning are presented

ev
with the results in table 2. According to table 2, research with the term Data is the most popular.
The student was mentioned 51 times, and the phrase Data was used 79 times. Additionally, the
words algorithm, information, and multimodal analysis are all used 31 times each. Meanwhile, the

r
keyword "machine learning" is used 26 times. The terminology which has developed in
multimodal learning data analysis, Students, Algorithms, Communication, Multimodal Analysis,
and Machine Learning—indicates the existence of transdisciplinary studies that combine
er
multimodality research. A treemap of multimodal learning research documents is also displayed
in Figure 4.
pe
Video
Language
Multimodal
Analisys Multimodal
Data Deep
ot

Learning
Information
Machine Multimodal
Learning Approach
tn

Article
rin

Student Algorithm
ep

Figure 4 Treemap of the document Multimodal Learning

Pr

A TreeMap based on the frequency of usage of phrases that are often used in multimodal learning
research can be seen in Figure 4. The map indicates that vocabulary related to data and students

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4298216
 predominates in multimodal learning research. This demonstrates that a multimodal learning study

ed
that includes the keywords Data and Student is popular research in Google Scholar and is
frequently utilized by academics and industry professionals.
Despite being the most widely used model, not all multimodal learning research hypotheses
are appropriate for and consistent with the characteristics of Indonesian learners (Ebrahimi et al.,

iew
2016; Firmansyah, 2021b; Firmansyah & Suwadi, 2021; Firmansyah, 2018; Firmansyah et al.,
2020; Ryu & Boggs, 2016). It must thus be modified or recreated to reflect the cultural traits of
Indonesian culture (Firmansyah & Suwadi, 2021; Ryu & Boggs, 2016).

r ev
er
pe
ot

Figure 5 Word map of a multimodal learning research companion

tn

Additionally, Figure 5 provides a depiction of the link between terms in studies on multimodal
learning. According to Figure 5, multimodal learning research is the one that is most frequently
seen in both individual and group forms of study. With a rational approach leading to the keywords
Algorithm, Information, Multimodal deep learning, Machine Learning, and multimodal approach
on the left side of the Map and a rational approach leading to the keywords Data and Student on
rin

the right side of the Map, the study of Data and Students is quite extensive, particularly in the use
of terminology and scope of the study. Transdisciplinary studies, an emerging phrase used in
conjunction with multimodality, aims to broaden the definition of studies. Additionally, this is an
attempt to highlight the fact that multimodal learning research with transdisciplinary studies and
ep

its variations is still a well-known multimodality research paradigm.

Pr

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4298216
 Number Of Document

ed
0 5120 3 322523271217 8 7 8 1714610 6 9 6 6 70 6 4 5 15 9 4235 4 11 7 6 21 5 1127 6 79 7 7 6

D Spikol, E Ruffaldi, G Dabisias, ...

K Mangaroska, K Sharma, D Gaševic, ...

M Tang, P Kumar, H Chen, A Shrivastava

A Andrade, G Delandshere, ...

AJ Magana, MI Serrano, ...

M Dyrba, F Barkhof, A Fellgiebel, M Filippi, ...

B Schneider, J Reilly, I Radu

H Li, J Huang, J Huang, S Chai, L Zhao, ...

M Harvey, A Deuel, R Marlatt

P Blikstein, M Worsley
L Crescenzi-Lanna
M Farías, K Obilinovic, R Orrego
D Di Mitri, J Schneider, M Specht, ...
A Emerson, EB Cloude, R Azevedo, ...
NJ Yelland

B Schneider, P Blikstein
X Ochoa, M Worsley
M Worsley, R Martinez-Maldonado, C D'Angelo

C Larmuseau, J Cornelis, L Lancieri, ...

KC Santosh
R Hampel, M Hauck

M Beardsley, J Martinez Moreno, ...

MB Firmansyah
SE Kahou, X Bouthillier, P Lamblin, C Gulcehre, ...
MJ Junokas, R Lindgren, J Kang, ...
M Vujovic, D Hernández-Leo, ...
A Ioannou, C Vasiliou, ...

Y Yang, F Gao, C Xing, J An, ...

Z Huang, X Xu, J Ni, H Zhu, ...
MN Giannakos, K Sharma, IO Pappas, ...
PT Yu, YS Lai, HH Tsai, YH Chang
L Jiang
A Perveen

K Sharma, M Giannakos
P Ortega, AA Faisal
HT Hung, YCJ Chiu, HC Yeh
H Zhou, K Chang, HX Bai, B Xiao, C Su, WL Bi, ...

SMSA Abdullah, SYA Ameen, MAM Sadeeq, ...

M Gellevij, H Van Der Meij, T De Jong, ...

SG Sadeghi, LB Minor, KE Cullen
iew
r ev
Years

er
Figure 6 The Most Productive Multimodal Learning Researchers

The productivity of the researcher network. The productivity and network between authors
pe
created by each researcher must be mapped to determine the network between writers in producing
an acceptable and thorough multimodal learning research that is unique. The top 20 multimodal
learning researchers are depicted in Figure 6.
Figure 6 demonstrates that there is, albeit not explicitly, a network of multimodal learning research.
The output of researchers each year demonstrates this. 20 researchers from various nations will
likely collaborate on research. Researchers who produce research are also mentioned in Figure 6.
ot

It affects other interdisciplinary that take part in developing multimodal learning research and
provides a good indicator in pertinent multimodal learning research.
tn

Discussion
Multimodal learning research trends. The results of the bibliometric analysis in this study showed
that the keyword Data was used 79 times and the keyword Student was used 51 times. Furthermore,
the keyword Algorithm is used 31 times Information is used 31 times and the keyword Multimodal
rin

Analysis is used 31 times. Meanwhile, the Machine Learning keyword appears 26 times. The
emergence of the terminology Data, Student, Algorithm, Information, Multimodal Analysis, and
Machine Learning in multimodal learning research indicates the existence of transdisciplinary
studies with multimodality research. This means that multimodal learning research with the
ep

keywords Data and Student is further developed in the topic of multimodality in multimodal
learning research.
Multimodal learning research began to emerge in the period 1994 to 2022. Interestingly,
multimodal learning research with a rational approach leads to the keywords Data and Student.
Pr

This shows that multimodal learning research that leads to the keywords Data and Student is
popular research in Google Scholar and is most often used by researchers and practitioners of

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4298216
 multimodal learning. Despite being the most popular model, not all conceptions in multimodal

ed
learning research are suitable and follow the characteristics of Indonesian learners (Ebrahimi et
al., 2016; Firmansyah & Suwadi, 2021). Therefore, it needs to be adjusted or re-created with the
cultural characteristics of Indonesian society (Firmansyah, 2018, 2019)s.
The findings demonstrate how transdisciplinary studies that are supportive of multimodal

iew
learning research are integrated and developed by demands. The advancement of multimodal
learning research results in the improvement of multimodal learning media, the development of
student multimodal competencies, application in the fields of industry, medicine, and
transportation, as well as the incorporation of empirical studies into multimodality implementation
(Nagaraju et al., 2012; Nouri, 2019; Pichler et al., 2010; Pinar, 2019). Empirical studies on issues
about student multimodal competence development, bolstering multimodal learning medium,

ev
application in the industrial, medical, and transportation areas, and research to be done to address
the issues faced, are necessary (Chi et al., 2009; Ebrahimi et al., 2016).
The trend in multimodal learning research is also increasing year after year in response to
issues with students’ multimodal competency, the development of multimodal learning media, and

r
its use in the domains of industry, medicine, and transportation. Students who are stated to possess
multimodal competence should be sensitive to possible meanings and modalities to comprehend

er
how texts or conversations are produced and how they are presented. Students who receive
instruction in multimodal competency become capable authors of multimodal discourse as well as
competent consumers of multimodal literature. Technology and information have emphasized the
pe
need of understanding semiotic resources like language and resources like language, audio, and
pictures using a multimodal approach (modalities).
Once experienced by a human, movement, sound, and all inanimate objects have
interactional significance (Coccetta, 2018; Freyn & Ed, 2017; Lim-Fei, 2013). As a multimodal
discourse analysis competency for students, sensitivity in the use of multimodal resources, and the
potential to shape learning experiences in the classroom using a variety of learning media, the
ot

perspective of multimodal competence in learning can provide an understanding that can lead to
more effective learning in the classroom through multimodal media and multisemiotic
experiences. as a vehicle (Coccetta, 2018; Freyn & Ed, 2017; Lim-Fei, 2013). For multimodal
tn

learning research to advance and become more inclusive of practitioners and technicians who are
interested in it, there has to be a shared understanding.
The growth of multimodal learning research does not necessarily correspond to the needs
of the community for the solutions being provided. It requires the development of multimodal
rin

competency in students, the improvement of multimodal learning media, and application in the
domains of business, medicine, and transportation. Integration of many modes of transportation,
which improves multimodal learning resources and is applied to the domains of industry, medicine,
and transportation (Coccetta, 2018; Firmansyah et al., 2020; Freyn & Ed, 2017; Lim-Fei, 2013)
ep

needs-based reality that takes into consideration the social, cultural, and personal context of a given
culture. The construction of multimodal learning as a component of multimodality is based on the
role and aim of the modality as a source of social semiotics.
It is necessary to enhance research as a tool for the scientific growth of multimodal
Pr

learning. To ensure that multimodal learning research may be accomplished as anticipated, it is

important to start with research integrity, scientific advancement, and expanding research networks

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4298216
 across universities. To offer a wider range of alternative viewpoints, multimodal learning research

ed
should focus on increasing student multimodal competence, enhancing multimodal learning
media, and implementation in the industrial, medical, and transportation fields (starting from the
locality, philosophy, to comprehensive theoretical views). As part of multimodality, the notion of
multimodal learning may also be modified. This is especially true if it relates to the development

iew
of student multimodal competence, the strengthening of multimodal learning media, and its
application in the sectors of industry, medicine, and transportation. By incorporating researchers
from diverse universities, collaboration is anticipated to blend and match geographical needs to
produce a full idea and innovation of research that is thought to be theoretically valid and accurate.

Conclusion

ev
This bibliometric analysis discovers and explains that the trends and implications of multimodal
learning research lead to the improvement of multimodal learning media, implementation in the
fields of industry, medicine, and transportation in a transdisciplinary manner, and development of
students' multimodal competence. This is demonstrated by looking at the data and student

r
keywords that are dispersed and provided in each of the 417 publications, demonstrating that
multimodality, which promotes multimodal learning, continues to dominate academics' thinking.

er
From the standpoint of the researcher, it is well known that university professors and students
continue to dominate. In the meanwhile, the domains of industry, medicine, and transportation still
need many multimodal practitioners and technicians. KC Santosh is the most productive researcher
pe
in multimodal learning research with 146 citations. Meanwhile, Abdullah, Ameen, and Sadeeq are
researchers with a total of 79 citations. The number of citations for multimodal learning research
is a total of 7138 citations. This study demonstrates that there are few networks of collaboration
between institutions that allow for the discovery of findings in transdisciplinary studies that use
modalities (language, images, music, sound, and movement) while working across fields,
specifically industry, medicine, and transportation. unique across institutions and nations that
ot

enhance the fusion of transdisciplinary knowledge modalities.

The dearth of multimodality-based research that concentrates on multimodal learning in
universities demonstrates that the trend of multimodal learning research is influenced by functions
tn

and strategies in the implementation of modalities (language, images, music, sound, movement),
which do not always have the same characteristics in multimodal learning in universities. These
fields include industry, medicine, and transportation. Researchers who use functions and strategies
in the application of the industrial, medical, and transportation fields as a scientific basis in
rin

developing modalities (language, images, music, sound, and movement) as a medium for obtaining
data continue to dominate the majority of multimodal learning research. The researcher suggests
using axiological principles when implementing the industrial, medical, and transportation areas
in universities for cross-country research partnerships, multimodal learning, and scientific
ep

integration. So that universities in need can feel and use the axiological elements, and so that
transdisciplinary studies of multimodal learning are more sufficient and thorough.

Reference
Pr

Alves, J., Mendes, J., B, L. O. B., C, A. Y. O., & D, M. C. G. (2022). Article history :
Keywords : INTERNATIONAL JOURNAL OF PROFESSIONAL BUSINESS REVIEW,

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4298216
 7(2022), 1–18. https://openaccessojs.com/JBReview/article/view/312

ed
Asih, A., Tantri, S., & Ganesha, U. P. (2017). NILAI-NILAI PENDIDIKAN KARAKTER
DALAM NOVEL TANTRI ( PEREMPUAN YANG BERCERITA ) KARYA COK DI
SEKOLAH DASAR. Proceedings Education and Language International Conference, 1(1),
57–68. http://jurnal.unissula.ac.id/index.php/ELIC/article/view/1211/920
Baharudin, E. (2010). Kearifan Lokal, Pengetahuan Lokal dan Degradasi Lingkungan. Forum

iew
Ilmiah, 7(1), 8–21. https://ejurnal.esaunggul.ac.id/index.php/Formil/article/view/1317
Bayoudh, K., Knani, R., Hamdaoui, F., & Mtibaa, A. (2022). A survey on deep multimodal
learning for computer vision: advances, trends, applications, and datasets. Visual Computer,
38(8). https://doi.org/10.1007/s00371-021-02166-7
Berry, J. W., Poortinga, Y. H., & Pandey, J. (1997). Handbook of Cultural Psychology Volume
1: Theory and Method. In Handbook of cross-cultural psychology Volume 1: Theory and

ev
Method (pp. 257–301). http://books.google.com/books?hl=en&lr=&id=i-
ZEIgFfXGgC&oi=fnd&pg=PR1&dq=Handbook+of+Cross-
Cultural+Psychology&ots=oKSSCgfFux&sig=FTzYnV7o-UXVM0B-
XgmzR4BN1gs%5Cnhttp://books.google.com/books?hl=en&lr=&id=i-

r
ZEIgFfXGgC&oi=fnd&pg=PR1&dq=Handbook+of+cultural+
Calantonea, R. J., Cavusgila, S. T., & Zhaob, Y. (2002). Machine Translated by Google Machine

er
Translated by Google. Articulo De Investigación Científica, 31(4), 515–524.
Cheng, Y., & Liu, W. (2014). A Multimodal Discourse Analysis of the Relationship between Pi
and Richard the Tiger in the Movie Life of Pi. International Journal of Language and
Literature, 2(4), 191–219. https://doi.org/10.15640/ijll.v2n4a11
pe
Chi, S. N., Zimmerman, M. A., Yao, X., Cohen, K. J., Burger, P., Biegel, J. A., Rorke-adams, L.
B., Fisher, M. J., Janss, A., Mazewski, C., Goldman, S., Manley, P. E., Bowers, D. C.,
Bendel, A., Rubin, J., Turner, C. D., Marcus, K. J., Goumnerova, L., Ullrich, N. J., &
Kieran, M. W. (2009). Journal of clinical oncology intensive multimodality treatment for
children with newly diagnosed CNS atypical teratoid rhabdoid tumour. Journal of Clinical
Oncology, 27(3), 385–389. https://doi.org/10.1200/JCO.2008.18.7724
ot

Coccetta, F. (2018). Developing university students ’ multimodal communicative competence :

Field research into multimodal text studies in English. The system, 1–9.
https://doi.org/10.1016/j.system.2018.01.004
tn

Ebrahimi, S., Xavier, K., Pascal, B., Caglar, L., Michalski, V., Konda, K., Froumenty, P.,
Nicolas, D., Chandias, B. R., Mehdi, F., Aaron, D. W., Pascal, C., & Roland, V. (2016).
EmoNets : Multimodal deep learning approaches for emotion recognition in video. Journal
on Multimodal User Interfaces, 10(1), 99–111.
rin

https://link.springer.com/article/10.1007/s12193-015-0195-2
Firmansyah, M Bayu, & Rokhmawan, T. (2018). Pengembangan Kompetensi Komunikasi
Multimodal Mahasiswa STKIP PGRI Pasuruan. Jurnal Ilmiah Edukasi Dan Sosial,
9(September), 124–133. http://www.jiesjournal.com/index.php/jies/issue/view/9
Firmansyah, Mochamad Bayu. (2021a). MULTIMODALITAS MODEL PEMBELAJARAN
ep

PSIKOLOGI SASTRA (T. D. P. Khatulistiwa (ed.); 2021st ed.). Delta Pijar Khatulistiwa.
https://scholar.google.com/scholar?cluster=12823019664925397591&hl=en&oi=scholarr
Firmansyah, Mochamad Bayu. (2021b). The Effectiveness Of Multimodal Approaches In
Learning. Journal of Education And Technology, 4(3), 469–479.
Pr

https://ejournal.ijshs.org/index.php/edu/article/view/194/154
Firmansyah, Mochamad Bayu, & Suwadi, S. (2021). Student’s multimodal competency in digital

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4298216
 learning in universities. Erudio Journal of Educational, 8(December), 120–136.

ed
https://www.erudio.ub.ac.id/index.php/erudio/article/view/383%0Ahttps://www.erudio.ub.a
c.id/index.php/erudio/article/download/383/286
Firmansyah, Mochammad Bayu. (2018). Multimodal conception in learning. ISLLAC : Journal
of Intensive Studies on Language, Literature, Art, and Culture, 2(1), 40–44.
http://journal2.um.ac.id/index.php/jisllac/article/view/4140

iew
Firmansyah, Mochammad Bayu. (2019). Literasi Multimodal Bermuatan Kearifan Lokal.
10(July), 60–68.
Firmansyah, Mochammad Bayu, Siswanto, W., & Priyatni, E. T. (2020). Multimodal
Smartphone : Millennial Student Learning Style. Test Engineering & Management,
82(January-February 2020), 9535–9545.
https://scholar.google.co.id/scholar?hl=id&q=Multimodal+Smartphone:+Millennial+Studen

ev
t+Learning+Style&as_sdt=0
Freyn, A. L., & Ed, D. (2017). Effects of a Multimodal Approach on ESL / EFL University
Students ’ Attitudes towards Poetry. 8(8), 80–83. ISSN 2222-1735 (Paper) ISSN 2222-288X
(Online)

r
Hermawan, B. (2013). Multimodality: Menafsir Verbal. Bahasa & Sastra, 13 No. 1 A.
Hung, H., Chiu, Y. J., & Yeh, H. (2013). Multimodal assessment of and for learning: A theory-

er
driven design rubric. British Journal of Educational Technology, 44(3), 400–409.
https://doi.org/10.1111/j.1467-8535.2012.01337.x
Jewitt, C. (2008). Multimodality and Literacy in School Classrooms. Review of Research in
Education, 32(1), 241–267. https://doi.org/10.3102/0091732X07310586
pe
Karsidi, R. (2017). Budaya lokal dalam liberalisasi pendidikan. 1(2), 19–34.
https://doi.org/10.26740/jsm.v1n2.p19-34
Kurniasih, N., Wanabuliandari, S., & Ristiyani, R. (2020). DigitalCommons @ the University of
Nebraska - Lincoln Science Mapping for Popular Topics in Cyberbullying Prevention
Articles Science Mapping for Popular Topics in Cyberbullying Prevention Articles. Library
Philosophy and Practice. https://digitalcommons.unl.edu/libphilprac/4101/
ot

Lee, H. (2014). Using an arts-integrated multimodal approach to promote English learning: A

case study of two Taiwanese junior college students HSIAO-CHIEN LEE. English
Teaching: Practice and Critique, 13(2), 55–75.
tn

Lim-Fei, V. (2013). Multimodal Literacy. Practice, 61, 52–57.

https://www.researchgate.net/publication/306080495
Lim, V. F. (2011). Multimodal SFG analysis of pedagogic discourse. In PhD thesis.
Mondada, L. (2016). Challenges of multimodality : Language and the body in social interaction
rin

1. Journal of Sociolinguistics, 20(3), 336–366.

https://onlinelibrary.wiley.com/doi/abs/10.1111/josl.1_12177
Nagaraju, M., Kumar, B. S., S, P., & Narayana, I. L. (2012). Multimodality video representation
multimodality video representation. November 2020.
Nesi, A., Rahardi, R. K., & Pranowo. (2010). NILAI-NILAI KEARIFAN LOKAL KAJIAN
ep

EKOLINGUISTIK. Jurnal Pendidikan Dan Kebudayaan, 11(3), 71–90.

http://jurnal.unikastpaulus.ac.id/index.php/jpkm/article/view/138
Nina Nørgaard. (2009). The Semiotics of Typography in Literary Texts\nA Multimodal Approach.
1979. http://www.sheltonography.com/resources/Articles/SemioticsofTypography.pdf
Pr

Nouri, J. (2019). Students Multimodal Literacy and Design of Learning During Self-Studies in
Higher Education. Technology, Knowledge and Learning, 24(4), 683–698.

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4298216
 https://doi.org/10.1007/s10758-018-9360-5

ed
Philosophy, L., Kurniasih, N., Kurniasih, N., & Wanabuliandari, S. (2020). DigitalCommons @
the University of Nebraska - Lincoln Bibliometrics Analysis in Articles of Verbal Bullying
in Schools Bibliometrics Analysis in Articles of Verbal Bullying in Schools. Library
Philosophy and Practice. https://digitalcommons.unl.edu/libphilprac/4087/
Pichler, B. J., Kolb, A., & Na, T. (2010). PET / MRI : Paving the way for the next generation of

iew
clinical multimodality imaging applications. The Journal of Nuclear Medicine, 51(3), 333–
336. https://doi.org/10.2967/jnumed.109.061853
Pinar, A. G. (2019). Multimodality – The driving force for engineering undergraduates in a
course of technical English. Universal Journal of Educational Research, 7(7), 1580–1589.
https://doi.org/10.13189/ujer.2019.070712
Rusydiana, A. S. (2019). Bibliometric Analysis of Scopus-Indexed Waqf Studies. Ekonomi Islam

ev
Indonesia, 1(1). http://journals.smartinsight.id/index.php/EII/article/view/1/1
Ryu, J., & Boggs, G. (2016). Teachers’ Perceptions about Teaching Multimodal Composition:
The Case Study of Korean English Teachers at Secondary Schools. English Language
Teaching, 9(6), 52. https://doi.org/10.5539/elt.v9n6p52

r
Salimova, T., Guskova, N., Krakovskaya, I., & Sirota, E. (2019). From industry 4.0 to Society
5.0: Challenges for the sustainable competitiveness of Russian industry. IOP Conference

899X/497/1/012090 er
Series: Materials Science and Engineering, 497(1), 0–7. https://doi.org/10.1088/1757-

Wibisono, H. K., Trianita, L. N., & Widagdo, S. (2013). Filsafat, Etika, dan Kearifan Lokal
untuk Konstruksi Moral Kebangsaan. In Filsafat, Etika, dan Kearifan Lokal untuk
pe
Konstruksi Moral Kebangsaan.
Zamroni, E., Hanurawan, F., Hambali, I., & Hidayah, N. (2022). Trends and Research
Implications of Guidance and Counseling Services in Indonesia From 2010 to 2020 : A
Bibliometric Analysis. Journals.Sagepub.Com/Home/Sgo, April, 1–16.
https://doi.org/10.1177/21582440221091261
ot
tn
rin
ep
Pr

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4298216