Originally published at Science 2.

0 for TEL workshop, ECTEL 2009 Conference, Nizza

Visualizing Research Patterns in the Field of eLearning
Muhammad Salman Khan1, Martin Ebner2, Behnam Taraghi3 Graz University of Technology Graz, Austria skhan@iicm.edu1, martin.ebner@tugraz.at2, b.taraghi@tugraz.at3

Abstract. In this paper we have performed a content analysis of Ed-Media conferences publications from 2003 to 2008 using our internally built visualization tool to realize various research patterns in the field of e-learning. We identified 3668 different research terms from 4607 articles’ titles. The visualization tool revealed the trends of contributions in the field over the years, evolution or decline of research terms, hot research terms, key researchers, leading institutions and nations across the world, evolution of countries, continents and institutions in comparison to each other. The results in this study will allow novice and experienced educators, researchers in the field of elearning to understand what kind of different research terms exist and to identify different research patterns over the last six years in this field.

1 Introduction
In any particular academic discipline the scientific research publications represents the knowledge structure of that discipline. This knowledge structure reflects the research areas, research trends, networks of papers and authors, key researchers and institutions, collaboration of different institutions and regions. Much can be learned about a field of study by analyzing the research manuscripts in a journal or conferences of a discipline [1]. The type of analysis that is usually carried out for this purpose is referred as content analysis. Content Analysis is basically a research tool that allows determining facts and figures from a document collection [2]. The effectiveness of information visualization tools in analyzing a knowledge domain or a field of study has been realized by many researchers. The authors in [3-4] used various interactive visualizations to analyze the hidden patterns in 8 years of InfoVis conferences such as: networks of researchers, trends of research topics, key authors and correlated research areas. In [5] the authors have used 100,000 unique computer science papers from ACM digital library to analyze the trends of research areas and collaborations of authors in the field of computer science using interactive node link diagrams. In [6], interactive co-citation networks of papers with time line have been visualized to analyze research trends and transient patterns in the field of mass-extinction. The incorporation of technology in education has been given a variety of terminologies such as e-learning, m-learning, open learning, flexible learning, web

based learning, online learning, networked learning, distance learning and technology enhanced learning. Since the inception phase of distance education many studies have been conducted to realize the role and value of new developments in technology to leverage education. Ultimately, this mode of education has facilitated to bring education to students as compared to bringing students to education. The importance of analyzing research patterns has also been realized by many researchers in distance education or e-learning. The authors in [7] have exposed hidden research trends by examining four well-known distance education journals. In [8], the authors used five e-learning journals for their analysi1s. The studies provided insightful information about overall research themes, research methods, research trends and important papers. Our work is also among these lines where we seek to uncover hidden research patterns in the field of e- learning. But in addition to finding overall trends in research themes and topics we are also considering locations (continents, countries, institutions) to determine how the world is progressing in the field of e-learning using our internally built visualization tool. In our analysis we have used articles published during last six years in the World Conference on Educational Multimedia, Hypermedia and Telecommunications (Ed-Media) conferences. It is an international conference, organized by Association for the Advancement of Computing in Education (AACE) annually since 1998. The conference aims at providing a multidisciplinary forum for the discussion, exchange of information regarding the new research, developments and applications on all topics related to elearning [9]. The visualization tool which is primarily based on animated bubble charts, pie charts and line charts will allow novice and experienced educators and researchers in the field of e-learning to understand trends of contributions in the field over the years. The research topics that have evolved or diminished over the period across the globe, hot research topics, key researchers across the globe, leading institutions and nations, evolution of countries, continents and institutions in comparison to each other.

2 Data Processing and Identification of Terms
In this study we have used articles published in Ed-Media conferences from 2003 to 2008. The complete description of extraction, augmentation and normalization of metadata about publications can be found in [10]. In this study, instead of using keywords or classification assigned to papers by Ed-Media, we are extracting dictionary of terms ourselves using an open source utility named doc2mat [11] from papers titles. We did so because clustering and classification puts a level of abstraction and do not originally portray the true picture of an individual term or keyword. However the keywords can be grouped in to more meaningful themes using advanced tools (e.g. Yahoo Term Extractor1, Term Extractor2) and techniques [12] that are similar to the coding schemes used in conventional qualitative or quantitative thematic analysis.
1 2

http://developer.yahoo.com/search/content/V1/termExtraction.html http://lcl2.di.uniroma1.it/termextractor/

The doc2mat utility extracts the terms by eliminating English stop words and using the standard porter-stemming algorithm. There were total 3668 terms that were extracted from 4607 papers’ titles.


Results and Discussion

The main interface of the visualization tool has been shown in Fig. 1. The complete description of its functionality can be found in [10]. However in current implementation the selection list for topics has been replaced with tag cloud of terms.

Fig. 1. Main Interface

The tag cloud will help the users to understand the most or least frequently used terms at first glance. The trends about locations and terms can be further explored with the help of line charts in the “Report” tab. In order to facilitate an easy analysis, we have divided the papers in three groups each spanning to 2 years i.e. 2003-2004, 2005-2006 and 2007-2008 inclusive. In the following sub-sections we will represent some interesting results with regards to four different kinds of views i.e. world view, continents view, countries view and institutions view. 3.1 World View
This view represents all publications as a single entity. From the experiments it was observed that up to year 2004 the total number of publications, authors and institutions were 1700, 3210 and 1358 respectively. It was also observed that there was a decline in them for the time period 2005-2006 (1311 paper, 2233 authors, and 693 institutions) and

again a rise for the time period 2007-2008 (1596 papers, 2743 authors and 860 institutions). The results also showed that there was a continuous inclusion of new authors and institutions in the field instead of being occupied by some groups of authors.

As is any academic discipline the research activity extensively depends on potential social interactions and scientific collaborations [13]. The users can also view top authors across the world in their field by sorting the list of authors by using the number of papers. As it can be seen in Fig. 2 that the author with the highest number of papers is “Kanji Akahori” from “Tokyo University of Japan” followed by “Ron Oliver” from “Edith Cowan University, Australia”. The user can also view the per year performance of any author. The Fig. 3 demonstrates the performance of “Kanji Akahori” over the years. It is clear from the figure that he has been publishing consistently in the field. In this way a user can find the experts of any particular field at any particular location based on the number of his/her publications. The user can also view the distribution of papers across different topics for a particular year. As it is impossible to talk about each term, in this paper we will be analyzing top 10 terms which have more than or equal to 400 publications. The Fig. 4 demonstrates the distribution of publications across top ten terms. It is clear from the figure that the top two terms are “learn” (2003-2004: 611, 2005-2006: 497, 20072008: 612) and “educ” (2003-2004: 259, 2005-2006:170, 2007-2008:257) followed by “base” (2003-2004: 218, 2005-2006: 179, 2007-2008: 180), “onlin” (20032004:192, 2005-2006: 165, 2007-2008: 154), “technolog” (2003-2004: 205, 20052006:125, 2007-2008: 150), “student” (2003-2004: 135, 2005-2006: 152, 2007-2008: 184), “develop” (2003-2004: 163, 2005-2006: 143, 2007-2008: 153), “design” (20032004:138, 2005-2006: 136, 2007-2008: 139), “system” (2003-2004: 148, 2005-2006: 119, 2007-2008: 140), “environ” (2003-2004: 143, 2005-2006: 133, 2007-2008: 124). The users also have the choice to see the trends across each topic more clearly in a line chart as shown in Fig. 5. It was observed that terms “onlin”, “base” and “environ” got continuous decline for all the time periods whereas the terms “learn”, ”educ”,” “technolog”, “system”, “develop” declined for the time period 2005-2006 and rise in 2007-2008. The term “design” remained more or less consistent for all the time periods. 3.2 Continents View
This view represents the distribution of publications across different continents. It can help the users to understand how different continents have evolved with the passage of time in comparison to each other. It has been observed that North American countries which include Barbados, Canada, Mexico, Trinidad and Tobago, and United States remain as the main source of publications in the field for all the time periods followed by Europe, Asia, Oceania, Africa and South America respectively. The Asian countries contributions are increasing over all the periods (2003-2004: 265, 2005-2006: 300, 2007-2008: 371). This analysis also confirms that distance learning is rising up in Asian countries as predicted by [14].

Fig. 2. Authors list across the world

Fig. 3. Performance of an author

The most important research topics among top ten topics in North America are “learn” and “onlin”. The terms “learn” and “develop” declined for the time period 2005-2006 and rise in 2007-2008. The terms “technolog” and “educ” declined sharply for the time period 2005-2006 and a little rise for the time period 2007-2008. The term “base” and “system” rise for the time period 2005-2006 and decline for the time period 2007-2008. The terms “onlin”, “base” and “environ” declined for all the time periods. The most prominent research terms in Europe are “learn” and “educ”. The terms “develop” rise for all the time periods. The term “base” declined for the time period 2005-2006 and got the same number of publications for the time period 2007-2008. The term “environ” remained more or less consistent for all the time periods. All other terms first declined for the time period 2005-2006 and then rose for the time period 2007-2008. The major research terms in Asia are “learn” and “base”. The terms “educ”, “design” first decline and then get rise for the time period 2007-2008. The term “onlin” first rise and then get decline. All other terms are getting consistent rise in Asia. The frequently used terms in Oceania are “learn” and “onlin”. The most prominent terms in Africa are “learn” and “develop”. The most frequently terms used in South America are “learn” and “educ”.

Fig. 4. Distribution of publications across top ten terms


educ onlin technolog







Fig. 5. Trends of top 10 terms across the world

3.3 Countries View
This view further provides insights into the publications patterns. It enables the users to understand the participation of each country in the field in comparison to each other. Which country is contributing more or less, which country is strong or passive in any particular research topic and how different research topics have evolved in each country.

Figs. 6 demonstrate that Unites States (2003-2004: 684, 2005-2006: 437, 20072008: 450) contributed most of the publications and is followed by Canada, Japan, Australia, United Kingdom and Taiwan respectively. Interestingly, Canada was behind in publication up till year 2004 but for the rest of the periods it started to evolve and succeeds other countries. Two Asian countries Japan and Taiwan are moving side by side with Australia, Canada and other top countries of Europe.

Japan: 104 papers Japan: 200 papers Australia:136 papers Australia: 220 papers Canada: 87 papers Taiwan: 188 papers Canada:227 Papers



Japan: 344 papers

Canada: 366 papers Australia: 333 papers Taiwan: 287 papers

(c) Fig. 6. (a) Publications up to 2004, (b) Publications up to 2006, (c) Publications up to 2008

3.4 Institutions View
This view demonstrates the contributions of each institution in the field. We selected United States with the most number of publications for our analysis. It was observed that the “Teachers College, Columbia University” (2003-2004: 11, 2005-2006: 11, 2007-2008: 13) is the leading institution in terms of number of publications and authors in United States. It was initially behind other leading institutions such as “University of Georgia”, “University of Texas at Austin” and “University of Hawaii at Manoa”. But it started to grow gradually and is currently one of the leading institutions in e-learning from the United States. The most prominent institutions from Japan, Australia, Canada, United Kingdom and Taiwan are “National Institute of Multimedia Education”, “University of Wollongong”, “University of Alberta”, “University of Teesside”, and “National Taiwan Normal University” respectively.

4 Conclusion & Future Work
We performed a content analysis to find the hidden research patterns in the field of elearning using Ed-Media conference papers from 2003 to 2008 with the help of our internally built visualization tool. In this paper the visualization tool has been utilized for a smaller dataset. As e-learning is a global phenomenon, in order to facilitate a comprehensive overview of this field. In future we would like to enlarge our dataset by including more publications from other e-learning conferences and journals such as ELearn, TLT, Computers and Education, Educational Technology Research and Development to name a few. This will also help us to examine the usability of this visualization tool on larger datasets.

5 References
1. Taylor, E. W.:Adult Education Quarterly From 1989 To 1999: A Content Analysis of All Submissions. AEQ, Vol. 51, No. 4, pp. 322-340 (2001) 2. Masood, M.:Trends and issues as reflected in traditional educational literature: A content analysis. Unpublished Ph.D. Thesis, Indiana University (2004) 3. Ahmed, A., Dwyer, T., Murray, C., Song, L., Wu, Y. X.:InfoVis 2004: WilmaScope Graph Visualization. In InfoVis’04, IEEE Symposium on Information Visualization (2004) 4. Ke, W., Börner, K., Wiswanath, L.:Major Information Visualization Authors, Papers and Topics in the ACM Library. In InfoVis’04, IEEE Symposium on Information Visualization (2004.) 5. Erten, C., Harding, P. J., Kobourov, S. G., Wampler, K., Yee, G.:Exploring the computing literature using temporal graph visualization. (Tech. Rep. TR0304), University of Arizona (2004) 6. Chen, C.: CiteSpace II: Detecting and Visualizing Emerging Trends and Transient Patterns in Scientific Literature, JASIST, Vol. 57, No. 3, pp. 359-377 (2006) 7. Lee, Y., Driscoll, M. P., Nelson, D. W.: The Past, Present, and Future of Research in Distance Education: Results of a Content Analysis. The American Journal of Distance Education, Vol. 18, No. 4, pp. 225-241 (2004) 8. Shih, M., Feng, J., Tsai, C. C.:Research and trends in the field of e-learning from 2001 to 2005: A content analysis of cognitive studies in selected journals. Computers & Education. Vol. 15, No. 2, pp. 955-967 (2007) 9. Ed-Media:http://www.aace.org/conf/edmedia/ (2008) 10. Khan, M. S., Ebner, M. Maurer, H.:Trends Discovery in the Field of e-Learning with Visualization. In Ed-Media’09, World Conference on Educational Multimedia, Hypermedia and Telecommunications (2009) 11. doc2mat:http://glaros.dtc.umn.edu/gkhome/files/fs/sw/cluto/doc2mat.html (2009) 12. Bourigault, D. and Jacquemin, C.:TERM EXTRACTION + TERM CLUSTERING: An Integrated Platform for Computer-Aided Terminology (1999) 13. Kraut, R., Edigo, C., Galegher, J.:Patterns of Contact and Communication in Scientific Research Collaboration“; Proc. ACM Conference on Computer-Supported Cooperative Work, Portland, Oregon, USA, 1-12 (1988) 14. McIssac, Stock, M. & Gunawardene, C. N.: “Distance Education.” In Handbook of Research for Educational Communications and Technology: A Project of The Association for Educational Communications and Technology. pp. 403-437 (1996)