Textbook Frontiers of Cyberlearning Emerging Technologies For Teaching and Learning J Michael Spector Ebook All Chapter PDF

Frontiers of Cyberlearning Emerging
Technologies for Teaching and

Learning J. Michael Spector
Visit to download the full and correct content document:
https://textbookfull.com/product/frontiers-of-cyberlearning-emerging-technologies-for-t
eaching-and-learning-j-michael-spector/
More products digital (pdf, epub, mobi) instant
download maybe you interests ...
Competencies in Teaching Learning and Educational

Leadership in the Digital Age Papers from CELDA 2014
1st Edition J. Michael Spector
https://textbookfull.com/product/competencies-in-teaching-
learning-and-educational-leadership-in-the-digital-age-papers-
from-celda-2014-1st-edition-j-michael-spector/
Digital Technologies Sustainable Innovations for

Improving Teaching and Learning 1st Edition Demetrios
Sampson
https://textbookfull.com/product/digital-technologies-
sustainable-innovations-for-improving-teaching-and-learning-1st-
edition-demetrios-sampson/
Emerging Technologies for Connected Internet of

Vehicles and Intelligent Transportation System
Networks: Emerging Technologies for Connected and Smart
Vehicles Mohamed Elhoseny
https://textbookfull.com/product/emerging-technologies-for-
connected-internet-of-vehicles-and-intelligent-transportation-
system-networks-emerging-technologies-for-connected-and-smart-
vehicles-mohamed-elhoseny/
Teaching Learning and New Technologies in Higher

Education N. V. Varghese
https://textbookfull.com/product/teaching-learning-and-new-
technologies-in-higher-education-n-v-varghese/
Teaching and Learning at a Distance Foundations of
Distance Education 6th Edition Michael Simonson
https://textbookfull.com/product/teaching-and-learning-at-a-
distance-foundations-of-distance-education-6th-edition-michael-
simonson/
Emerging Practices in Scholarship of Learning and

Teaching in a Digital Era 1st Edition Siu Cheung Kong
https://textbookfull.com/product/emerging-practices-in-
scholarship-of-learning-and-teaching-in-a-digital-era-1st-
edition-siu-cheung-kong/
Educational Psychology for Learning and Teaching Susan

Duchesne
https://textbookfull.com/product/educational-psychology-for-
learning-and-teaching-susan-duchesne/
Portfolio Assessment for the Teaching and Learning of

Writing Ricky Lam
https://textbookfull.com/product/portfolio-assessment-for-the-
teaching-and-learning-of-writing-ricky-lam/
Emerging Technologies for Waste Valorization and

Environmental Protection Sadhan Kumar Ghosh
https://textbookfull.com/product/emerging-technologies-for-waste-
valorization-and-environmental-protection-sadhan-kumar-ghosh/
Lecture Notes in Educational Technology
J. Michael Spector
Vivekanandan Kumar · Alfred Essa
Yueh-Min Huang · Rob Koper
Richard A. W. Tortorella · Ting-Wen Chang
Yanyan Li · Zhizhen Zhang Editors
Frontiers of
Cyberlearning
Emerging Technologies for
Teaching and Learning
Series editors
Ronghuai Huang, Smart Learning Institute, Beijing Normal University, Beijing,
China
Kinshuk, College of Information, University of North Texas, Denton, TX, USA
Mohamed Jemni, University of Tunis, Tunis, Tunisia
Nian-Shing Chen, Department of Information Management, National Sun Yat-sen
University, Kaohsiung, Taiwan
J. Michael Spector, University of North Texas, Denton, TX, USA
The series Lecture Notes in Educational Technology (LNET), has established itself
as a medium for the publication of new developments in the research and practice of
educational policy, pedagogy, learning science, learning environment, learning
resources etc. in information and knowledge age, – quickly, informally, and at a
high level.
More information about this series at http://www.springer.com/series/11777

J. Michael Spector Vivekanandan Kumar
•
Alfred Essa Yueh-Min Huang

•
Rob Koper Richard A. W. Tortorella

•
Ting-Wen Chang Yanyan Li

•
Zhizhen Zhang
Editors
Frontiers of Cyberlearning
Emerging Technologies for Teaching
and Learning
123
Editors
J. Michael Spector Richard A. W. Tortorella
Department of Learning Technologies School of Computing
University of North Texas University of Eastern Finland
Denton, TX Joensuu
USA Finland
Vivekanandan Kumar Ting-Wen Chang

School of Computing and Information Smart Learning Institute
System Beijing Normal University
Athabasca University Beijing
Athabasca China
Canada
Yanyan Li
Alfred Essa School of Educational Technology
Analytics and R&D Beijing Normal University
McGraw-Hill Education Beijing
Boston, MA China
USA
Zhizhen Zhang
Yueh-Min Huang School of Educational Technology
Department of Engineering Science Beijing Normal University
National Cheng Kung University Beijing
Tainan China
Taiwan
Rob Koper
Open University in the Netherlands
Heerlen
The Netherlands
ISSN 2196-4963 ISSN 2196-4971 (electronic)

ISBN 978-981-13-0649-5 ISBN 978-981-13-0650-1 (eBook)
https://doi.org/10.1007/978-981-13-0650-1
Library of Congress Control Number: 2018943385
© Springer Nature Singapore Pte Ltd. 2018

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part
of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission
or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar
methodology now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publication does not imply, even in the absence of a specific statement, that such names are exempt from
the relevant protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authors or the editors give a warranty, express or implied, with respect to the material contained herein or
for any errors or omissions that may have been made. The publisher remains neutral with regard to
jurisdictional claims in published maps and institutional affiliations.
This Springer imprint is published by the registered company Springer Nature Singapore Pte Ltd.
The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721,
Singapore
Contents
Learning Any Time, Anywhere: Big Educational Data from Smart

Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Mark A. Riedesel and Patrick Charles
Framing Learning Analytics and Educational Data Mining
for Teaching: Critical Inferencing, Domain Knowledge,
and Pedagogy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
Owen G. McGrath
Learning Traces, Competence Assessment, and Causal Inference
for English Composition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
Clayton Clemens, Vivekanandan Kumar, David Boulanger,
Jérémie Seanosky and Kinshuk
QUESGEN: A Framework for Automatic Question Generation
Using Semantic Web and Lexical Databases . . . . . . . . . . . . . . . . . . . . . 69
Nguyen-Thinh Le, Alexej Shabas and Patrick McLaren
A Big Data Reference Architecture for Teaching Social
Media Mining . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
Jochen Wulf
Big Data in Education: Supporting Learners in Their Role
as Reflective Practitioners . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
Sabine Seufert and Christoph Meier
Towards Big Data in Education: The Case at the Open University
of the Netherlands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
Hubert Vogten and Rob Koper
Learning Analytics in Practice: Providing E-Learning
Researchers and Practitioners with Activity Data . . . . . . . . . . . . . . . . . 145
J. Minguillón, J. Conesa, M. E. Rodríguez and F. Santanach
v
vi Contents
Using Apache Spark for Modeling Student Behavior at Scale . . . . . . . . 169

Nicholas Lewkow and Jacqueline Feild
Towards a Cloud-Based Big Data Infrastructure
for Higher Education Institutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
Stefaan Ternier, Maren Scheffel and Hendrik Drachsler
Cloud Services in Collaborative Learning: Applications and
Implications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
Ding-Chau Wang and Yong-Ming Huang
Cloud Computing Environment in Big Data for Education . . . . . . . . . . 211
Dharmpal Singh
Head in the Clouds: Some of the Possible Issues with Cloud
Computing in Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
Richard A. W. Tortorella, Kinshuk and Nian-Shing Chen
Learning Any Time, Anywhere: Big
Educational Data from Smart Devices
Mark A. Riedesel and Patrick Charles
Abstract For many people, especially young people, a smartphone is a constant

companion. Mobile apps which allow individuals to use a smart device to enhance
their learning have the potential to be very useful for mastering basic educational
material. In order to evaluate and enhance the effectiveness of such applications when
deployed at large scale, an infrastructure designed specifically for the collection of
educational analytics data from such mobile apps is required. We detail here a set
of applications and their associated infrastructure which was developed to allow
students in courses using digital textbooks to enhance their knowledge of the basic
course content anywhere and anytime by using their smart device to do spaced
practice of the knowledge components of a course. The power of current smart devices
allows the entire application, including content and adaptive algorithm to be hosted
and run locally on the user’s smart device, so it functions fully even when no network
connection is available. The infrastructure for the collection and analysis of the
educational analytics data is entirely cloud-based, using AWS S3 for data collection
and storage, and the Apache Spark parallel computing framework for data analysis.
Thus, the entire system requires only laptop computers for the mobile developers
who create the applications and this is also sufficient for the learning scientists who
analyze the data. Both the data collection system and the data analysis system can
scale to handle the data from many millions of users with no modification to their
architecture. Similar architectures are now used for the Internet of Things (IOT)
but have not yet been widely used for educational applications. These applications
have currently been deployed to thousands of users’ smart devices and analytics data
is being received from these users’ smart devices from a wide range of locations
on several continents. In our highly connected world, this type of application will
become much more common. We describe here the type of infrastructure, security,
and analytic methods needed to use these apps to advance learning and learning
science.
M. A. Riedesel (B) · P. Charles

McGraw-Hill Education, 281 Summer Street, Boston, MA 02210, USA
e-mail: mark.riedesel@gmail.com
P. Charles
e-mail: patrick.charles@mheducation.com
© Springer Nature Singapore Pte Ltd. 2018 1

J. M. Spector et al. (eds.), Frontiers of Cyberlearning, Lecture Notes in Educational
Technology, https://doi.org/10.1007/978-981-13-0650-1_1
2 M. A. Riedesel and P. Charles
Keywords Spaced practice · Adaptive flashcards · Mobile learning

Messaging systems · Parallel computing · Cloud computing
1 Introduction
For many subjects, memorizing basic facts is an important first step in learning
and mastering content. In learning a foreign language, for example, basic vocabulary
must be memorized as an essential part of learning syntax and grammar. In medicine,
it is still considered essential for doctors, nurses, medical assistants, and EMTs
to have basic knowledge of anatomy and physiology, pharmacology, and medical
terminology memorized for immediate recall in critical situations. Even for subjects
which involve higher levels of cognition, retention of basic knowledge is commonly
the foundation upon which higher levels of abstraction are built (Bloom, Engelhart,
Furst, Hill, & Krathwohl, 1956).
Applications designed for long-term memorization are often conceptually based
on models for human memory which grew out of early work on how memories
decay with time but can be reinforced by repetition spaced in time (Ebbinghaus,
1885). Further research, particularly in the past 20 years, has shown that three related
methods can help optimize the memorization of material: spaced practice (Cepeda,
Vul, Rohrer, Wixted, & Pashler, 2008), active retrieval (Roediger & Karpicke, 2006;
Karpicke & Roediger, 2008), and interleaving (Brown, Roedinger, & McDaniel,
2014).
Spaced practice is a method where material one wants to learn is repeated on a
schedule designed to reinforce decaying memories. Ideally, an item (an atomic piece
of knowledge) would be repeated just before it would otherwise be forgotten. Using
very specific models for the forgetting process allows mathematical optimization
methods to be employed to produce optimal schedules for spaced practice (Pavlik
& Anderson, 2005; Pavlik & Anderson, 2008; Mozer & Lindsey, 2016; Settles &
Meeder, 2016).
Active retrieval is the principle that requiring a learner to recall material in
response to a challenge is more effective than re-reading material. This is true even
when there is some active involvement when reading, such as highlighting the most
important parts of a passage. Thus, requiring a learner to respond to a question is
significantly more effective than having the learner re-read a text containing the same
basic piece of knowledge. Questions can be in several modes such as multiple-choice
single-answer, multiple-choice multi-answer, fill-in-the-blank, or matching.
Interleaving is a method where questions on different topics or subtopics are
intermixed. It has been shown in learning arithmetic problems, for example, that
mixing up different types of problems in practice sessions is more effective than
concentrated drill on just one type (Rohrer & Taylor, 2007).
A learning program which can easily incorporate all three of these methods is
through the use of flashcards. Flashcards always require active retrieval and can also
be timed and sequenced to incorporate spaced practice and interleaving. Manual
Learning Any Time, Anywhere: Big Educational Data … 3
methods of doing this have been used at least since the 1970s (Leitner, 1972) but to
implement a system which can optimize the user experience, a computer program
is an especially effective way to accomplish this. In a computer-based flashcard
system, algorithms can be employed to create schedules for spaced practice which
incorporate the user’s record of correct and incorrect responses, the subject matter
of each question, and the timing of the appearance of each item.
Several such systems have been deployed over the past 25 years or so. These
are typically designed for learning hundreds or thousands of facts overtimes
extending over weeks, months, or years. Such applications include SuperMemo,
Anki, Duolingo, Brainscape, FireCracker, and Memorang (http://www.supermem
o.com, http://ankisrs.net, http://www.duolingo.com, http://www.brainscape.com,
http://www.firecracker.me, http://www.memorangapp.com). Most of these applica-
tions require a web browser with an active network connection. Those which have
self-contained mobile versions which can operate offline do not allow a centralized
collection of user interaction data. More academically based applications which do
allow this have not yet been widely deployed (Kam, Kumar, Jain, Mathur, & Canny,
2009; Pavlik, Kelly, & Maass, 2016).
With the smart devices currently available, it is possible to develop self-contained
mobile applications which include all of the educational content and which have the
software needed to adaptively schedule spaced practice completely independent of a
central computational service. Such applications can generate user interaction data
messages which can be sent to a central collection point. Such data is needed for
learning scientists to evaluate student performance and to further refine scheduling
algorithms. This allows users to make optimal use of their time in learning the material
with these applications. As large datasets accumulate, such data will also be very
valuable for advancing basic understanding of the learning process.
2 Mobile Practice of Course Content
The Higher Education division of McGraw-Hill Education (MHE) was interested in

a way for students in a course using one of MHEs online, interactive textbooks, called
SmartBooks (http://www.mheducation.com/highered/platforms/smartbook.html), to
be able to use their smartphones to memorize some of the declarative knowledge
presented in a title. They quickly realized that a mobile phone application which had
been developed to allow candidates in India to study for the U.S. Medical Licensing
Exam (USMLE), called StudyWise, could be adapted for this purpose.
This application was developed to utilize the existing homework questions from
medically related titles and present them as flashcards on a smart mobile device.
The application was entirely self-contained, with the content, the user interface, and
the scheduling algorithm all entirely contained on the smart device. This allowed
the application to function even when a network connection was not available, a
requirement for use in areas with spotty WiFi or cellular coverage.
It was recognized that this software could be re-targeted to provide optimized

practice of course material within the time frame of a college semester by modifying
the scheduling algorithm to optimize for practice of dozens to hundreds of items over
a few weeks or months rather than the USMLE application which was designed for
study of thousands of items over the course of 1 to 2 years. By leveraging published
research on learning and memory, a new algorithm was successfully developed which
fit this use case (Riedesel, Zimmerman, Baker, Titchener, & Cooper, 2017).
2.1 Smart Device Mobile Applications
As currently deployed, each of eight separate content titles has its own IOS and
Android app. Each app presents the homework questions in a title as an electronic
flashcard, grouped by topic. The questions, known as probes or items, come from
MHE’s existing SmartBook database of probes. There are currently about 1500
SmartBook titles on a wide range of subjects. For all of these titles combined, there
are some 2,350,000 distinct probes with almost three billion recorded answers for
them from the web-browser-based SmartBook system.
In SmartBook, each probe is associated with a knowledge component called a
Learning Objective (LO). The LOs are organized by topic, which in turn are related
to the title’s subject. In a course that uses a SmartBook title, the instructor creates an
assignment for a specified set of LOs. Students see only probes associated with the
LOs for that assignment, which they do online through a web browser.
StudyWise was designed as a mobile application which would allow students
to practice all of the LOs in a title, organized by topic. The mobile nature of the
applications allows users to learn and master material whenever they have a few
spare moments and wherever they happen to be. The algorithm is designed to allow
the learner to master each LO by repeated practice. Once an LO’s associated probes
have been answered correctly three times, it is considered learned.
The fact that it is entirely self-contained on the mobile device means that it can
be used whenever the user has a few spare minutes. The pattern of session durations
suggests shows that these applications are indeed being used primarily for a few
minutes at a time, as can be seen in Fig. 1. This is rather different from the originally
envisioned usage model, which was for dedicated 30 min sessions each day.
At present, there are separate IOS and Android apps for each of eight titles in sub-
jects including Anatomy and Physiology, Medical Terminology, Psychology, Human
Resources Management, American History, Majors Biology, Human Anatomy, and
Medical Assisting Certification Prep.
Fig. 1 A histogram of session duration times in minutes. More than one-fourth of all sessions are
just one minute or less
2.2 User Interface
The StudyWise apps are all native applications specifically written for smartphones,
with an IOS and an Android version available for each title. When a user opens an
app for the first time, they are asked if they want to register with an email address.
This information is used only to allow the synchronization of a user’s progress in the
app between two different devices, such as, for example, an iPhone and an iPad or
between an Android phone and an iPhone. There is a hamburger menu in the upper
left-hand corner of the screen which allows the user to (a) study, (b) get help, or (c)
create an account for synching between devices and to also check for updates to the
app’s content.
To start a learning session, the user goes to the “Study” page, which gives the
app’s name at the top of the page and has two modes: “Targeted” which presents a
list of Topics from which the user then chooses a topic from which questions will be
drawn or “Review” which presents an overall measure of progress through the app
and then will present questions from the set of Topics a user has previously studied
(Fig. 2).
Once a topic is selected, the algorithm selects the first LO and one of that LO’s
probes is then chosen at random. This first probe is then displayed on the next
screen. The question presented will be one of several types found in SmartBook:
multiple-choice single-answer (as shown in Fig. 3), fill-in-the-blank, multiple-choice
multi-answer, matching, matching-rank, or deconstruction (a special type for medical
terminology).
At the bottom of the page, a question about the learner’s confidence about their
knowledge of the question appears: “Do you know the answer?”, with the two possible
choices “Yes, I know it” or “I’m unsure”. The user must answer this question in
Fig. 2 The Topic selection page for the IOS anatomy and physiology for StudyWise app, with
targeted mode selected
order to move on to the next page. For fill-in-the-blank, matching, matching-rank,

and deconstruction questions, answering the confidence question submits the answer
for grading.
For multiple-choice single-answer and multiple-choice multi-answer, the next
page is the answer selection page (Fig. 4). For this case, the user can then either
scroll back to the question page and update their confidence after seeing the possible
answers (after which the answer page is again displayed) or simply select their
answer(s) from those shown.
Once the user’s answer is submitted, the answer is checked against the correct one
and an answer response page is then shown (Fig. 5). This page reiterates the user’s
answer and indicated confidence, whether or not the answer was correct, and then
gives additional background information on the question’s content. The level of this
additional information varies from question to question depending upon how much
such background the author of the SmartBook probe included when it was originally
created.
Fig. 3 The question

presentation page
The user then can select the “NEXT” button at the bottom of the page to move on
to a new LO’s probe chosen by the algorithm or the user can hit the back arrow in the
upper left-hand corner to return to the Topic selection page, ending that session. At
that point, the user can again choose a Topic or can hit their device’s “Home” button
to exit the application.
2.3 Algorithm Self-contained Within the Smart Device
A key element of effective memorization is repeating an item often enough for it to be

remembered. However, we do not want to waste the learner’s time by repeating items
already well learned. The algorithm used in StudyWise is a proprietary one that uses
a mathematical algorithm which has adjustable parameters that vary the appearance
time of the questions associated with each LO. This spacing also depends on whether
or not an LO’s previous question was answered correctly or not, if this was not the
first appearance of that LO. For an LO to be considered learned, a total of three
correct answers to that LO’s associated questions is required.
Fig. 4 The answer selection

page for multiple-choice
questions
When an LO first appears, it will be repeated frequently until the first correct
answer is given. After this, the LO is repeated somewhat less frequency, and after a
second correct answer, the repetition interval is lengthened even more. The spacing
also depends on the number of LOs included in a given topic. The desired practice
schedule was specified by subject matter experts, who wanted topics of a particular
size to fit within a specified practice schedule. A sample pattern of LO appearances
with time is shown in Fig. 6, for a Topic with 100 LOs (Riedesel et al. 2017).
This algorithm is compact in both size and computational complexity and is
easily self-contained within a learner’s smart device. The learner’s complete record
of progress through each topic within a title is also stored locally. This means that the
full adaptive experience can be presented to the learner even when not connected to
the Internet. A learner can practice for hours, days, weeks, or even months without a
network connection. This was originally motivated by the desire for medical students
in rural India to be able to use this application but it also means that it can be used
seamlessly even on an airplane flight or other locations where cellular data service
is not available.
Fig. 5 The answer response

page
3 Data Messaging System
The availability of cloud-based computing and storage allows a smart device to

return information on user interactions with StudyWise apps without the need for
a database to present the user interface or to store data messages produced by user
interactions. In this way, smartphones hosting StudyWise operate in the same way
as devices which are part of the Internet of Things (IOT). The particular architecture
used for StudyWise is based on Amazon Web Service’s “Serverless Computing”
technology (http://aws.amazon.com/serverless/), which is frequently employed for
IOT systems.
This architecture works very well for applications which are designed to stand
alone on a user’s smart device, and it is also at the heart of how Internet-connected
sensors and devices which are part of IOT return the data used in machine learning
and other predictive analytics. Using this type of architecture, as noted, there is no
database behind the application. This means that the usual methods of doing both
business and educational analytics by querying relevant data from an OLTP database
are not available. All of the data that business analysts and learning scientists use
must come from a custom-designed system of messages which are sent as learners
interact with the system.
Fig. 6 Plots of the first 30 LOs from a 100 LO deck. Five 30-min sessions were needed to complete
100 LOs. Red dots indicate incorrect answers, as computed by the model, and blue dots are for
correct answers. The start times for the sessions are separated by 24 h
When designing the messaging system, we solicited input from both the business
owners of the application and learning scientists. We wanted to ensure the availability
of the data needed for an understanding of the frequency and patterns of use needed
for business analytics and for doing the Learning Science related to spaced practice
and active retrieval. Both types of information can be used to improve the user
experience and the educational effectiveness of this suite of applications.
By design, as little personally identifiable information (PII) is ever solicited from
the users as possible and none is transmitted to the data collection system. The only
PII information StudyWise is the email address used to synchronize a user’s progress
between devices. Even this information is contained in the JSON messaging data only
as a non-reversible hash of the input email address.
3.1 Questions for Business Analytics
To determine what data might be needed for business analytics, we asked the Higher
Education group for a list of the type of reports they might want to be able to construct
from the messaging data. They provided a list which guided our design.
Here are some examples of the types of questions the business analysts wanted to
be able to address:
– How much was each application used in terms of questions answered per unit of
time (hour/day/week/month)?
– What is the pattern of usage in terms of time of day, in local time?
– How is usage correlated with the time of year, such as the start, middle, or end of
a semester?
– How many learning objectives have been answered for each topic for each title?
– What is the average user progress through each topic?
– What is the usage of IOS versus Android for each title?
One field added specifically to address some of the business questions was the
local time zone, in terms of offset from UCT.
3.2 Questions for Learning Science
In addition to data for business analysis, we also wanted to be able to measure the
efficacy of the apps and to be able to gain insight into the learning science related to
spaced practice and active retrieval. To be as comprehensive as possible, we consulted
with the MHE Data Science team and with academic learning scientists while defining
the messaging fields.
Among the types of questions we would like to be able to address in this area
include:
– Which app (i.e., which Title) is the learner using?
– Which question (including its Topic and Learning Objective) has just been
answered?
– Which answer was selected and was it the correct answer?
– How long did it take to answer the question?
– How long was spent looking at the explanation of the answer?
– What do the learning curves look like for each learning objective and or each
student?
– What confidence level was chosen and how does it correlate with the correctness
of the answer?
– How do performance and confidence vary as a given LO is repeated?
– How far through a given topic has the user progressed?
3.3 Data Fields and JSON Schema for the Messaging System
To be able to answer all of the above questions, the data fields in Table 1 were created.
There are also several fields to identify the version of each app overall and the version
of the adaptive algorithm which are not shown in this table. As the application is in
Table 1 Data messaging schema

Item Type Example Source
IOS version number String 9.0.1 Mobile OS
Device type String iPhone 6+ Mobile OS
IP address String 10.10.10.10 Mobile OS
StudyWise application String A&P App
Software version String 1.0.0 App
Algorithm version String 1.0.0 App
Topic IDs String [224562,135283,252034] App
Topic titles List string [“Countries and capitals”, App
“Present tense irregular
verbs”, “Past tense of verbs”]
StudyWise topic size Number 68 App
User ID String 315352FD-44B1-406E- App
A785-B74100A0B2A9
Session ID String 3B20792D-F46B-48B4- App
8F45-AA79AE620EA
Date/time session start String 2015-08-01T06:00:00.000Z App
Event type String One of “targeted” or App
“review”
Learning objective ID String CL323778e1-8d23-425f- App
8be4-996f20ae4933
Question identifier Number 589823749 App
Question type String Multiple-choice/MCQ App
Date/time presented String 2015-08-01T06:00:00.000Z Mobile OS
Date/time answered String 015-08-01T06:00:00.000Z Mobile OS
Answer selected String CL323778e1-8d23-425f- App
8be4:243565606:0
Success/failure Number 0 for failure, 1 for success App
Confidence List 1 or 2 [1, 2] App
User interrupted details List or string [“phone”, “other”] Mobile OS
Session progress Number 53% App
Product title String Anatomy and physiology App
Product Id Number 151514 App
Local time zone String −5 Mobile OS
Date/time session end String 2015-08-01T06:00:00.000Z App
use, each time an answer to a question is submitted, information is sent. Fields which
are directly linked to a user’s practice session come from the StudyWise “App” and
those which give information on the user’s device, time zone, and other information
on the session’s context come from the mobile devices “Mobile OS” and hardware.
Although both IOS and Android mobile devices can transmit a user’s location,
the business owners were concerned that asking the user for permission to include
their location would be seen as unnecessarily intrusive and might also raise student
privacy issues, so this information is not collected. Local time zone information is
collected for determining the time of day an app is being used.
3.4 JSON Schema
JSON was specified by MHE’s data engineering group for encoding the messages in
order to be compatible with a company-wide standard for data messaging systems
for future educational software products. IMS Caliper was considered as the basis
for this API, but at the time this API was designed (in mid-2016). Caliper lacked a
number of fields needed to satisfy both the educational and business requirements,
and MHE’s data architects approved the schema shown here.
One great advantage of using JSON is the ability to query a collection of JSON
documents using standard SQL query commands. This can be done either through
systems such as Apache Spark (http://spark.apache.org), jQuery (https://jquery.com),
or other similar tools. Even quite complex queries have worked successfully and
an experienced database analyst can fully leverage their background in the SQL
language while analyzing this type of data. In analyses done thus far, essentially all
SQL queries attempted using Apache Spark SQL have worked on this dataset.
3.5 Online and Offline Modes
The first version of StudyWise was created to allow medical students overseas to
study for the United States Medical Licensing Exam (USMLE). Such students could
possibly be in areas where network coverage was spotty, so StudyWise was designed
to be fully usable when the user’s mobile device had no cellular data connection. This
means that all of the logic and code needed for a fully adaptive experience needed to
be contained in JavaScript code on the mobile device. Similarly, all of the questions,
answers, explanations for incorrect answers, and ability to determine if an answer
was correct or incorrect are included in the mobile app.
However, it was still desired to have a record of user interaction data, even for
sessions done offline. In order to accommodate this, the JSON analytics data message
for each interaction is either (1) buffered and then sent to the collection server in
MHE’s AWS virtual private cloud (see below) if the user is connected or (2) stored
locally on the mobile device for later transmission if a network connection is not
currently available. Stored data from offline sessions is later sent to the collection
server when a user is running the StudyWise app and has a network connection. At
present, this is the only widely available mobile educational application of which we
know that has this capability.
3.6 Receiving System
As an app is in use, each time a learner answers a question and a JSON record is
generated. This record is stored locally in a buffer and when this buffer is full or
the session ends, the buffer of records is securely transmitted to a receiving service
running on an AWS instance. The receiving service validates the data, as described
below, and stores it in a flat file in a secure location in AWS S3 file storage.
The storage is a simple directory hierarchy organized by year, month, day, and
hour. Many utilities exist to read JSON data stored in this way, including in Apache
Spark. As noted above, this type of JSON file system can also be directly queried
using SQL just as if it was a relational database.
This method of collecting and storing the data is limited in speed by the trans-
mission time of the Internet but otherwise is capable of very high data throughput
and can be set up in parallel to scale almost arbitrarily, if necessary. Total storage
available is essentially limited only by the ability to pay for it.
This means that this data system could handle potentially millions of users daily by
simply scaling out the receiving and storage systems, with no change in architecture.
Using the capabilities of AWS means that no server or storage hardware need be
purchased to set up such a system.
4 Security and Privacy
As a system which is used in education, it is very important to maintain the privacy of

each user’s data. This requires an architecture which protects the integrity and security
of the data at each step of the collection and analysis. An important first step, as noted
above, is to store no personally identifiable information in the analytics data stream.
4.1 Data Encryption
Data encryption is a critical element of security. Strong encryption of data both at

rest and in transit prevents an attacker who might gain access to a storage system
or communication channel from obtaining sensitive information. In StudyWise, this
means that all communications between the mobile application, the data collection
end point, and the processing pipeline are encrypted using HTTPS/SSL.
Data stored for analysis, and the derived datasets that comprise the output of
those analyses, are encrypted in cloud storage. Amazon S3 supports multiple mecha-
nisms including SSE-S3 (transparent server-side encryption), SSE-KMS (server-side
encryption using AWS key management), and SSE-C (server-side encryption using
customer-provided keys).
Furthermore, the pipeline runs in a virtual private cloud on infrastructure not

directly accessible from public networks.
4.2 Access Policies and Controls
User authentication controls who can access the analytics environment. Integrated
identity management can simplify user management by leveraging a service to
authenticate users. Examples include SAML2.0 and Active Directory.
In order to ensure that only authorized users can access data stored and pro-
cessed in the analytics pipeline, role-based access provides fine grain controls on
storage (who can access data sources in the processing environment), on clusters
(who/what is authorized to configure/launch/terminate/restart clusters), on process-
ing jobs (who/what can attach and run processing on clusters) and the environment
(configuration and settings).
Auditing and logging provide the ability to alert on, monitor, and review key
events in the environment.
4.3 Data Integrity
In the compute layer, within Apache Spark, the RDD (see below) provides data
immutability and fault tolerance by design. In the storage tier, AWS S3 provides not
only an extremely high level of durability (99.999999999%) but also the ability, via
the optional Content-MD5 request header, to verify the integrity of data stored there.
4.4 Certifications
An end-to-end data analytics pipeline, especially one reliant on third-party managed

or cloud services, is a system based on the integration of many components. No
matter how strong the data encryption and access policies in place, the system is
only as secure as its weakest link. Managed service and cloud providers certify their
platforms according to documented compliance standards.
The FERPA (Family Educational Rights and Privacy Act) standard governs access
to educational information and records. Other standards, many originating in the
financial and health industries, are also relevant in education.
AWS documents their compliance at (https://aws.amazon.com/compliance/). Cer-
tifications include FERPA, HIPPA, GLBA, FISMA, RFR, and PCI DSS.
Databricks, the computing environment used here for data analysis, documents
their platform security and compliance at http://go.databricks.com/. Their security
measures are based primarily on SOC2 Type-1 Certification. SOC2 Certification
encompasses five key areas: security, availability, processing integrity, confidential-
ity, and privacy.
5 Data Processing and Analysis
In order to use the data collected for either educational or business insights, we need
to set up a system to read, clean up, and analyze the JSON data produced originally
on each user’s device which has been sent to the AWS S3 collection system. It is
possible to do all of this entirely in the cloud using AWS and other services, and this
is the approach that we have taken with StudyWise.
In particular, we would like a system which has the following characteristics: (1)
straightforward to access, use, and maintain and (2) powerful and scalable enough to
handle increasing amounts of data as StudyWise is more widely deployed. In the past
10 to 15 years, several systems have been developed which have been designed to
satisfy these types of needs. All are based on horizontally scalable clusters of servers
to allow computing to be done in parallel.
Many high-performance computing systems in use by the academic and govern-
ment research communities use the Message Passing Interface (MPI) to do parallel
computing. This requires writing code in either C or Fortran. This is scalable to the
very largest systems in existence and can be used for the most complex types of paral-
lel computing but it requires a very high level of specialized knowledge and expertise
to use. MPI can be used not just for processing large amounts of data in parallel but
also for doing very large parallel theoretical calculations for such applications as
global climate models and modeling the interiors of stars.
A system designed to make parallel computing more accessible to a wider range
of users is the Apache Hadoop/MapReduce system of software developed for the
parallel processing of large amounts of data. This system is simpler than MPI but
still requires the use of the Java programming language and also requires writing
intermediate results in its processing pipeline to disk, which can make it rather slow.
To overcome the limitations of MPI and Hadoop/MapReduce, a system called
Apache Spark (http://spark.apache.org/) has been developed by the open-source com-
munity which allows all of the computation to be done in a compute node’s memory
and which also combines some of the steps in map/reduce processing, making the
writing of code to process large amounts of data in parallel considerably easier than
previous systems. Apache Spark has come into widespread use in just the last 3 to
4 years but is becoming the dominant method of processing very large amounts of
data.
Spark also is available with APIs for programming in Scala, Python, R, and
SQL, making it accessible to a much wider audience. On a practical level, a Spark
program can be many, many times faster (x100, in some cases) than an equivalent
Hadoop/MapReduce program, as long as all of a given subset of data in a parallel
processing job can be fit into the memory of one of the servers in a Spark computing
cluster.
In this section, we show how Apache/Spark can be used to process and analyze
large-scale data with examples of its use in StudyWise.
5.1 Apache Spark
Apache Spark is both an engine and API for large-scale data processing, making very
large-scale parallel computing accessible to a much wider range of users than previ-
ous parallel computing methods. Large organizations have deployed Apache Spark on
clusters consisting of thousands of nodes, processing petabyte-sized datasets which
grow on the order of terabytes per day (Tsai, 2017).
The main abstraction in Apache Spark is the “RDD”, or resilient distributed
dataset.
“Resilient” refers to redundancy of data across compute nodes, Apache Spark’s
fault tolerance and ability to continue processing when compute nodes fail.
“Distributed” refers to the fact that data is partitioned across many multiple cluster
nodes. More importantly, computations are moved to the data, rather than the tradi-
tional approach of moving data to central location(s) for processing. This eliminates
I/O bottlenecks and allows processing to scale in a linear fashion as the amount of
data grows.
“Dataset” refers to large-scale collections of data. An important attribute of these
datasets in Apache Spark is immutability. Immutability drastically simplifies the
cost and complexity of coherently processing distributed data. Spark represents pro-
cessing steps in a directed acyclic graph. This traversable graph, combined with the
immutability of data at each intermediate step in the process, allows datasets to be
easily reconstructed on failure, contributing both to the resiliency and distributed
characteristics of the system.
More recently, Apache Spark has added an additional abstraction layer on top of
the RDD called a dataframe. The dataframe makes the manipulation and processing
of the data conceptually simpler than is the case for RDDs and it is very similar to
dataframes found in Python Pandas and in R (where the concept was first developed).
Data streaming Apache Spark supports both batch-oriented and stream processing
using a single unified API. The use of streaming allows essentially up to the second
processing to be done on the analytics data as it arrives from the user’s mobile devices,
if desired.
The Apache Spark Structured Streaming API mimics the Apache Spark batch
API, but behind the scenes it utilizes the SparkSQL engine to continuously update
data as new information arrives. The same SparkSQL queries and operations that
work on batch-loaded information can also be performed on streaming dataframes.
In the concrete examples below, Spark Structured Streaming is used to perform
event-driven processing.
APIs and Analysis Tools Apache Spark’s processing API supports multiple lan-
guages, including Scala/Java, Python, and R as well as a dialect of SQL. In the
following example, data is ingested into a Spark dataframe. Here are brief exam-
ples of how to read JSON flat files into a Spark dataframe for the different Spark
languages:
Table 5.1. Scala

val eventSampleDf = spark.read.json ("pathBatch / * . json")
Table 5.2. Pyspark (Python)
eventSample Df = spark.read.json ("pathBatch / * . json")
Table 5.3. SparkR (R)
e v e n t S a m p l e D f = r e a d .df (s q l C o nt e x t , " p a t h B a t c h / . j son" , "json")
The Apache/Spark API also allows SQL-like operations to be performed on

dataframes.
A simple select/filter operation in Scala using the SparkSQL API:
Table 5.4. SQL query using Scala

va l a p D f = e v e n t s B a t c h D f .
f i l t e r ($"s e s s i o n .st udy wise Appl i c at i on" = = = " A & P " )
The same operation expressed in Spark SQL syntax:
Table 5.5. SQL query directly in Spark SQL
% s q l se l e c t f r o m a p D f w h e r e s e s s i o n .s t u d y w i s e A p p l i c a t i o n = ’ A & P ’
5.2 Processing Pipeline
Pipelines typically have at least three layers: input, processing, and output. These
are shown in Fig. 7.
In StudyWise, these layers coordinate with software components that facilitate
user authentication, data collection, transmission, and persistence.
Layers of Processing
Ingestion via messaging system: The input layer is preceded by event hooks instru-
mented in the client mobile application. These event hooks trigger calls to REST APIs
that authenticate the user and transmit events to an event collector and cloud-based
storage.
Once these events are persisted (in AWS S3, for example), the Apache Spark
datasources API can be used to ingest the events into a dataframe for processing.
The Apache Spark API supports both batch and streaming semantics via the same
interface.
Read all events in a bulk batch read:
Table 5.6. Batch Read of a JSON file
val eventsBatchDf = spark.read.sc he ma (sc he ma ). json ("path / . json")
Stream events as they arrive via streaming.
Table 5.7. Streaming Read of a JSON input stream
val eventStreamDf = spark.readStream.schema(sche ma ). json ("path / . json")
In the case of a batch read, all source events presented at the time of the read oper-
ation will be read in a bulk read operation and stored in an Apache Spark dataframe.
In the case of the streaming read, all source events present at the time of the
read operation will be initially loaded into the target dataframe. The dataframe is
unbounded, however. As new events arrive, new rows are added to the dataframe.
Data Cleaning and Transformation: Cleaning and transformation consists of
Apache Spark and SparkSQL operations on the source dataframe. As discussed,
dataframes in Spark are immutable. Every operation on the dataframe creates a new
dataframe. Operations are distributed across the worker nodes in the cluster.
Fig. 7 The processing

pipeline for analytics data for
the mobile apps
Table 5.8. Filtering out unwanted data fields

// filter out events based on criteria,
// in this case columns not used in the processing
val 2Df = eventsStreamDf.drop("legacyData")
// use SQL−like syntax to group events based on some criteria

// in this case, by application and product title
// counted and sorted
// to produce a continuously updated dataframe containing
// the most common application/title combination
val activityByProductTitlesandApplicationStreamDf =
v2Df
. groupBy("session.studywiseApplication", "session.productTitle")
. count
. sort (desc ("count"))
Intermediate Storage: Whether transforming data for subsequent analysis, or pro-

ducing a set of potentially valuable insights, it is useful to persist these results. The
Apache Spark datasources API can write the data back into S3 cloud storage in a
variety of formats including JSON or to a more optimized and compact format such
as Parquet. These operations can also be performed in batch or streaming modes.
Table 5.9. Creating a Parquet File from raw JSON data

v2Df
. writeStream
. format ("parquet")
. option ("path", "targetPath/v2DfStream . parquet")
. start
5.3 Filtering
The data stream can also be filtered in order to produce more compact datasets for
specific types of analysis and also to clean up the data. For example, during the test
phase of the initial StudyWise deployment, the apps would sometimes produce dupli-
cate records. By applying a filter to the stream, these duplicates can be eliminated.
Other minor data issues have also been filtered out in a similar way.
It is also possible to filter the data based on any of the data fields in Table 1. A
common filter used is to look at a particular title, type of question, correctness of
answer, or level of confidence. Spark SQL makes the construction and application
of such filters to dataframes straightforward.
Time Windowing: A very common way to want to filter data is by time. To facilitate
this, a daily time stamp is added to the StudyWise data stream. With this, Spark SQL
has a built-in Window function which allows the data to be grouped by user-specified
time windows, such as hourly, daily, weekly, or monthly using SQL-like command.
Operations of many types can be done within these windows, including calculating
moving averages and ranking rows, operations difficult to do with standard SQL.
We have used SparkSQL Window functions, for example, to look at daily usage
of each app and also to prepare data for calculating learning curves. While these
types of calculations can be done by spreadsheets, such as Excel, Spark can do this
on many millions or billions of records using parallel computation.
Versioning: Versioning components of the pipeline is a useful method for tracking fea-
tures and compatibility as the pipeline evolves. Versioning applies to API endpoints
and message/event schema. Increments to the major version number indicate “break-
ing” changes, while increments in the minor version indicate backward-compatible
changes.
An example of a minor version change would be the addition of a new field that
can be safely ignored. More drastic changes to an interface, such as a change in
structure or names of existing fields, require a major version increment.
Schema Changes: By including versioning information in the event envelope, the pro-
cessing pipeline can programmatically handle disparate versions of message formats
through a process of transformation and normalization.
Table 5.10. Filtering events based on a specific software or message version.

val eventSampleDf.filter ($"session.softwareVersion" === "2.1.5")
Table 5.11. Defining a function to conditionally process events based on version.

// event normalization, Scala function
def normalizeEvent (version:String, event:String): String = {
if ( version == "2.0" | | version == "2.1") {
// transformation specific to v2
}
else if (version == "1.0" | | version == "1.1" | | version == " 1.0.1 ") {
// transformation specific to v1
}
else {
// other or unrecognized versions
}
}
Another random document with
no related content on Scribd:
has mixed up some tradition of this ancient work in Nubia. At any
rate, whatever be the truth about Lake Mœris, his account proves
beyond all question that the idea of a Reservoir was familiar to the
ancient Egyptians.
The tradition of a Reservoir somewhere on the upper waters of
the Nile lingered long in Egypt. There is a curious reference to it in a
book of travel by F. Vansleb, a Dutchman who visited Egypt about
the year 1670. The fertility of the Nile flood is caused, he says, by a
fall of dew, which usually takes place on June 17, just after the
appearance of the ‘green’ water. This dew purifies the foul water, and
makes it swell by fermentation. ‘Some of the country,’ however, he
proceeds, ‘that are ignorant of the true causes of this increase,
imagine that it proceeds from a large pond in Ethiopia in the river
itself, which the Abyssins begin to open about June 12, and let the
water out by degrees, more and more till September 14, by which
time they begin to shut it again. But this is a foolish fancy of the
Copties.’
We have seen how the tradition of Lake Mœris fascinated
Mehemet Ali; but the methods of Haroun-al-Raschid were not suited
to solid engineering works, as the history of the Barrage too plainly
shows. None of his descendants, with the exception of the Khedive
Ismail, had the wit to conceive or the ability to execute such an
undertaking, and Ismail’s fantastic imagination was fully occupied in
other directions. Fortunately for Egypt, the project had to wait until
the success of the Barrage made the time ripe for its execution, and
until skilful brains and strong hands were ready to plan and carry it
out in the most efficient manner possible.
There were three problems to be faced: first, Where could such a
Reservoir be erected? second, What arrangements could be devised
to avoid the danger of large silt deposits, which would soon seriously
diminish the capacity of the Reservoir, and, if allowed to accumulate,
render it in no long time entirely useless? third, Supposing that the
difficulties of site and design could be overcome, where was the
money to be found? During the first years of the British occupation,
while Egypt was still painfully struggling upwards from the abyss of
bankruptcy into which she had been cast by the mad whirlwind of
extravagance in Ismail’s reign, it was no time for the inception of
original works on a grand scale. But in 1890 the matter became an
affair of practical politics, and was at last seriously taken in hand.
Meantime discussion had been raging as to the best locality for the
Reservoir. An American gentleman, Mr. Cope Whitehouse, took up
the case of the Wadi Rayan, a depression in the desert to the south-
west of the Fayoum. This, he maintained, was the real site of the
ancient Lake Mœris, and here the Reservoir ought to be. He had no
professional knowledge, and he was utterly wrong in his ideas; but
his vehement method of controversy kept the subject thoroughly
alive. The whole land was filled with his clamour, and every expert
was forced to give his own views in self-defence. The debate served
a useful purpose. Gradually it came to be recognised that the river-
bed itself was the proper place for storing the water by means of a
Dam. Authorities differ as to whom belongs the credit of first making
this suggestion, or, rather, of first reviving the tradition of the past;
but it seems pretty clear that Sir Samuel Baker suggested the
construction of a Dam at the first cataract at Assouan as far back as
1867.
However that may be, in 1890 the Government took the matter
up, and charged Mr. W. Willcocks with the task of examining the river
north of Wadi Halfa, reporting upon the best available site for the
Dam, and preparing a design for it. After a careful survey, his plans
were completed in 1894, and his design for a Reservoir at Assouan
was then submitted to an International Committee of Engineers,
consisting of Sir Benjamin Baker, M. Boulé, and Signor Torricelli. Mr.
Willcocks’ plans were, with some modifications, accepted by a
majority of the Commission, and to him belongs the honour of having
designed the Dam. The selection of the Assouan site solved the first
of the three difficulties. There is at this point an extensive outcrop of
granite clean across the valley of the Nile, which it was thought
would give sound rock everywhere at a very convenient level for the
foundations of the Dam. Moreover, the trough of the river above the
cataract and a long way south of it is exceptionally deep, and this
makes it possible for a greater amount of water to be stored up
behind the Dam. But the prime necessity was for a solid foundation.
Elsewhere in Egypt the bed of the Nile is composed of shifting
sands, on which it would have been impossible to build a Dam
capable of holding up so great a head of water.
Mr. Willcocks’ design solved the second difficulty, the problem of
constructing a Dam strong enough for the purpose, and yet of
avoiding the danger of filling up the Reservoir by too great
accumulations of silt. The other great Dams in the world, as, for
instance, that built by Sir Arthur Cotton on the Godavery River, in
India, are solid throughout. They are planned so that the rising flood
shall pass freely over the top of them. But the Assouan Dam is of a
type previously unknown, and its success ought to stimulate
perennial irrigation in many parts of the world where such projects
have hitherto proved failures. Its principle is that even the highest
flood shall pass, not over it, but through it. To this end it is pierced
with 180 openings, which are like tunnels in the great mass of
masonry. The openings are controlled by powerful sliding-gates
worked from above. During the months of the flood every gate will be
up, and the ‘red’ water, carrying all its heavy burden of silt, will pass
through without impediment. Later in the year, about the end of
November, when the flood has subsided and very much less matter
is carried in suspension, the sluice-gates begin to be gradually
closed, and by the end of February the Reservoir is full, without
having affected the normal discharge of the river in any appreciable
degree. From April to July the water thus stored up is let out by
degrees for employment, according to the state of the river and the
requirement of the crops. By the time the next flood begins to come
down all the stored water will have passed out, and every sluice will
be once more open to give free passage to the rising stream.
Although the Nile in December and January carries an insignificant
amount of sediment compared to that brought down in August and
September, it yet brings down a very considerable quantity, far
greater than most other rivers at any time, and quite enough to go a
long way towards silting up the bed of the Reservoir, if it was allowed
to remain. But for this the river provides its own remedy: every year
the force of the flood will act like a gigantic broom, sweeping the floor
of the Reservoir. The sluices, arranged in sets of five, are distributed
at different levels, according to the formation of the river-bed on the
upstream side, so as to facilitate this process to the utmost. During
the months of the inundation the Nile at Assouan pours down for
weeks together a volume of 10,000 tons of water per second, and
sometimes as much as 14,000 or 15,000 tons per second. The rush
of this stupendous mass is sufficient to assure us that there will be
no silting up of the Reservoir.
Two of the difficulties had been thus overcome, when, from a new
and unexpected quarter, a storm sprang up, which very nearly
brought to a standstill the rising fabric of Egyptian prosperity. The
project of the Reservoir would have raised the level of the water, and
held up the river above the Dam to a head of 100 feet; this would
have involved the temporary submersion every year of the island of
Philæ, with its famous Temple of Isis, Pharaoh’s Bed, and other
monuments. A terrific hubbub arose. Archæological and antiquarian
societies, which until then had sometimes belittled the monuments of
Philæ as belonging to an inferior period, poured in their protests.
People who had never heard of Philæ before, but who were none the
less influential for that, joined in the outcry. Diplomatists, whose one
desire was to embarrass our progress in Egypt, took up the cause of
Art with a will. These champions of humanity at large forgot the poor
fellaheen, to whom the extra water means all the difference between
misery and happiness; nothing would satisfy them but the complete
abandonment of the project. The engineers fought stoutly in the
interests of Egypt; they offered to raise the whole of the monuments
bodily, or to transport them to the neighbouring island of Bigeh; but,
though they saved the Dam, the original design was lost, and the
Dam to-day is 33 feet lower than it ought to have been. The
foundations of Philæ have been underpinned and strengthened, the
island will only be partially submerged, and the injury to Egypt can
only be faintly estimated.
Was the sacrifice worth it? The value of Philæ lies in its beauty
more than in its antiquarian interest. No one who has witnessed
night after night the glorious sunsets on the Nile, the mysterious
charm of the changing waters, the dark belt of palms reflected in the
river below and standing out in strong contrast against the sky, the
limestone cliffs of the desert clear-cut in the dry air, and flushing pink
in the radiance of the indescribable after-glow, no one who has seen
the Temples of Karnak, could hesitate to make so small a sacrifice,
in comparison, for the sake of the river-side people. Moreover, Egypt
is rich in treasures of the past, as yet undiscovered, and wanting
only money for their development, which the Reservoir would in time
supply. And how few people visit Philæ at all! Surely, even in a
country a thousand times poorer than Egypt in artistic and
archæological interests, the well-being of the living and of the unborn
should have prevailed.
If all those who joined to swell the uproar had been really
disinterested lovers of the beautiful, there would have been small
reason to complain of their insistence. Enthusiasts can hardly be
expected to listen to the voice of reason, and Philæ has charms to
soften the heart of the most savage utilitarian. Its fate is a mournful
necessity, but it is a necessity, for the question of the Reservoir had
come to be a question of existence for Egypt. Even the advantage
gained by the opposition in lowering the height of the Dam is only a
delay. On the pylon of the Temple of Isis at Philæ is carved a huge
representation of the Pharaoh of the day, one of the most
degenerate of the Ptolemies, catching his defeated enemies by the
hair of their heads with one hand, an uplifted sword in the other. The
whole is a copy of the work of his warlike ancestors, and even as a
copy it is a delusion and a sham; for he won no victories, defeated
no enemies, and, indeed, scarcely ventured outside the walls of his
harem. The apparent victory of these lovers of Philæ, to call them by
their more honourable title, was not less delusive. Philæ is doomed.
Between half drowned and wholly drowned there is not much
difference in the case of an island, certainly not a difference worth
fighting for, and the Dam will be raised to its full height, perhaps as
soon as Egypt is ready for the extra water.
The financial difficulty remained. In spite of the prosperity of
Egypt, she is, as everyone acquainted with her history is aware,
bound hand and foot by international fetters in matters financial. The
Caisse de la Dette, founded to protect Egyptian creditors against the
dangers of bad administration, has remained to be an obstacle to
any improvements that must benefit these interests. The practical
outcome of the system is that, if the Caisse be hostile—and hostile it
has often been—no public work like the Nile Reservoir can be
carried out without the imposition of extra taxation to the amount of
double the annual expenditure required.
Time passed, the need became more pressing, but the prospects
of the Reservoir seemed further off than ever. Besides the regulation
of her water-supply, Egypt had on her hands the question of the
Soudan. From every point of view the reconquest of that province
and the upper waters of the Nile was a prime necessity; no one
could tell how long the war might last, or how great the expense
might be. It seemed impossible that she could bear the cost of two
such enterprises simultaneously, and under such circumstances her
credit would not have been sufficient to raise the capital sum
required on anything like reasonable terms. Not only so, but by the
peculiar constitution of Egyptian finance it was illegal for her to raise
a loan without the consent of the Caisse, a consent which it was
impossible to obtain.
But, fortunately for Egypt, there were a few men with clearer
vision and more faith in the future, and chief among these was Lord
Cromer. The statesman who had controlled the tangled destinies of
Egypt through so many dark years, and baffled so many tortuous
intriguers, as well as more open foes, was not the man to despair in
such a situation. In 1897 the first negotiations were quietly opened
with Sir E. Cassel. Then came the vote of the majority of the Caisse
to grant £500,000 towards the Soudan Railway, and the successful
action taken in the Courts against that vote. Everyone knows how
this seeming defeat was turned to overwhelming victory by the
decision of the English Government to grant £750,000 for the railway
on certain conditions. The enemies of England in Egypt received a
staggering blow.
But the story was not yet complete. In April, 1898, Sir E. Cassel
arrived at Cairo; in one day the details of the arrangements to
finance the Dam were settled; all that night the lawyers drafted the
necessary documents; a Council of Ministers was hastily called in
the morning, and the contracts were signed. Sir E. Cassel was to
provide the necessary funds for the execution of the work,
£2,000,000; repayment by the Egyptian Government was to be
deferred altogether for five years, and then to be spread over a
period of thirty years. The first payment of about £78,000 is included
in the Budget for 1903.
Looking back now after five years of prosperity, when Egyptian
securities have actually increased in value, while Consols
themselves have so greatly declined, it is easy to see that the
statesman and the financier were justified in their faith. But in those
days it needed a clear vision and a stout heart to calculate thirty
years ahead—nay, even five—in a country so much the sport of
international politics. The Soudan Campaign was not yet ended;
behind the dervishes there loomed vague possibilities of worse
complications. The Egyptian Government made a good bargain then,
though it would doubtless make a better now. But it was then, and
not now, that the business had to be settled.
CHAPTER VIII
THE DAM AND THE NEW BARRAGES
Once the financial difficulty was settled, no time was lost in setting to
work. As soon as the flood of 1898 began to subside, Messrs. Aird
and Co., the contractors, were busy with the foundations of the Dam.
Five years was the period allowed by the contract, but a succession
of low Niles gave unusual facilities for the work, and everything was
completed before the flood of 1902, a year before the specified time.
From its vast proportions, the Dam is infinitely more impressive to
the imagination than any other of the irrigation works in Egypt. But
from an engineering point of view its construction was a plain,
straightforward business compared with the difficulties of building a
Barrage, where the river-bed offered no more solid foundation than
shifting sands. Still, there was a moment, on the first uncovering of
the river-bed, when its fate seemed to hang in the balance. The
Assouan site had been selected principally because the outcrop of
granite, there running clean across the valley, would give, it was
thought, solid foundation at a convenient level. It was found that in
some places the rock was rotten to a depth of 40 feet. It was an
anxious moment, both from an engineering and a financial point of
view. Every foot of rotten rock meant a considerable addition to the
calculated expense, besides modifying the building plan. Once more
Lord Cromer’s strong will saved the situation. On the financial side
he stood on firm ground, and he proved as good an engineer as he
had been a financier. Solid rock was reached, and the work went
steadily forward. Ten thousand men was the usual sum of those
employed, and of these 800 were Italian stone-cutters specially
brought over to deal with the tough granite of which the Dam is built.
Granite and Portland cement are the two great materials used for
welding the fetters of the Nile.
The Dam is about one mile and a quarter in length, and at its
deepest point it is 126 feet high. Sixty-five feet of water can be held
up when the reservoir is full, and it is capable of storing about
1,200,000,000 cubic metres of water—that is to say, about the same
amount of water as passes through Assouan in a single day when
the flood is at its height. The face of the wall is a slope on the
downstream side, and its width at the bottom corresponds
approximately to its height. Seven hundred and eighty thousand
cubic yards of masonry have been used. On the western side a
ladder of four locks gives passage to boat and steamer traffic at all
seasons.
All parts of Egypt are to benefit in a greater or less degree from
the extra summer supply. The original calculation assumed an
amount to be distributed of 1,065,000,000 cubic metres. It was
allotted as follows:
Cubic Metres.
South of Assiout 170,000,000
Assiout to Cairo with the Fayoum 510,000,000
Gizeh Province 85,000,000
Lower Egypt 300,000,000
This division meant that 52,000 acres could be reclaimed to

cultivation in the Fayoum, and 120,000 acres in the Delta. Further,
south of Assiout 200,000 acres could be converted to perennial
irrigation by means of pumps upon the Nile banks. In Middle Egypt
458,000 acres could be converted to perennial irrigation, and in
Gizeh Province 106,000.
A low Nile on the average comes about once in five years, but
assuming that it happened every year, these results may be
expressed in terms of money on the basis that conversion to
perennial irrigation increases the yield per acre by £2 annually, and
that reclaimed land produces a yield valued at £5 annually. This
gives an increase of value in the annual yield:
£E
South of Assiout 420,000
Middle Egypt and Fayoum 1,176,000
Gizeh Province 212,000
Lower Egypt 600,000
Total £E2,408,000
while the direct annual gain to the State Exchequer in rental and
taxation would amount to £E378,400.
This does not exhaust the full extent of the benefits of the
Reservoir on which a money value can be placed. For some time
Egypt had been living beyond her real resources in the matter of
water. Encouraged by a series of good years, the acreage of cotton
had been greatly extended. The cotton-plant is very hardy, and can
retain its vitality for a certain period on a very scanty and irregular
supply of water. Cultivators, especially in the Delta, had taken
advantage of this quality up to the very hilt, and the annual crop had
reached an amount of 6,000,000 kantars.
Taking the not very high price of 175 piastres per kantar (1 kantar
= nearly 100 pounds), the annual value reached £10,000,000. A
season like that of 1889, when the summer supply was very low, and
not the lowest on record, would mean the loss of at least one-tenth
of this amount, and even more, in spite of the most successful
working of the Barrage, and the most careful system of rotations.
Against such a loss the Reservoir is a complete insurance, and, as a
low year cannot safely be reckoned as occurring less than once in
five years, the annual value of such insurance must be set down as
£200,000. The figures give some idea of the value of the new supply.
The estimate does not err on the side of exaggeration. It was framed
in the most cautious and conservative manner possible, and, in fact,
it would be by no means rash to put the total annual value a good
deal higher.
According to the financial arrangement, the first payment towards
defraying the cost of the works was included in the budget for 1903,
but no fresh taxation for the purpose was to be imposed till 1904,
and even then the full amount of direct benefit to the Exchequer will
not be realized till 1910. Time is thus given for the full effects of the
change to be felt, and the Reservoir will be paid for out of its own
profits. The alteration of the basin lands to the new system must take
some time, and their cultivators will thus be given full opportunity to
familiarize themselves with the new methods of agriculture which
they will have to employ.
The scheme of distribution allotted nearly 50 per cent. of the
Reservoir supply to the Fayoum and the province between Assiout
and Cairo. Just south of Assiout is the head of the great Ibrahimiyah
Canal, which not only supplies these provinces, but also feeds the
Bahr Yusuf, which waters the Fayoum. It was necessary, in view of
the increased discharge, to widen the upper reaches of this canal,
and to provide it with a new regulating head. But more than this was
required to insure its receiving the proper proportion of water
whenever the sluice-gates of the Dam were opened. Accordingly, at
the same time that the foundations of the Dam at Assouan were laid,
a new Barrage was begun at Assiout just downstream of the head of
the Ibrahimiyah Canal.
In principle the Assiout Barrage is exactly the same as that at the
point of the Delta, and the difficulties of construction were also
exactly similar, for in both cases the foundations had to be laid on
the same shifting sands, and as each section of the work was
undertaken a portion of the river had to be diverted from its course
by means of temporary earthen dams. The shifting nature of the
river-bed, the almost personal malignity of the water, constantly
bursting through in countless springs, each of which had to be
separately dealt with and suppressed, called forth the highest
exercise of engineering skill. But the experience gained in the long
struggle with the imperfections of the earlier Barrage infallibly told its
tale. Every difficulty was successfully encountered, and the Assiout
Barrage was completed by the summer of 1902, and was able to
hold up the 10 feet of water required of it, without any failure, at the
first attempt.
The visible part of this Barrage, which is just over half a mile in
length, consists of a viaduct or bridge with 111 archways, each 16
feet 5 inches in width, closed by strong iron gates, working in
grooves made in the supporting piers, and raised or lowered from
above. In contrast to the Delta Barrage, it is built throughout of stone,
and not of brick. At the western end is a lock, the largest in Egypt,
through which the largest of the boats that ply upon the Nile can
easily pass. Below the water lies the strength of the structure. The
viaduct rests upon a solid platform of granite and cement, 10 feet
deep and 87 feet wide, set at a suitable depth below the bed of the
river. As a further precaution against the action of the water, there
are also below the platform two continuous lines of iron sheet-piles,
with hermetically sealed joints. With such a series of obstacles to
encounter, the danger of the water forcing its way through
underneath the Barrage is small indeed. The whole amount of
masonry used in this Barrage is 220,000 cubic yards.
The prosecution of these great works in Upper Egypt by no
means exhausted the activity of the Irrigation Department; indeed, it
would almost seem that the building of Barrages has become part of
its ordinary routine, for a third remains to be chronicled. This is the
Zifta Barrage on the Damietta branch of the Nile, halfway between
the point of the Delta and the sea. Because it lies in a district
unvisited by tourists, though very important commercially, its
construction has not been heralded by any blowing of trumpets. Yet
it is a work of the very first class, and deserves to be reckoned
among the greatest of the triumphs of the department. Built on the
same plan as the Assiout Barrage, and, like it, capable of holding up
10 feet of water, it is designed to secure a better distribution of the
supply north of the Delta Barrage. As the area of the cultivated land
extended gradually northward, it became apparent that the canal
system taking off from above the original Barrage was becoming too
long to admit of the water in times of pressure reaching the
northernmost parts of the country. There were some who held that a
second Barrage, with a new system of canals taking off from it,
should have been erected on the Damietta branch, even in
preference to the new stone weirs, which have increased the
strength and distributing power of the old one. The dispute has been
happily settled by the adoption of both projects, and with the Zifta
Barrage completed in time for the summer of 1903, and the new
supply from Assouan, the reclamation of the northern lands will go
steadily forward.
It has been already pointed out that it is almost as important to get
the water off the land as to get it on, and the proper drainage of the
Delta lands has been the necessary complement of all the new
schemes. Though eclipsed by the splendour of the Reservoir and
other creations more taking to the eye, the performance of the
engineers in this direction during the last few years has been
sufficient to make them very memorable in irrigation annals. Since
1896 about 1,000 kilometres of new drains have been dug, and
nearly as great a length of existing drains remodelled. It has cost the
Egyptian Government close upon a million of money, but that this
expenditure has not been thrown away is proved by the great rise in
the value of all the lands affected.
The Zifta Barrage cost about £500,000, the Assouan Dam and the
Assiout Barrage £3,200,000. Apart from the ordinary expenditure on
maintenance and the suppression of the corvée, the twenty years
ended 1902 have seen an expenditure of £9,000,000 devoted to
irrigation and drainage. There could be no greater proof of the
wisdom of those who have directed Egyptian policy during that
period. However pressed they have been at different times by
demands for immediate expenditure on other objects when
resources were low, they have always adhered steadily to a policy of
liberality towards public works likely to be of a remunerative
character. In no other country do economic laws work out their
results with greater directness and certainty. Reproductive
expenditure is really worthy of its name, and brings its visible and
tangible fruit almost without a moment’s delay. As they have sown,
so have they reaped. There can be no comparison between this
expenditure and the benefits it has conferred upon Egypt. Three
salient examples may be given to point the force of these remarks.
First, the works undertaken by Colonel Ross to improve the
system of basin irrigation in Upper Egypt. 1877 was a year of very
low flood, and nearly 1,000,000 acres were sharaki—that is, entirely
exempted from taxation owing to absence of irrigation. In 1899, a
worse year—in fact, the lowest flood of the century—the sharaki
lands were no more than 250,000 acres, a result mainly due to the
successful reforms carried out by Colonel Ross in 1888-89. 1902
was a year very similar to 1899, and the sharaki acreage was no
more than 140,000 acres.
Second, the case of the Assiout Barrage. The work was finished a
year before the time named in the contract. On August 15, 1902, the
usual date of filling the basins, the flood was exceptionally low, and it
was decided to lower the gates of the Barrage. By so doing the
water-level of the canals was raised by 1½ metres, an increase
which was more than sufficient to avert the threatened disaster. The
money value of the crops thus secured to the land-owners of the
Fayoum and Middle Egypt is estimated at not less than £600,000.
The cost of the new works at Assiout, including the new regulator on
the Ibrahimiyah Canal, was about £875,000. Thus, in the first year of
their existence they nearly repaid their whole cost, and that, too, as it
were, by a sidewind; for the Barrage’s real function is to hold up the
river in the low summer season, and not in the flood.
Third, the case of the Assouan Reservoir itself. The summer
supply in 1903 has been the lowest on record; the discharge at
Assouan has fallen to 200 cubic metres per second. By means of the
Reservoir this supply has been actually doubled. Had the flood been
late in coming down, it would have been impossible to distribute the
water at so liberal a rate. The Soudan gauges gave warning that an
early flood was to be expected, and thus the authorities were able to
calculate with certainty, and open the sluice-gates with much greater
freedom. But the calculation would have been a useless exercise
unless there had been a store to draw upon. At the lowest
computation the loss avoided may be reckoned at a couple of
millions.
The administrators of Egypt have had many difficulties and
obstacles in their path, but to be able to point to such results is a
great compensation; their efforts need no formal monument.
CHAPTER IX
THE INAUGURATION OF THE RESERVOIR
December 10, 1902, was the official date of the inauguration of the
Reservoir, a memorable day in the history of Egypt, and worthy to be
marked with red even in the unchanging Mohammedan calendar.
The making of the Dam has been a great time for Assouan. The
town has thriven and prospered beyond all knowledge since the
days—not so very long ago—when two British battalions occupied
the barracks on the hill overlooking the river to the south. The
barracks are crumbling to pieces now, and only one or two
blockhouses remain as memorials of the past state of siege and fear
of dervish raids. The tide of war has rolled far away and spent itself
utterly in remote corners. The whole of the Soudan lies between
Assouan and the frontier of any possible enemy. Even the yellow fort
is untenanted save by a few policemen. True, for four years an army
of 10,000 men has been marshalled here, but it was an invasion of
the arts of peace, creative and not destructive. Possibly the
inhabitants would have liked their occupation to go on for ever; but
even the best of times must have an end, and it was a good
occasion for a holiday. In every Nile village flags and bunting were
flying. For once the fields were deserted, and everywhere the people
crowded to the bank in the hope of catching a glimpse of the
Khedive and his distinguished guests. Here and there the gaffirs, or
local policemen, lined the shore, standing stiffly to attention, or
saluting with their Remingtons in the regulation attitude of European
soldiers, which contrasted quaintly with their loose, flowing robes
and white turbans.
At Assouan the faithful subjects of the Khedive surpassed
themselves. Sunny Assouan lends itself readily to a festal garb.
Situated where the Nile broadens out after emerging from the rocky
defile of the cataract, the town has a most picturesque aspect at all
times; its embanked river-front makes it the neatest of all the cities of
Upper Egypt. Elephantine and the Sirdar’s Island rise green and
smiling out of the broad bosom of the river; the perpetual blue sky
makes everything doubly attractive to the Northern visitor. Dressed
for the festival, it was a charming scene. Triumphal arches of the
sacred yellow and brown, gorgeous hangings and many-coloured
festoons, bore testimony to the Oriental love of vivid hues; steamers
and dahabiehs, moored in line along the shore as well as along
Elephantine Island, vied with each other in their decorations, and
numerous feluccas were plying to and fro, half hidden by their
burdens of flags and palm-leaves. At night thousands of lamps
decorated shore and river alike, and the whole scene resembled
nothing so much as a Henley in Regatta Week, with its illuminations
unrestrained by doubts of weather. To the ear, however, the voices of
the night told a very different tale. The crooning song of the Nubian
boatmen, ‘Great is the Prophet, praise be to him!’ accompanying the
creaking of their clumsy oars with monotonous persistency, sounded
weird and barbaric over the twinkling waters. The bustle in the town,
too, was no mere ordinary festal murmur; for this was the month of
Ramadan, and the feast of lamps meant a great deal to all faithful
Moslems. All day long they have abstained from food or drink, and
the going down of the sun is keenly welcomed as the end of one day
more of fasting.
The secret of the prosperity of Assouan lies in its granite. It is the
granite bed of the river at this point that makes the Reservoir
possible; here are the granite quarries from which the Dam was built,
and from which every ruler of Egypt who wished to raise a
monument for all time has drawn his supplies. Nothing that I have
seen in this country brings the past so near as these quarries. Here
lies a rough-hewn obelisk, just ready to be rolled away; here an
enormous block of stone half hollowed into a bath for an Emperor, or
a sarcophagus for an Apis bull, designed by some mighty ruler who
‘thought in continents,’ and recked little of the lives and labours of
thousands provided he gratified his whim. But suddenly death or
some other fate intervened, and a feebler or more merciful
generation has never taken up the work. You may see the marks of
the wedges on some great face of rock, as fresh as if it was only
yesterday that Pharaoh’s workmen had driven them in and poured
water on the wood till it swelled and burst the stone; down below is
the fallen piece still waiting for the mason who never came back to it.
Perhaps some of the very stones cut by Cheops or Rameses have
been smoothed and planed and set in the Great Dam.
Passing along the raised causeway, down which so many great
monuments have been rolled slowly to the river, I came through a
long stretch of burning desert to a spur on the northern extremity of
the granite hills. Here, unexpectedly, I found myself overlooking the
lake formed by the filling of the Reservoir. Graceful lines of palm-
trees showed where the banks of the river had once been. Philæ
was but an insignificant speck on the blue waters, overpowered by
the fantastic piles of granite boulders that hem in the valley. In the far
distance rose some lofty hills, crowned by dazzling sand that might
easily have been mistaken for snow. The Dam builders have been
accused of vandalism, but they have created a standing pool in the
wilderness of surpassing beauty. The view of the lake was not the
only attraction of the spot; at my feet lay a colossal statue of Osiris,
destined for some temple, but never moved from the spot where it
was hewn.
Ancient Egypt may well look on with scornful wonder at our pride
in our achievements. The Great Pyramid at Gizeh contains three
times as much solid masonry as the Dam, cut from these same
quarries. Every one of those huge blocks had to be dragged to the
river, and carried down 600 miles, before it was hoisted into its place.
As an achievement of mechanical power the Great Dam cannot
compare with the Great Pyramid; but when at last I climbed a little
hill hard by the river, below the Reservoir itself, and saw the whole
length of the great stone rampart, stretching right across the valley,
the contrast between the world of Pharaoh and our own came strong
upon me.
Pharaoh, to whom time and life were nothing, out of the misery of
the forced labour of his subjects, raised a perfectly useless
monument of his own folly; yet he achieved his object, and made his
tomb one of the wonders of the world. We, with the free labour of
voluntary workers—paid, fed, and cared for, instead of being driven
by the whip—have dared to harness Nile himself. It is a work vital to
the interests of millions of dwellers by the river. Yet who can say that
the fame of the Pyramid will not endure the longer? Hundreds of
years after the time of Cheops a mighty Dam was built in Southern
Arabia for a like purpose of irrigation; it lasted for eight centuries or
more, and its bursting in 100 A.D. is mentioned in the Koran. Its ruins
remain to this day, and show that it was larger than the Nile Dam.
Eight hundred years is a long life for a reservoir, but if this one lasts
a quarter of that period it will have repaid its cost many hundred
times over.
Certainly it looks strong enough to last as long as the Nile itself.
Strength, and nothing but strength, shows in every stone of it.
Square, solid, and massive, it runs from shore to shore in an
absolutely straight line, without the slightest attempt at any trace of
ornament or decoration. Clearly, effect has been the last thing
thought of. Even the sluice-gates have absolutely plain rectangular
openings, and it detracts from the symmetry of the design that,
owing to engineering exigencies, they are not all of one height, but
run in sections of five, some of them lower than others. The top of
the wall runs in a simple, unbroken level; there is nothing to catch
the eye as it travels up the steep face of masonry except the
slightest change in the angle of the slope to a nearly complete
perpendicular. The wall simply leaves off because its builders
thought it high enough for the present. Along the broad surface of
the summit runs a tramway, with plenty of room for a man to walk on
either side, flanked by perfectly plain, solid parapets as high as the
waist, and more than a yard thick, of a piece with the masonry below.
On the eastern side the Dam is unostentatiously built into the
living rocks; no arch or pylon marks its start. On the western side it is
flanked by a ladder of four immense locks, set in a mountainous
embankment. The gallows-like arms of the draw-bridge, hideous in
appearance, but a marvel of mechanical ingenuity, over the upper
gate of the highest lock are the only break in the long, unrelieved
level. The hard gray colour of the granite strengthens the general
impression, though in time every part exposed to the action of the
water will be coated with the shining black varnish which the Nile
mud always lays on granite, and it will look exactly the same as the
natural bed of the stream.
How different all this is from the prettiness of the Delta Barrage,
with its brickwork, originally designed and built under French
influence, adorned with archways and towers, of which the lovely
garden, with its flowers and shrubs, its green lawns and leafy trees,
on the tongue of land which it crosses, seems a natural and
appropriate part! The Assouan Dam is the work of a practical,
unimaginative race. Its builders have had before them the problem of
harnessing the great river with a yoke that cannot be broken; they
had to hold up a reservoir containing 100,000,000 tons of water, so
that for 140 miles the river is turned back upon itself; and they have
succeeded.
In December all but a few of the sluice-gates are shut, for the
reservoir has to be filled. But imagine it at the height of the flood,
when the collected rainfall of half a continent is crashing past at the
rate of nearly 1,000,000 tons a minute, and through each of the 180
openings shoots a solid cube of dark water to dash thundering in
clouds of white foam on the rocks below, and rush tumultuously
down the swirling slopes of the cataract. Think of the huge bulk of
water held up when the reservoir is full. Then you will understand
something of the difficulties of the work. Solidity and strength could
not but be the first and overpowering idea in the minds of the
builders. The longer you look, the more you are impressed. The vast
dimensions of the Dam grow upon you from moment to moment.
There is, after all, a fierce beauty in those uncompromising features,
grimly set in the determination to hold the river in bondage. The
massive structure is in harmony with the forces of Nature. Feeble
and puny it may be compared with even the least of their handiwork,
but it is impossible not to feel that its builders have been inspired
with a spark of the same creative power. They drew their plans, and
dug and built and strove their best to control the great river. They
have succeeded because a portion of that spirit of Nature against
which they struggled has passed into their work.
Is it too much to hope that a scheme of decoration may yet be
found in consonance with these ideas? Some day the Dam will be
raised to the full height of the original design, thus doubling the
present capacity of the Reservoir. Then will be the time to finish the
work magnificently, and make of it a stately monument, to be the
glory of Egypt as well as the foundation of her material prosperity.
The subject is worthy of a great artist. Only a scheme conceived on
grand lines, perfectly simple and bold, can have the least chance of
success. Anything else would be as ridiculous as a proposal to place
a statue of ordinary dimensions on the top of the Great Pyramid. The
difficulties are great, and so would be the expense, so great, indeed,
that it would be far better to avoid any attempt at decoration, unless
the results are to be admirable beyond all question. Are there no
possible successors to the architects of Karnak?
The ceremony of inauguration was, like the masonry of the Dam
itself, sensible and solid, but it was not impressive. Those who
arranged its details had forgotten the vastness of the theatre in
which it was performed. From the purely spectacular point of view it
was a failure. Egypt has no money yet to spend on functions.
Perhaps a better stage management might have made a better
display without any greater expense. The material benefits of English
rule would be appreciated none the less gratefully for a little gilding.
But the Englishman in Egypt has had other things to think about than
the organization of what the native would call ‘fantasias.’ So he fell
back on the established custom of his own country. Wherever he
goes he carries with him law and order and equity and righteousness
and commonsense, and he also carries a peculiar kind of public
ceremonial routine. Everybody knows it. The invited guests arrive in
special trains, and perspire in top-hats and frock-coats for an
interminable time, until Royalty arrives full of gracious smiles amid a
cheering crowd. The distinguished persons pass into a pen carpeted
with red baize, where all the notables are assembled. Somebody
makes a perfectly inaudible speech, which receives a gracious and
inaudible reply. A button is pressed here, a lever turned there, and
several extraordinary things begin to happen in consequence. Then
a number of good men and true receive some well-earned
decorations, Royalty graciously departs, and everybody presses
home as best he can, while the band plays the National Anthem. The
whole is accompanied by the clicking of innumerable cameras, and

Textbook Frontiers of Cyberlearning Emerging Technologies For Teaching and Learning J Michael Spector Ebook All Chapter PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Textbook Frontiers of Cyberlearning Emerging Technologies For Teaching and Learning J Michael Spector Ebook All Chapter PDF

Uploaded by

Copyright:

Available Formats

Frontiers of Cyberlearning Emerging

Technologies for Teaching and

Competencies in Teaching Learning and Educational

Digital Technologies Sustainable Innovations for

Emerging Technologies for Connected Internet of

Teaching Learning and New Technologies in Higher

Emerging Practices in Scholarship of Learning and

Educational Psychology for Learning and Teaching Susan

Portfolio Assessment for the Teaching and Learning of

Emerging Technologies for Waste Valorization and

More information about this series at http://www.springer.com/series/11777

Alfred Essa Yueh-Min Huang

Rob Koper Richard A. W. Tortorella

Ting-Wen Chang Yanyan Li

Vivekanandan Kumar Ting-Wen Chang

ISSN 2196-4963 ISSN 2196-4971 (electronic)

© Springer Nature Singapore Pte Ltd. 2018

Learning Any Time, Anywhere: Big Educational Data from Smart

Using Apache Spark for Modeling Student Behavior at Scale . . . . . . . . 169

Mark A. Riedesel and Patrick Charles

Abstract For many people, especially young people, a smartphone is a constant

M. A. Riedesel (B) · P. Charles

© Springer Nature Singapore Pte Ltd. 2018 1

Keywords Spaced practice · Adaptive flashcards · Mobile learning

2 Mobile Practice of Course Content

The Higher Education division of McGraw-Hill Education (MHE) was interested in

It was recognized that this software could be re-targeted to provide optimized

2.1 Smart Device Mobile Applications

2.2 User Interface

order to move on to the next page. For fill-in-the-blank, matching, matching-rank,

Fig. 3 The question

2.3 Algorithm Self-contained Within the Smart Device

A key element of effective memorization is repeating an item often enough for it to be

Fig. 4 The answer selection

Fig. 5 The answer response

3 Data Messaging System

The availability of cloud-based computing and storage allows a smart device to

3.1 Questions for Business Analytics

3.2 Questions for Learning Science

Table 1 Data messaging schema

3.4 JSON Schema

3.5 Online and Offline Modes

3.6 Receiving System

4 Security and Privacy

As a system which is used in education, it is very important to maintain the privacy of

4.1 Data Encryption

Data encryption is a critical element of security. Strong encryption of data both at

Furthermore, the pipeline runs in a virtual private cloud on infrastructure not

4.2 Access Policies and Controls

4.3 Data Integrity

An end-to-end data analytics pipeline, especially one reliant on third-party managed

5 Data Processing and Analysis

5.1 Apache Spark

Table 5.1. Scala

Table 5.2. Pyspark (Python)

eventSample Df = spark.read.json ("pathBatch / * . json")

Table 5.3. SparkR (R)

e v e n t S a m p l e D f = r e a d .df (s q l C o nt e x t , " p a t h B a t c h / . j son" , "json")

The Apache/Spark API also allows SQL-like operations to be performed on

Table 5.4. SQL query using Scala

The same operation expressed in Spark SQL syntax:

Table 5.5. SQL query directly in Spark SQL

5.2 Processing Pipeline