Cognitive Computing applications in education

• Humans learn from a very small number of examples.

• In learning and education, we generate a very large amount of data
through our learning activities, which presents a potential as well as a
challenge.
• Can be related to Big Data.
• Big data characterized by 5 Vs – volume, variety, velocity, veracity and
value.
• In the education and learning settings, first four Vs are visible.
• Eg. LMS or MOOCs generates a huge amount of data. Data are
generated in a variety of forms.

Educational Data Mining (EDM) and Learning Analytics (LA)

• Emerging areas, major applications of cognitive computing to education

and learning.
• Humans play a major role in LA and EDM is performed through
automation.
Cognitive Computing applications in education

EDM

• Help teachers, students and other stakeholders achieve their respective

objectives by utilizing big data in education.
• The International Educational Data Mining Society (2016) has defined
the field as “concerned with developing methods for exploring the
unique and increasingly large-scale data that come from educational
settings and using those methods to better understand students and
the settings which they learn in “.

Eg. Tutor has access to multidimensional data about student’s learning

behavior on an online learning system - student’s number of sign-ins and
percentage of lesson completed. Goal of the teacher is to classify students
according to their behavior.
Cognitive Computing applications in education

EDM

• Unsupervised learning problem, use of k-means clustering algorithm,

finally formation of four clusters, based on this the tutor will assign
different learning paths for different types of students.
• Main focus of EDM is automated discovery.

LA

• It is a new field with the overarching goal of synergistically combining

human judgement with technological advances for a wide range of
learning activities ranging from classroom learning to informal learning.
• In 2011, LA community defined it as “is the measurement, collection,
analysis and reporting of data about learners and their contexts, for
understanding and optimizing learning and the environments in which it
occurs”.
• 5 major branches of LA – analytic approaches, methods and tools,
theories and theoretical concepts, measures of learning , change and
success
Cognitive Computing applications in education

LA

• Learning activities, applications and interventions and data sources.

• EDM is focused on automation and concerned with unearthing hidden

patterns from data, LA is more holistic and human-centric.
• LA is more concerned with predicting the outcome of a learning
approach or system. It is an area of analytics and various analytical tools
such as discourse analysis and social network analysis are more
prominent.
INTelligent tutoring systems (ITS)

• Modern ITS – cognitive architecture behind ITS is Anderson’ s Adaptive Character

of Thought – Rational (ACT-R) theory.
• The central concept is that human cognition is the result of interactions among
numerous small, indivisible units of knowledge in certain ways.
• Two types of knowledge – declarative and procedural.
• Declarative knowledge – states of cognition - objects in the environment.
• Procedural units of knowledge, also known as productions, transform human
cognition from one state to another.
• Today Python and Java implementations of the ACT-R theory is available.
• ITS is being used for helping students learn programming.

CLUSTERING and STUDENT MODELING

• Clustering is a common technique in EDM for aggregating student data in order to

examine student behavior.
• Student modeling – modeling learning curves.
• Here an individual’s learning curve represents the error percentage over time.
• Modeling prior knowledge is equally important.
PREDICTING STUDENT PERFORMANCE

• Popular line of research in EDM.

• Student performance has been measured in individual disciplines.
• Some questions – how would one predict student performance ?, what factors are
most important for a student’s success? , How can an ITS make use of this
prediction?

AFFECT DETECTION and STUDENT ENGAGEMENT

• Affective states are closely related to learning and cognition.

• Students react differently to different experiences during the learning process.
• Assessing affective states is gaining importance within both the EDM and LA
communities.
• Eye tracking - an active research topic in HCI, since its inception and is now
contributing to CC in learning.
• Students’ affective states are closely tried to their learning and therefore to their
performance.
IBM Watson in Education

Areas
(i) Administration – analyzes all possible data.
(ii) Research – offer research capability to different industries.
(iii) Research dedicated to students – make student life easier - a student
coming from an international exchange can access via their mobile and
interact in their native language on legal issues about their visa,
campus information, legal information that may affect them, etc.

• Coccoli et al. (2014) – smarter university model – CC based applications

and services should be adopted for administration, management and
learning activities.
• E-learning point of view – CC can be an accelerator for students’
achievements and a valuable support for the teachers. (i) integrating CC
services in software applications can strongly enhance students’
performances in computer science classes, (ii) studying CC behavior can
lead to significant results in AI related studies, (iii) using a CC layer for
digital interactions with students can enhance their performances and
ease the teacher’s job in managing classes and learning materials.
https://www.learntechlib.org/p/173468/
 Cognitive Computing applications in education

LA example
• Gudivada (2016) proposed an
application architecture for
personalized learning.

• Learning applications
generate huge volumes of
structured and unstructured
data.
• Structured data – LMS log’s
• Unstructured data – blogs,
emails, course messages,
discussion etc.
• Dealing with unstructured
data is essential.
• This framework works with
both structured and
unstructured data.
• Interactive System for
Personalized eLearning
(ISPeL)
 Cognitive Computing applications in education

ISPeL

• Hierarchical structure
• Network and hardware layer, databases and data analytics layer,
cognitive analytics layer and personalization layer.
• The bottom layer consists of 9 databases
• These databases dynamically keep track of information from various
sources in order to provide the students a personalized learning
experience.
(i) DITA topics database – keeps information about the topics being
taught.

(ii) Assessment data – each student’s score, time to complete tests,

performance level on tests and other related information.
(iii) User Models database – information about individual students, their
preferences and progress and learning styles.
(iv) Question generation models, Scaffolding & Feedback models and
Personalization models – main components for generating
personalized questions, feedback and lesson plans.
- capability to automatically generate questions for assessing learners
-practice exams before the actual test.
 Cognitive Computing applications in education

(iv) Question generation models, Scaffolding & Feedback models and

Personalization models –

- features capability to provide guided learning and context-

appropriate feedback.
- uses prerequisite dependencies among DITA topics and user models to
achieve personalization.
Each learner progresses at his/ her own pace, selects learning materials in
any order subject to prerequisite constraints.

(v) Cognitive analytics – uses data along with various algorithmic and
analytical techniques, to provide personalized learning to students.

(vi) anytime, anywhere, just-in time – multiple modes of personalized

learning.

(vii) Authentication and authorization

(viii) User interface - users with different credentials interact with
software application.
Emerging Cognitive Computing Areas

• Capabilities of CC systems in several industries, including health care,

manufacturing etc.
• At the heart of every CC system is a continuous learning engine that
improves with experience and that can return probabilistic results when
the data supports multiple candidate answers.
• Can use images, gestures, sound and streaming data from internet-enabled
sensors.
• Increased volume of unstructured data and advancements in technology are
the major driving forces.
• Other factors – cloud based technological advancements, development of
innovative hardware and software systems and cognitive experience
interfaces.
• NL processing is the most prominent technology.
• Adoption of cognitive analytics in healthcare and BFSI sectors – major
share.
Emerging Cognitive Computing Areas
https://www.wrcbtv.com/story/41595103/cognitive-computing-technology-
market-2020-2023-business-trends-emerging-technologies-size-global-
segments-and-industry-profit-growth

Top players dominating the global CC technology market

(i) Expert System S.p.A (Italy) (xvii) DataRobot (USA)
(ii) SparkCognition Inc. (USA) (xviii) DigitalGenius (UK)
(iii) IBM (USA) (xix) Cylance (USA)
(iv) Microsoft Corporation (USA) (xx) Ross Intelligence (USA)
(v) DeepMind Technologies Limited (UK) (xxi) Planet Labs (USA)
(vi) Numenta (USA) (xxii) Orbital Insight (USA)
(vii) CustomerMatrix Inc. (USA) (xxiii) Darktrace (UK)
(viii)Cisco Systems Inc. (USA) (xxiv) Vicarious (USA)
(ix) HP Enterprise (USA) (xxv) Indico (USA)
(x) CognitiveScale (USA) (xxvi)Cyberlytic (UK)
(xi) Google, Inc. (USA) (xxviii) Vicarious (USA)
(xii) Saffron Technology, Inc. (USA)
(xiii)Palantir Technologies (USA)
(xiv)Enterra Solutions LLC (USA)
(xv) ColdLight Solutions (USA)
(xvi)Airware (USA)
CC Technology market segment analysis

• By technology – NLP, ML, automated reasoning etc.

• By organization size – small, medium and large enterprise
• By end-user – banking & financial service, retail, aerospace & defence,
IT & telecommunication, energy & power, automotive & transportation
etc.
• By deployment – on-premises, on-cloud based on security and pricing
• By region – North America, Europe, Asia Pacific others.
Emerging Cognitive Computing Areas

• Ideal candidates for CC applications include:

(i) Industries with rapidly increasing or changing volumes of domain-
specific knowledge. Eg. Retail, travel etc. Presale advice is valuable to
buyers and costly to sellers.
(ii) Industries where post-sales upport / diagnostics / professional services
are cost centers or revenue opportunities due to complex products. Eg.
Industries requiring call centers, requiring constant training.
(iii) Industries with lot of specialty knowledge or experiences that are highly
concentrated in a small group of experts.
(iv) Industries following a modern apprentice / intern model for training and
certification. Eg. Law, medicine and financial services.
(v) Industries in which there is a wide variance between the best and least-
effective practitioners in the field.
(vi) Industries in which sudden availability of sensor data creates
opportunities that cannot be exploited using conventional means. Eg.
Transportation to healthcare.
(vii) Industries in which success depends on discovering patterns in a large
volume of data, particularly unstructured natural language text. Eg.
Natural sciences, pharmaceuticals etc.
Retail

• Need to anticipate what products to purchase based on forecasting trends

in advance.
• Understand the impact of changing economic, social and demographic
factors.
• Several factors can affect buying behavior.
• Large firms have used predictive analytics and scenario planning tools.
• Retailers miss subtle changes in buying preferences and do not anticipate
opportunities.
• CS have the potential to help retailers leverage knowledge in inventive
ways.
• Changes may be detected earlier, enabling the retailers to implement
innovative new practices and approaches that can change the customer
experience.
• Personalized customer service - personalized attention to a buyer’s wants
and needs by using extensive knowledge about the products.
• Retail staff training and support – critical need for training, employees need
to understand the products they sell and need access to best practices.
• Use of IBM Watson kind of product - help sales associates analyze
customer demographics and purchasing histories, provide access to
product information and market feedback from sentiment analysis.
Travel

• Need for personal touch of an experienced travel agent who can understand
the customer, ask qualifying questions to ensure good fit between personal
goals and desires and available inventory of transportation, lodging and
experience options.
• CC applications captures information explicitly by capturing patterns of
traveler’s behavior.
• Potential for implicitly understanding travelers by monitoring social media
streams.
• Allow travelers to interact via an NLP interface with the system.

Transportation and Logistics

• Face stiff competition, regulatory pressures, danger from man-made and

natural causes etc.
• Keeping infrastructure safe, identifying patterns of customer behavior
• Use of CC to use process sensor data that needs to be interpreted in near
real time to identify opportunities for efficiency and safety.
• Provide diagnostic and preventative maintenance recommendations.
Security and Threat Detection

• Commercial network security is a concern for business continuity and

general risk management.
• Networks, websites and applications in cloud are all targets.
• Cyberterrorism

• Use of CC for threat detection - speed at which new threats are developed,
speed with which damage can be done before an attack can be controlled,
complexities of networks that are beyond the capabilities of conventional
systems and network managers.
• Ml algorithms can identify anomalies b comparing current activity with
historical activity.

Other areas

(1) Call centers - cross industry function that is critical to the reputation and
management of an organization. Need for deep knowledge, high rates of
personnel turnover, need to store the best practices and knowledge, need for
call center people to understand and comply with compliance requirements for
their industries.
Other areas

(2) Financial services – gain an understanding an individual’s requirements

and the best possible offerings, large volume and variety of data, CC system
can learn based on patterns of success to provide best next actions.

(3) Legal applications – heavily focused on unstructured documents, data

comes from emails to tweets etc. Need to use advanced NLP systems and
pattern recognition algorithms. Today common practice is to use the
Electronic Discovery Reference Model (EDRM).

(4) Marketing applications – analyzing existing campaigns, use of predictive

analytics to anticipate future customer requirements, actively monitor
information related to customer and prospect interactions. Need for a well –
trained continuous learning system that could evaluate alternatives and help
marketers refine messaging and pricing by asking the right questions early in
the process.
Health care

• Humans generate enormous amount of health related data -

personal fitness trackers, mobile apps, EMR and genomic and
clinical research.
• Much of this information is either discarded or underutilized.
Health care

• Patients want personalized, transparent, integrated and high-

quality care.
• By 2020 - foot print of medical data will double every 73 days,
80 % of this data will be unstructured.
• Medial professionals have access to this data, but volume is too
significant for them to consume, analyze and apply in
meaningful ways, growth and pace of medical research, clinical
trials and treatment options are staggering, a doctor may need
at least 150 hours each week to read every piece of content
published in his / her domain.
• Data – EMR, clinical research, pathology reports, lab results,
radiology images, voice recording and exogenous data =
difficult to share as they are fragmented.
• More than 1 million GBs of health related data in the life time of
an individual – equivalent of 300 million books.
Health care

https://pulse.embs.org/may-2017/cognitive-
computing-and-the-future-of-health-care/
Health care

Why CC in Health care?

 Enables researchers to uncover new insights in relationships

among genes, proteins, pathways, phenotypes and diseases.
 Helps identify the most critical attributes of a patient case and
can provide easy-to-consume summaries for both patients and
health care providers.
 Cognitive capabilities have been used in clinical trial matching
to optimize patient selection and recruitment.
 Assist in creating individualized treatment plans and thereby
improve the patient and physician experience.
 Help attention-fatigued radiologists quickly identify anomalies
of interest in images.
 Patients will be able to send updates to their doctors through
speech or text, anytime and from anywhere.
 Analyzing patients’ descriptions of symptoms in the context of
their demographics can help diagnose cardiovascular disease.
 https://pulse.embs.org/may-2017/cognitive-
Health care
computing-and-the-future-of-health-care/
Health care

• Transformation of healthcare industry is not only occurring at

the point of care.
• Indexing diverse wealth of knowledge found in medical
textbooks, research papers, clinical studies and the human
genome will benefit research and education.
• Persistent challenges is the need to find patterns and outliers in
both structured and unstructured data that can help to improve
patient care.
• Transformation from document-centric silos to well –integrated
knowledge bases.

Healthcare industry ecosystem

• Health care providers

• Health care payers
• Medical device manufacturers
• Pharmaceutical firms
• Independent research labs
• Health information providers
• Government regulatory agencies.
Constituents in the Health care Ecosystem

• Different sources of relevant health care data

• Some of this data is shared and most of it is controlled by
regulations and security requirements.
• For better performance, need for more predictive analysis and
ML and for this continued improvement in the consistency of
data shared across the ecosystem.
Constituents in the Health care Ecosystem
Data managed and leveraged includes:
(i) Patients – from family history and habits to test results,
aggregation of information from different individuals.
(ii) Providers – data covers a broad range of unstructured and
structured sources. Eg. Patient medical records (EMR,
doctor’s office notes and lab data), data from sensors and
medical devices, intake records from the hospital, medical
text books, journal articles, clinical research studies,
regulatory reports, billing data and operational expense data.
(iii) Pharmaceutical companies – data to support pharmaceutical
research, clinical trials, drug effectiveness, competitive data
and drug prescription by medical providers.
(iv) Payers – billing data and utilization review data
(v) Government agencies – regulatory data
(vi) Data service providers – prescription drug usage and
effectiveness data, healthcare terminology taxonomies and
software solutions to analyze health care data.
Basic steps to build a CA in health care:
• To develop an application, need to begin by defining the target
end user and train the system to meet the needs of the user
base.
• A CS needs to start with a base level of information from which
it can begin to find the linkages and patterns that can help it to
learn.
• The CS can make associations between questions, answers
and content to help the user understand the subject matter at a
deeper level.

(1) Define the questions the users will ask

• Assembling the types of questions that will be asked, assemble
knowledge base required to answer these questions and train
the system effectively.
• Need to define the overall application strategy.
• If you begin with the corpus, cannot meet the needs of the end
users, when moved into operational state.
• Various types of questions, is its consumer-focused application
used by general users or technical experts.
• Start with 1000 -2000 QA pairs. Answers determined by subject
experts.
Basic steps to build a CA in health care:
(2) Ingest content to create the corpus

• All documents needed to answer queries to be included in the

corpus.
• Need to identify the resources already existing and needed to
provide the right knowledge base.
• Eg. Medical texts, background information on health subjects
such as pharmaceutical research, clinical studies and
nutrition, medical journal articles, patient records and
ontologies and taxonomies.
• Content needs to be validated.
• Adding meta tags to the content helps in creating association
between documents.
• Need to optimize the format for some of the source data to
ensure that it can be properly identified and searched.
• Need to understand the lifecycle of documents that are
ingested to plan for appropriately scheduled updates.
• To be notified about new and updated content.
• The corpus needs to be updated continuously throughout the
life of the application to make it viable.
Basic steps to build a CA in health care:
(3) Training the Cognitive system

• The more QA pairs that are analyzed, the more the system
learns and understands.
• This is the key part of the overall training process.
• Although it is important for representative users to generate
the questions, the experts need to generate the answers and
finalize the QA pairs.
• The questions need to be consistent with the level of
knowledge of the end user.
• Experts need to ensure that the answers are accurate and in
line with the content of the corpus.

(4) Question enrichment and adding to the corpus

• Training process to ensure the application works as intended,

when it becomes operational.
• Training process needs to be repeated initially.
• Add content to the corpus to cover areas in which there is
inadequate information.
Basic steps to build a CA in health care:
(4) Question enrichment and adding to the corpus

• Continuous training process to update QA pair and adding to

the corpus.
• Use of expansion algorithms to determine the additional
information required.

IBM Watson in Health Care

• April 2015 – IBM Watson Health and Watson Health Cloud

Platform.
• Improve the ability of doctors, researchers, insurers etc., by
providing insights from massive amounts of personal health
data created and shared daily.
• New York’s Memorial Sloan-Kettering Cancer Centre
• MD Anderson Cancer Centre at the University of Texas, Austin
• Minnesota’s Mayo Clinic
• American Diabetes Association
• Baylor College of Medicine, Texas
IBM Watson for Clinical Trial Matching

• Clinical trials are at the heart of all medical advances.

• Current process of trial matching – matching patient with a
study protocol through records, 46 requirements , huge range
of data to be matched, makes this a complex and difficult task.
• IBM Watson analyzes patient’s data against clinical trial
databases and offers feedback
• Support higher level of personalized care by enabling health
information to be securely connected with a vast array of
clinical trials.

Virtual Cancer Patient Health Advisor

• Combining the breadth and depth of cancer information from

American Cancer Society (ACS) and IBM to create a patient
‘virtual advisor’.
• Will mine ACS’s cancer.org website, which contains 14000
pages of detailed information on more than 70 cancer topics
including healthy lifestyles, risk reduction and early detection.
• Watson will ingest ACS’s National Cancer Information Center’s
patient data regarding self-management, support groups
Virtual Cancer Patient Health Advisor

• Health / wellness activities and cancer education.

• The advisor will anticipate the needs of the people with
different types of cancer, at different stages and at various
points in treatment. It will be dynamic and engaging.
• Use of Watson’s voice recognition technology and NLP
technology enabling users to ask questions and receive audible
responses.
• Eg. Person with breast cancer asks reasons for experiencing
unusual pain – advisor gives information on symptoms and self-
management options associated with current and future phases
of treatment, based on experiences of similar patients.