LITERATURE REVIEW

A Report submitted by

Devansh Parikh (R004)

70281018004
Under the guidance of

Mrs. Kriti Srivastava

in partial
fulfillment for the award of the degree of

MASTERS IN TECHNOLOGY
IN
ARTIFICIAL INTELLIGENCE
AT

MUKESH PATEL SCHOOL OF TECHNOLOGY AND 

MANAGEMENT STUDIES, NMIMS, MUMBAI 
2018-19 
 CERTIFICATE

This is to certify that the literature review entitled “Machine learning with Mice Protein
Expression Dataset”​​, has been done by ​Mr. Devansh Parikh under my guidance and
supervision for the degree of Masters of Technology in Artificial Intelligence of MPSTME,
SVKM’s NMIMS (Deemed-to-be University), Mumbai, India.

_______________

Mrs. Kriti Srivastava

Date
Place: Mumbai
1. Introduction
Down syndrome (DS) is the most common genetic cause of learning/memory deficits. It is
due to the presence of an extra copy of the long arm of human chromosome 21. The
overexpression of genes encoded by the extra copy of a normal chromosome in DS is
believed to be sufficient to perturb normal pathways and normal responses to stimulation,
causing learning and memory deficits.
The dataset chosen is the Mice Protein Expression dataset, from the UCI Machine Learning
repository.

In this project, we will aim to do two things: the first one is to correctly classify the type of
mouse, based on expression levels of 77 proteins. This will be a supervised learning problem.
The second one is an unsupervised learning problem, that aims to cluster groups of mice, that
have been treated differently(different classes), in order to assess the effect of the drug
memantine in recovering the ability to learn in trisomic mice, some mice have been injected
with the drug and others have not.

As mentioned before, down syndrome (DS) is the most common genetic cause of
learning/memory deficits. It is due to an extra copy of the long arm of human chromosome 21
(Hsa21) and the consequent increased level of expression, due to dosage, of some subset of
the genes it encodes. Although no pharmacotherapies for learning deficits in DS are
available, because the incidence is high (one in 1000 live births worldwide), there is
considerable interest in their identification.
However due to improvements in machine learning

Dataset Description

The data set consists of the expression levels of 77 proteins/protein modifications that
produced detectable signals in the nuclear fraction of cortex. There are 38 control mice and
34 trisomic mice (Down syndrome), for a total of 72 mice. In the experiments, 15
measurements were registered of each protein per sample/mouse. Therefore, for control mice,
there are 38x15, or 570 measurements, and for trisomic mice, there are 34x15, or 510
measurements. The dataset contains a total of 1080 measurements per protein. Each
measurement can be considered as an independent sample/mouse.

The eight classes of mice are described based on features such as genotype, behavior and
treatment. According to genotype, mice can be control or trisomic. According to behavior,
some mice have been stimulated to learn (context-shock) and others have not (shock-context)
and in order to assess the effect of the drug memantine in recovering the ability to learn in
trisomic mice, some mice have been injected with the drug and others have not.

Classes:
c-CS-s: control mice, stimulated to learn, injected with saline (9 mice)
c-CS-m: control mice, stimulated to learn, injected with memantine (10 mice)
c-SC-s: control mice, not stimulated to learn, injected with saline (9 mice)
c-SC-m: control mice, not stimulated to learn, injected with memantine (10 mice)

t-CS-s: trisomy mice, stimulated to learn, injected with saline (7 mice)

t-CS-m: trisomy mice, stimulated to learn, injected with memantine (9 mice)
t-SC-s: trisomy mice, not stimulated to learn, injected with saline (9 mice)
t-SC-m: trisomy mice, not stimulated to learn, injected with memantine (9 mice)

The aim is to identify subsets of proteins that are discriminant between the classes.
What is Down Syndrome?

In every cell in the human body there is a nucleus, where genetic material is stored in genes.
Genes carry the codes responsible for all of our inherited traits and are grouped along
rod-like structures called chromosomes. Typically, the nucleus of each cell contains 23 pairs
of chromosomes, half of which are inherited from each parent. Down syndrome occurs when
an individual has a full or partial extra copy of chromosome 21.

This additional genetic material alters the course of development and causes the
characteristics associated with Down syndrome. A few of the common physical traits of
Down syndrome are low muscle tone, small stature, an upward slant to the eyes, and a single
deep crease across the center of the palm – although each person with Down syndrome is a
unique individual and may possess these characteristics to different degrees, or not at all.

Memantine Drug

Memantine is used to treat moderate to severe Alzheimer’s disease. It acts on the

glutamatergic system by blocking NMDA receptors. It was first synthesized by Eli Lilly and
Company in 1968 as a potential agent to treat diabetes; the NMDA activity was discovered
in the 1980s.

Memantine is used to treat moderate-to-severe Alzheimer’s disease, especially for people

who are intolerant of or have a contraindication to AChE (acetylcholinesterase) inhibitors.
Memantine has been associated with a moderate decrease in clinical deterioration with only
a small positive effect on cognition, mood, behavior, and the ability to perform daily
activities in moderate to severe Alzheimer’s disease. There does not appear to be any benefit
in mild disease.

Saline

Saline, also known as saline solution, is a mixture of sodium chloride in water and has a
number of uses in medicine. Applied to the affected area it is used to clean wounds, help
remove contact lenses, and help with dry eyes. By injection into a vein it is used to treat
dehydration such as from gastroenteritis and diabetic ketoacidosis. It is also used to dilute
other medications to be given by injection.
2. Literature Review:
Summary of papers I have studied, while working on this project.

1. ​A Comparison of Prediction Accuracy, Complexity, and Training Time of Thirty-Three

Old and New Classification Algorithms
Authors:​ Tjen-Sien Lim, Wei-Yin Loh, Yu-Shan Snih
​Summary​:
Twenty-two decision tree, nine statistical, and two neural network algorithms are compared
on thirty two datasets in terms of classification accuracy, training time, and (in the case of
trees) number of leaves.
Classification accuracy is measured by mean error rate and mean rank of error rate. Both
criteria place a statistical, spline-based, algorithm called POLYCLASS at the top, although it
is not statistically significantly different from twenty other algorithms.
Another statistical algorithm, logistic regression, is second with respect to the two accuracy
criteria. The most accurate decision tree algorithm is QUEST with linear splits, which ranks
fourth and fifth, respectively.
Although spline-based statistical algorithms tend to have good accuracy, they also require
relatively long training times. POLYCLASS, for example, is third last in terms of median
training time. It often requires hours of training compared to seconds for other algorithms.
The QUEST and logistic regression algorithms are substantially faster. Among decision tree
algorithms with univariate splits, C4.5, IND-CART, and QUEST have the best combinations
of error rate and speed. But C4.5 tends to produce trees with twice as many leaves as those
from IND-CART and QUEST.

2. ​An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and

Variants
Authors​: Eric Bauer, Ron Kohavi
​Summary​:
Methods for voting classification algorithms, such as Bagging and AdaBoost, have been
shown to be very successful in improving the accuracy of certain classifiers for artificial and
real-world datasets. Review of these algorithms and describe a large empirical study
comparing several variants in conjunction with a decision tree inducer (three variants) and a
Naive-Bayes inducer.
The purpose of the study is to improve our understanding of why and when these algorithms,
which use perturbation, reweighting, and combination techniques, affect classification error.
We provide a bias and variance decomposition of the error to show how different methods
and variants influence these two terms.
This allows us to determine that Bagging reduces variance of unstable methods, while
boosting methods (AdaBoost and Arc-x4) reduced both the bias and variance of unstable
methods but increased the variance for Naive-Bayes, which was very stable. It was observed
that Arc-x4 behaves differently than AdaBoost if reweighting is used instead of resampling,
indicating a fundamental difference. Voting variants, some of which are introduced in this
paper, include: pruning versus no pruning, use of probabilistic estimates, weight perturbations
(Wagging), and backfitting of data.
It was found that Bagging improves when probabilistic estimates in conjunction with
no-pruning are used, as well as when the data was backfit. Measure of tree sizes show an
interesting positive correlation between the increase in the average tree size in AdaBoost
trials and its success in reducing the error. Comparison of the mean-squared error of voting
methods to non-voting methods and show that the voting methods lead to large and
significant reductions in the mean-squared errors. Practical problems that arise in
implementing boosting algorithms are explored, including numerical instabilities and
underflows.

3. ​Self-Organizing Feature Maps Identify Proteins Critical to Learning in a Mouse Model of

Down Syndrome
Authors​: Clara Higuera, Katheleen J. Gardiner, Krzysztof J. Cios
Summary​:
Down syndrome (DS) is a chromosomal abnormality (trisomy of human chromosome 21)
associated with intellectual disability and affecting approximately one in 1000 live births
worldwide. The overexpression of genes encoded by the extra copy of a normal chromosome
in DS is believed to be sufficient to perturb normal pathways and normal responses to
stimulation, causing learning and memory deficits. In this work, a strategy has been designed
based on the unsupervised clustering method, Self Organizing Maps (SOM), to identify
biologically important differences in protein levels in mice exposed to context fear
conditioning (CFC).
They analyzed expression levels of 77 proteins obtained from normal genotype control mice
and from their trisomic littermates (Ts65Dn) both with and without treatment with the drug
memantine. Control mice learn successfully while the trisomic mice fail, unless they are first
treated with the drug, which rescues their learning ability. The SOM approach identified
reduced subsets of proteins predicted to make the most critical contributions to normal
learning, to failed learning and rescued learning, and provides a visual representation of the
data that allows the user to extract patterns that may underlie novel biological responses to
the different kinds of learning and the response to memantine. Results suggest that the
application of SOM to new experimental data sets of complex protein profiles can be used to
identify common critical protein responses, which in turn may aid in identifying potentially
more effective drug targets.

4.​Protein Dynamics Associated with Failed and Rescued Learning in the Ts65Dn Mouse
Model of Down Syndrome
Authors​:
Md. Mahiuddin Ahmed, A. Ranjitha Dhanasekaran, Aaron Block, Suhong Tong, Alberto C.
S. Costa, Melissa Stasko,Katheleen J. Gardiner
Summary​:
The study examines the effect of metamine on the Ts65Dn mouse model. These mice display
many features relevant to those seen in DS, including deficits in learning and memory (L/M)
tasks requiring a functional hippocampus. The N-methyl-D-aspartate (NMDA) receptor
antagonist, memantine, was shown to rescue performance of the Ts65Dn in several L/M tasks
The results obtained are as follows:
(i) Of the dynamic responses seen in control mice in normal learning, >40% also occur in
Ts65Dn in failed learning or are compensated by baseline abnormalities, and thus are
considered necessary but not sufficient for successful learning.
(ii) Treatment with memantine does not in general normalize the initial protein levels but
instead induces direct and indirect responses in approximately half the proteins measured and
results in normalization of the endpoint protein levels.
5. ​Discovery and genetic localization of Down syndrome cerebellar phenotypes using the
Ts65Dn mouse
Authors​: ​Laura L. Baxter, Timothy H. Moran, Joan T. Richtsmeier, Juan Troncoso, Roger H.
Reeves
Summary​:
Down syndrome (DS) is the most common genetic cause of mental retardation and affects
many aspects of brain development. DS individuals exhibit an overall reduction in brain size
with a disproportionately greater reduction in cerebellar volume. The Ts65Dn mouse is
segmentally trisomic for the distal 12–15 Mb of mouse chromosome 16, a region that shows
perfect conserved linkage with human chromosome 21, and therefore provides a genetic
model for DS.
In this study, high resolution magnetic resonance imaging and histological analysis
demonstrate precise neuro- anatomical parallels between the DS and the Ts65Dn cerebellum.
Cerebellar volume is significantly reduced in Ts65Dn mice due to reduction of both the
internal granule layer and the molecular layer of the cerebellum. Granule cell number is
further reduced by a decrease in cell density in the internal granule layer. Despite these
changes in Ts65Dn cerebellar structure, motor deficits have not been detected in several tests.
Reduction in granule cell density in Ts65Dn mice correctly predicts an analogous pathology
in humans; a significant reduction in granule cell density in the DS cerebellum is reported
here for the first time.
Results:
The candidate region of genes on chromosome 21 affecting cerebellar development in DS is
therefore delimited to the subset of genes whose orthologs are at dosage imbalance in
Ts65Dn mice, providing the first localization of genes affecting a neuroanatomical phenotype
in DS. The application of this model for analysis of developmental perturbations is extended
by the accurate prediction of DS cerebellar phenotypes.

6. ​Unsupervised Learning Using Generative Adversarial Training And Clustering

Authors​:Vittal Premachandran and Alan L. Yuille
Summary​:
In this paper, an unsupervised learning approach has been proposed that makes use of two
components; a deep hierarchical feature extractor, and a more traditional clustering algorithm.
They train the feature extractor in a purely unsupervised manner using generative adversarial
training and, in the process, study the strengths of learning using a generative model as an
adversary. They also show that adversarial training as done in Generative Adversarial
Networks (GANs) is not sufficient to automatically group data into categorical clusters.
Instead, we use a more traditional grouping algorithm, k-means clustering, to cluster the
features learned using adversarial training. They experiment on three well-known datasets,
CIFAR10, CIFAR-100 and STL-10.
Results:
The experiments show that the proposed approach performs similarly to supervised learning
approaches, and, might even be better in situations with small amounts of labeled training
data and large amounts of unlabeled data.

7. ​Adversarial Autoencoders
Authors​:
Alireza Makhzani, Jonathon Shlens, Navdeep Jaitly, Ian Goodfellow, Brendan Frey
Summary​:
In this paper, they propose the "adversarial autoencoder" (AAE), which is a probabilistic
autoencoder that uses the recently proposed generative adversarial networks (GAN) to
perform variational inference by matching the aggregated posterior of the hidden code vector
of the autoencoder with an arbitrary prior distribution. Matching the aggregated posterior to
the prior ensures that generating from any part of prior space results in meaningful samples.
As a result, the decoder of the adversarial autoencoder learns a deep generative model that
maps the imposed prior to the data distribution. We show how the adversarial autoencoder
can be used in applications such as semi-supervised classification, disentangling style and
content of images, unsupervised clustering, dimensionality reduction and data visualization.
They performed experiments on MNIST, Street View House Numbers and Toronto Face
datasets and show that adversarial autoencoders achieve competitive results in generative
modeling and semi-supervised classification tasks.
Summary:

Sr. no Paper title Paper Summary Analysis

1. Comparison of Prediction Implementing 22 decision trees, 9 Refer tree algorithms

statistical learning and 2 neural for prediction of mice
Accuracy, Complexity,
networks and comparing accuracy classes.
and Training Time of and other metrics.
Thirty-Three Old and New
Classification Algorithms

2. An Empirical Comparison .Methods for voting classification Study bagging and

algorithms, such as Bagging and boosting techniques, for
of Voting Classification
AdaBoost, have been shown to be improving algorithms in
Algorithms: Bagging, very successful in improving the previous classification
accuracy of certain classifiers for problems.
Boosting, and Variants
artificial and real-world datasets.

3. Self-Organizing Feature Clustering algorithm applied to This the primary study

protein expression dataset, and paper, and will be
Maps Identify Proteins
studying the cluster of mice. referred for analysis.
Critical to Learning in a SOM Algorithm applied to create
clusters. Then the mice were
Mouse Model of Down
checked for down syndrome based
Syndrome on the Ts65Dn mice model. Effects
of the drug memantine were
observed and it was checked if it
made a difference in motor
functions.

4. Protein Dynamics This paper further examines the More biologically

effects of memantine motor learning inclined will be used for
Associated with Failed
in the Ts65Dn model of mice. further analysis.
and Rescued Learning in This is a static analysis
of results and not a
the Ts65Dn Mouse Model
machine learning
of Down Syndrome model.

5. Discovery and genetic Down Syndrome causes reduction in Static biological

brain size and analysis of cerebellum analysis. Will be used
localization of Down
in Ts65Dn model of mouse. for study and future
syndrome cerebellar work.
phenotypes using the
Ts65Dn mouse

6. Unsupervised Learning GAN’s are at the cutting edge of New and improved
unsupervised learning, and along algorithm for clustering.
Using Generative
with a traditional method like
clustering, they are shown to give
promising results.
 Adversarial Training And
Clustering

7. Adversarial Autoencoders This is an unsupervised learning New algorithm for

algorithm, and is a probabilistic clustering with
autoencoder that uses the recently promising results.
proposed GAN’s to perform
variational inference.