Linking and Mining Heterogeneous and

Multi-view Data Deepak P

Visit to download the full and correct content document:
https://textbookfull.com/product/linking-and-mining-heterogeneous-and-multi-view-dat
a-deepak-p/
More products digital (pdf, epub, mobi) instant
download maybe you interests ...

Data Science For Fake News: Surveys And Perspectives

Deepak P

https://textbookfull.com/product/data-science-for-fake-news-
surveys-and-perspectives-deepak-p/

Learning Representation for Multi-View Data Analysis:

Models and Applications Zhengming Ding

https://textbookfull.com/product/learning-representation-for-
multi-view-data-analysis-models-and-applications-zhengming-ding/

Data Mining and Big Data Ying Tan

https://textbookfull.com/product/data-mining-and-big-data-ying-
tan/

Mobile Data Mining and Applications Hao Jiang

https://textbookfull.com/product/mobile-data-mining-and-
applications-hao-jiang/
Multi-View Geometry Based Visual Perception and Control
of Robotic Systems First Edition Chen

https://textbookfull.com/product/multi-view-geometry-based-
visual-perception-and-control-of-robotic-systems-first-edition-
chen/

Data Mining and Data Warehousing: Principles and

Practical Techniques 1st Edition Parteek Bhatia

https://textbookfull.com/product/data-mining-and-data-
warehousing-principles-and-practical-techniques-1st-edition-
parteek-bhatia/

R Data Mining Implement data mining techniques through

practical use cases and real world datasets 1st Edition
Andrea Cirillo

https://textbookfull.com/product/r-data-mining-implement-data-
mining-techniques-through-practical-use-cases-and-real-world-
datasets-1st-edition-andrea-cirillo/

Transparent Data Mining for Big and Small Data 1st

Edition Tania Cerquitelli

https://textbookfull.com/product/transparent-data-mining-for-big-
and-small-data-1st-edition-tania-cerquitelli/

Data Mining Yee Ling Boo

https://textbookfull.com/product/data-mining-yee-ling-boo/
Unsupervised and Semi-Supervised Learning
Series Editor: M. Emre Celebi

Deepak P
Anna Jurek-Loughrey Editors

Linking and Mining

Heterogeneous and
Multi-view Data
Unsupervised and Semi-Supervised Learning
Springer’s Unsupervised and Semi-Supervised Learning book series covers the
latest theoretical and practical developments in unsupervised and semi-supervised
learning. Titles – including monographs, contributed works, professional books, and
textbooks – tackle various issues surrounding the proliferation of massive amounts
of unlabeled data in many application domains and how unsupervised learning
algorithms can automatically discover interesting and useful patterns in such
data. The books discuss how these algorithms have found numerous applications
including pattern recognition, market basket analysis, web mining, social network
analysis, information retrieval, recommender systems, market research, intrusion
detection, and fraud detection. Books also discuss semi-supervised algorithms,
which can make use of both labeled and unlabeled data and can be useful in
application domains where unlabeled data is abundant, yet it is possible to obtain a
small amount of labeled data.

More information about this series at http://www.springer.com/series/15892

Deepak P • Anna Jurek-Loughrey
Editors

Linking and Mining

Heterogeneous and
Multi-view Data

123
Editors
Deepak P Anna Jurek-Loughrey
Queen’s University Belfast Queen’s University Belfast
Northern Ireland, UK Northern Ireland, UK

ISSN 2522-848X ISSN 2522-8498 (electronic)

Unsupervised and Semi-Supervised Learning
ISBN 978-3-030-01871-9 ISBN 978-3-030-01872-6 (eBook)
https://doi.org/10.1007/978-3-030-01872-6

Library of Congress Control Number: 2018962409

© Springer Nature Switzerland AG 2019

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology
now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this book
are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or
the editors give a warranty, express or implied, with respect to the material contained herein or for any
errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional
claims in published maps and institutional affiliations.

This Springer imprint is published by the registered company Springer Nature Switzerland AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
Preface

With data being recorded at never seen before scales within an increasingly complex
world, computing systems routinely need to deal with a large variety of data types
from a plurality of data sources. More often than not, the same entity may manifest
across different data sources differently. To be able to arrive at a well-rounded
view across all data sources to fuel data-driven applications, such different entity
representations may need to be linked, and such linked information be mined
while being mindful of the differences between the different views. The goal of
this volume is to offer a snapshot of the advances in this emerging area of data
science and big data systems. The intended audience includes researchers and
practitioners who work on applications and methodologies related to linking and
mining heterogeneous and multi-view data.
The book covers a number of chapters on methods and algorithms—including
both novel methods and surveys—and some that are more application-oriented
and targeted towards specific domains. To keep the narrative interesting and to
avoid monotony for the reader, chapters on methods and applications have been
interleaved.
The first four chapters provide reviews of some of the foundational streams of
research in the area. The first chapter is an overview of state-of-the-art methods
for multi-view data completion, a task of much importance since each entity may
not be represented fully across all different views. The second chapter surveys the
field of multi-view data clustering, the most explored area of unsupervised learning
within the realm of multi-view data. We then turn our attention to linking, with
the third chapter taking stock of unsupervised and semi-supervised methods to
linking data records from across data sources. Record linkage is a computationally
expensive task, and blocking methods often come in handy to rein in the search
space. Unsupervised and semi-supervised blocking techniques form the focus of the
fourth chapter.
The fifth chapter is the first of application articles in the volume, and it surveys a
rich field of application of multi-view data analytics, that of transportation systems
that encompass a wide variety of sensors capturing different kinds of transportation
data. Social media discussions often manifest with intricate structure along with the

v
vi Preface

textual content, and identifying discourse acts within such data forms the topic of
the sixth chapter. The seventh chapter looks at imbalanced datasets with an eye on
improving per-class classification accuracy. They propose novel methods that model
the co-operation between different data views in order to improve classification
accuracy. The eighth chapter is focused on an enterprise information retrieval
scenario, where a fast and simple model for linking entities from domain-specific
knowledge graphs to IR-style queries is proposed.
Chapter 9 takes us back to multi-view clustering and focuses on surveying
methods that build upon non-negative matrix factorization and manifold learning for
the task. The tenth chapter considers how heterogeneous data sources are leveraged
in data science methods for fake news detection, an emerging area of high interest
from a societal perspective. Chapter 11 is an expanded version of a conference
paper that appeared in AISTATS 2018 and addresses the task of multi-view metric
learning. Social media analytics, in addition to leveraging conventional types of
data, is characterized by abundant usage of structure. Community detection is
among the fundamental tasks within social media analytics; the last chapter in this
volume considers evaluation of community detection methods for social media.
We hope that this volume, focused on both linking heterogeneous data sources
and methods for analytics over linked data sources, will demonstrate the significant
progress that has occurred in this field in recent years. We also hope that the
developments reported in this volume will motivate further research in this exciting
field.

Belfast, UK Deepak P
Belfast, UK Anna Jurek-Loughrey
Contents

1 Multi-View Data Completion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Sahely Bhadra
2 Multi-View Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Deepak P and Anna Jurek-Loughrey
3 Semi-supervised and Unsupervised Approaches to Record
Pairs Classification in Multi-Source Data Linkage . . . . . . . . . . . . . . . . . . . . . 55
Anna Jurek-Loughrey and Deepak P
4 A Review of Unsupervised and Semi-supervised Blocking
Methods for Record Linkage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
Kevin O’Hare, Anna Jurek-Loughrey, and Cassio de Campos
5 Traffic Sensing and Assessing in Digital Transportation Systems. . . . . 107
Hana Rabbouch, Foued Saâdaoui, and Rafaa Mraihi
6 How Did the Discussion Go: Discourse Act Classification
in Social Media Conversations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
Subhabrata Dutta, Tanmoy Chakraborty, and Dipankar Das
7 Learning from Imbalanced Datasets with Cross-View
Cooperation-Based Ensemble Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
Cécile Capponi and Sokol Koço
8 Entity Linking in Enterprise Search: Combining Textual
and Structural Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
Sumit Bhatia
9 Clustering Multi-View Data Using Non-negative Matrix
Factorization and Manifold Learning for Effective
Understanding: A Survey Paper . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
Khanh Luong and Richi Nayak

vii
viii Contents

10 Leveraging Heterogeneous Data for Fake News Detection. . . . . . . . . . . . . 229

K. Anoop, Manjary P. Gangan, Deepak P, and V. L. Lajish
11 General Framework for Multi-View Metric Learning . . . . . . . . . . . . . . . . . 265
Riikka Huusari, Hachem Kadri, and Cécile Capponi
12 On the Evaluation of Community Detection Algorithms
on Heterogeneous Social Media Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295
Antonela Tommasel and Daniela Godoy

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335
Chapter 1
Multi-View Data Completion

Sahely Bhadra

Abstract Multi-view learning has been explored in various applications such as

bioinformatics, natural language processing and multimedia analysis. Often multi-
view learning methods commonly assume that full feature matrices or kernel
matrices for all views are available. However, in partial data analytics, it is common
that information from some sources is not available or missing for some data-points.
Such lack of information can be categorized into two types. (1) Incomplete view:
information of a data-point is partially missing in some views. (2) Missing view:
information of a data-point is entirely missing in some views, but information for
that data-point is fully available in other views (no partially missing data-point in a
view).
Although multi-view learning in the presence of missing data has drawn a great
amount of attention in the recent past and there are quite a lot of research papers on
multi-view data completion, but there is no comprehensive introduction and review
of current approaches on multi-view data completion. We address this gap in this
chapter through describing the multi-view data completion methods.
In this chapter, we will mainly discuss existing methods to deal with missing view
problem. We describe a simple taxonomy of the current approaches. And for each
category, representative as well as newly proposed models are presented. We also
attempt to identify promising avenues and point out some specific challenges which
can hopefully promote further research in this rapidly developing field.

1.1 Introduction

Multi-View Data Multi-view learning is an emerging field to deal with data

collected from multiple sources or “views” to utilize the complementary information
in them. In case of multi-view learning, the scientific data are collected from diverse

S. Bhadra ()
Indian Institute of Technology, Palakkad, India
e-mail: sahely@iitpkd.ac.in; https://sites.google.com/iitpkd.ac.in/sahelybhadra/home

© Springer Nature Switzerland AG 2019 1

Deepak P, A. Jurek-Loughrey (eds.), Linking and Mining Heterogeneous
and Multi-view Data, Unsupervised and Semi-Supervised Learning,
https://doi.org/10.1007/978-3-030-01872-6_1
2 S. Bhadra

domains or obtained from various feature extraction techniques. For example, in

current biomedical research, a large amount of data is collected from a single
patient, such as clinical records (e.g., age, sex, histories, pathology and therapeutics)
and high-throughput omics data (e.g., genomics, transcriptomics, proteomics and
metabolomics measurements) [22]. Similarly in computer vision, multimedia data-
set can simultaneously contain recorded audio as well as video that can come from
different sources or cameras placed in different positions [24, 25, 38]. Also in case
of natural language processing, data can be collected as content of the documents,
meta information of the documents (e.g., title, author and journal) and possibly the
co-citation network graph for scientific document management [11, 33].
The feature variables collected from multiple sources for each data-point there-
fore exhibit heterogeneous properties and hence can be naturally partitioned into
groups. Each group is referred as a particular view. Each view contains one
type of information which is possibly arriving from a single source and is more
homogeneous. Figure 1.1 shows a pictorial presentation of multi-view data captured
through multiple sources.
Multi-View Learning Existing algorithms for multi-view learning can be classi-
fied into three groups based on their main principle: (1) co-training, (2) subspace
learning and (3) multiple kernel learning. Co-training [7, 21] is one of the earliest
schemes for multi-view learning. The main aim of co-training is to maximize the
mutual agreement between two distinct views of the data. The success of co-
training lies in the assumption of data sufficiency in each view and information
consistency among all views [39]. Whereas subspace learning-based approaches
such as canonical correlation analysis (CCA) [15, 19] and factor analysis (FA) [8]
aim to obtain a latent subspace shared by multiple views. Here the assumption is
that, data in different views are generated from a global latent subspace along with
some view-specific local factors. The task of learning the common latent subspace
is based on observed data in all views. Thus, these methods need complete data
availability in each view. For non-linear modeling of multi-view data the most
popular method is the multiple kernel learning (MKL) [36], which was originally
developed to control the search space capacity of possible kernel matrices to achieve
good generalization [17]. Currently, MKL has been widely applied to problems
involving multi-view data [10, 21, 37]. MKL defines multiple kernels corresponding
to different views and combines them either linearly or non-linearly. However, for
defining a kernel it needs complete data availability in each view.
Although existing multi-view learning algorithms are effective and have shown
promising performance in different applications, they share a serious limitation. The
success of most of the existing multi-view learning algorithms mainly relies on the
assumption of completeness of data-sets, i.e., any multi-view example must always
be fully observed in all views.
Missing Values Due to various failures or faults in collecting and pre-processing
the data in different views, existing models are more likely to be challenged with
missing information. Missing data-points, whether be partially or fully missing in
1 Multi-View Data Completion 3

Fig. 1.1 Multi-view data when data about one data-point is captured by M different cameras from
different positions resorting to different views

a view, exists in a wide range of fields such as social sciences, computer vision,
biological systems and remote sensing (as in Fig. 1.2).
Missing View Often, information for a data-point corresponding to a view can
be fully missing. As shown in Fig. 1.2 (top), for each data-point, the full picture
from a view is totally missing for data-points while other pictures corresponding
4 S. Bhadra

to other views of that data-point are fully available. This kind of missing data-
points will be denoted henceforth as data-point with missing view. For example,
in remote sensing, some sensors can go off for some time, leaving gaps in the data.
A second example is that when integrating legacy data-sets, some views may not
be available for some data-points, because integration needs were not considered
when collection and storage happened. Again, gene expression value may have been
measured for some of the biological data-points, but not for others. On the other
hand, some measurements may be too expensive to repeat for all data-points; for
example, patient’s genotype may be measured only if a particular condition holds.
All these examples introduce data-point with missing view, i.e., all features of a view
for a data-point can be missing simultaneously.
Incomplete View On the other hand, it can happen that, we have partial information
in a view. As shown in Fig. 1.2 (bottom), for each data-point the pictures from all
views are available but some of them are corrupted or partially missing. This kind
of data-points with missing features will be denoted henceforth as data-point with
incomplete view. For example, some unwanted object might have come in the frame
while capturing image in a camera. Then the information corresponding to the main
object becomes missing for the position of the unwanted object. A second example
can be a feedback data containing various codes indicating lack of information like
Don’t know, No comments, etc. [26]. All these examples introduce data-point with
incomplete view, i.e., some features of a view for a data-point can be missing.
Neglecting views with missing entries definitely restricts models to exploit useful
information collected from that view through the data-points for which feature
values corresponding to that view are known. Similarly, neglecting data-points
with missing entries reduce the number of training data-points for models. Hence,
accurate imputation of missing values is required for better learning.
Although multi-view learning in the presence of missing data has drawn a great
amount of attention in the recent past and there are quite a lot research papers on
multi-view data completion, but there is no comprehensive introduction and review
of possible current research approaches on multi-view data completion. Figure 1.3
describes a simple taxonomy of the current approaches. In the worst case, missing
view and incomplete view can occur simultaneously. In this chapter we will mainly
discuss existing methods which exclusively solve the missing values problem in
multi-view data-set. These methods were proposed mostly to deal with missing
views, i.e., row-wise or/and column-wise missing values (top part of Fig. 1.2) in
the data matrices and can also be applied for incomplete view case.
Multi-View Data and Kernel Completion In order to apply most of the multi-
view learning approaches to data-sets with missing views, one needs to pre-process
the data-set first for filling in the missing information. Usually, different views
contain semantically related content for a given data-point. Often different views
share common characteristics and they can also have view-specific characteristics.
This property can be useful to impute the missing views in multi-view data-sets.
The conventional matrix completion algorithms [9, 20] solve the problem of
recovery of a low-rank matrix in the situation when most of its entries are missing.
Fig. 1.2 This figure gives a pictorial presentation of two kinds of missing data in multi-view data-
sets. The top figure shows view-wise missing data where a data-point is missing in a view and
hence the corresponding complete feature row is also missing in data-set. The bottom figure shows
that data-points are partially missing in a view and hence create an incomplete view situation in
data matrix. But both type of missing values create a missing row and a missing column in the
corresponding kernel matrix
6 S. Bhadra

Fig. 1.3 Taxonomy of works

in the field of multi-view
completion

Missing features in individual views can be imputed using these techniques in

case of multi-view data-set with incomplete view. But such techniques are meant
for imputing missing features for a single-view data-set and unable to exploit the
information present in complementary views to complete the missing data in case
of multi-view data-set. Hence these methods are out of scope of this chapter.
There are two major approaches to handle missing data problems specifically in
multi-view data-set: (1) imputing feature values or data matrices, and (2) imputing
kernel matrices. Multi-view data completion methods often complete the data
matrices by using within-view relationships and between-view relationship among
data-points.
Ashraphijuo et al. [4] and Lian et al. [23] present a method to impute missing
data in feature matrices using matrix factorization-based technique. They learn a
factor matrix common for all views and view-specific factor matrix for each view
along with factor’s weight loadings. This is applicable for multi-view data-set with
both incomplete view and missing view.
Multiple kernel learning (MKL) framework [17] is another efficient approach to
accumulate information from multiple sources where no explicit feature information
is required but multi-view data is accessed through multiple kernels defined on them.
In MKL, the kernel matrices built on features from individual views are combined
for better learning. MKL needs to know full kernel matrices for each view. Hence it
is enough to impute missing values in kernel matrices instead of imputing missing
values in feature space. For both kind of missing information, i.e., incomplete views
and missing views, the missing values for a data-point in a view result into a missing
row and a missing column in the kernel matrix of that view.
Existing methods for kernel completion use various assumptions, for example,
the presence of a complete kernel matrix in at least one view [30, 31], or all
kernel matrices are allowed to be incomplete [6, 23], kernels have homogeneous
[27, 30, 31] or heterogeneous eigen structures [6, 23], linear [23] or non-linear
approximation [6]. Based on different assumptions, various methods have been
developed for kernel completion. In this chapter we will discuss the following
methods: (1) matrix factorization-based methods [6], (2) expectation maximization-
based methods [27] and (3) generative model-based methods [23]. Moreover, Lian
et al. [23] complete the missing values in data and kernel simultaneously.
Similar to multi-view data completion methods, the multi-view kernel comple-
tion methods also complete the missing rows and columns in kernel matrices by
1 Multi-View Data Completion 7

learning some within-view and between-view relationships among kernels. Bhadra

et al. [6] proposed to learn a relationship among kernel values directly while [31]
proposed to learn relationship among eigen structures, and Lian et al. [23] and
Bhadra et al. [6] proposed to learn relationship through global component factor
matrices for all views. In addition to between-view relationship, Lian et al. [23] and
Bhadra et al. [6] learn low rank approximation of view-specific kernel matrices for
better generalization which can be considered as learning within-view relationships.
In this chapter we will discuss both imputation of missing values in data matrices
and kernel matrices. There are methods for multi-view learning with incomplete
views which learn from partially observed data without data completion. One
such example is Williams and Carin [34] which does not complete the individual
incomplete kernel matrices but complete only the aggregated kernel matrices when
all kernels are Gaussian. Those kind of work is out of scope of this chapter.
The remainder of the chapter is organized as follows. In Sect. 1.2, we formulate
the problems for missing view and incomplete view setup for both feature matrices
and kernel matrices. We also discuss the challenges in missing view problems
in the multi-view learning. Section 1.3 presents the state-of-the-art methods for
completing missing data in multi-view learning and Sect. 1.4 presents the state-of-
the-art multi-view kernel completion methods.

1.2 Incomplete Views vs Missing Views

We assume N data-points X = {x1 , . . . , xN } from a multi-view input space X =

X (1) × · · · × X (M) , where X (m) is the input space generating the mth view. We
denote by

(m) (m)
X(m) = {x1 , . . . , xN }, ∀ m = 1, . . . , M,

(m)
the set of data-points for the mth view, where xi ∈ X (m) is the ith observation in
the mth view and X (m) is the input space.
Incomplete Views In incomplete view setup some feature values are missing at
(m)
random in a view. For the tth data-point xt of the mth view, there can be a group
(m)
of features, indexed by J , are missing, i.e., xt,J are missing. Figure 1.4a shows a
pictorial presentation of missing features in incomplete view setup in data matrices
of M views, where the J feature set in tth row in X(1) is missing.
Missing Views In missing view setup, a data-point is completely present or
completely missing in a view. Hence in this case only a subset of data-points
is completely observed in each view, and correspondingly, a subset of views is
observed for each data-point. Let IN = [1, . . . , N] be the set of indices of all
data-points and I (m) be the set of indices of all available data-points in the mth
(m)
view. Hence for each view, only a data sub-matrix (XI (m) ) corresponding to the rows
indexed by I (m) is observed and other rows are missing, i.e., X(m)
I /I (m)
are missing.
N
8 S. Bhadra

Fig. 1.4 We assume N data-points with M views. (a) For incomplete view setup few feature
values of some data-points are missing from each individual view and (b) for missing view setup
few data-points are completely missing in some views and consequently corresponding rows are
missing (denoted by ‘?’) in data matrices (X(m) )

Fig. 1.5 We assume N data-points with M views, with a few data-points missing from each
individual view, and consequently corresponding rows and columns are missing (denoted by ‘?’)
in kernel matrices (K(m) )

Figure 1.4b shows a pictorial presentation of missing data-points in missing view

setup in data matrix of M views, where the tth row in X(1) is missing.
Missing Rows and Columns in Kernel In this situation a kernel matrix has a
complete missing row and a complete missing corresponding column.
1 Multi-View Data Completion 9

Considering an implicit mapping of the observations of the mth view to an inner

product space F (m) via a mapping φ (m) : X (m) → F (m) , and following the
usual recipe for kernel methods [5], we specify the kernel as the inner product
in F (m) . The kernel value between the ith and the j th data-points is defined as
kij(m) = φi(m) , φj(m) , where φi(m) = φ (m) (x(m) (m)
i ) and kij is an element of K
(m) , the

kernel Gram matrix for the set X(m) .

(m) (m)
To define an element kij in general, one needs complete feature vectors xi and
x(m) (m)
j and hence kij has to be considered as missing kernel values if any of the data-
(m) (m)
point xi or xj is partially or completely missing. If the ith data-point has any
missing values in the mth view (in both incomplete view and missing view cases),
the ith row and the ith column of the kernel matrix K(m) cannot be calculated and
will be missing. Hence while dealing with multi-view kernel completion all existing
methods consider missing view setup and hence assume that a subset of data-points
is observed in each view, and correspondingly, a subset of views is observed for
each data-point. Let IN = [1, . . . , N ] be the set of indices of all data-points and
I (m) be the set of indices of all available data-points in the mth view. Hence for each
(m)
view, only a kernel sub-matrix (KI (m) I (m) ) corresponding to the rows and columns
indexed by I (m) is observed. Hence K(m) has one observed block and 3 unobserved
block as follows:
⎡ ⎤
(m) (m)
KI (m) ,I (m) KI (m) ,I /I (m) = ?
K(m) = ⎣ (m)T N ⎦.
KI (m) ,I /I (m) = ? K(m)
I /I (m) ,I /I (m)
=?
N N N

Figure 1.5 shows a pictorial presentation of kernel matrices of M views, where the
tth row and column of K(1) is missing.

1.3 Multi-View Data Completion

In this section we will review some recent methods for imputing missing data in
missing-view setting. By imputing data we will restrict ourselves to imputing feature
vectors. We will discuss methods for kernel completion in the next section. Recall
(m)
that in case of missing view data-sets, only a data sub-matrix (XI (m) ) corresponding
to the rows indexed by I (m) is observed and other rows (X(m)
IN /I (m)
) are completely
missing (Fig. 1.4b).
There are three major approaches in missing-view setting:
– CCA-based robust multi-view data completion (CCAbased ) by Subramanya et al.
[28, 29]
– Multi-view learning with features and similarities (MLFS) by Lian et al. [23]
– Rank constrained multi-view data completion (LowRank) by Ashraphijuo et al.
[4]
10 S. Bhadra

Fig. 1.6 We assume N data-points with M views, with a few data-points missing from each
individual view, and consequently corresponding rows and columns are missing (denoted by ‘?’)
in data matrices (X(m) ). Existing methods predict the missing rows/columns (e.g., the tth column
in views 1) with the help of other data-points of the same view (within-view factorization, blue
arrows) and the corresponding data-point in other views (relationship among latent factors, green
arrows)

Note that, although the above-mentioned feature completion methods have been
originally proposed for imputing data in case of missing views, they can be applied
to impute partially missing data-points for a view too. All the state-of-the-art multi-
view data completion methods predict missing feature vectors x̂(m) t by assuming
the features for each data-point across different views are generated from a total
or partially shared latent factors along with some view-specific factor weights.
LowRank [4] assumes that the number of shared and non-shared latent factors for
various views is known a priori. On the other hand, CCAbased and MLFS [23, 29]
do not make such assumption and learn the number of latent factors. MLFS was
originally proposed for completing data when number of views is greater than two.
On the other hand, both CCAbased and LowRank are mainly applicable for two-
view data-set but can be extended easily for more than two views.
(m)
All three models, while predicting unknown x̂t , learn the within-view factor-
ization and the relationships among the latent factors across views (Fig. 1.6).

1.3.1 Within-View Factorization

A multi-view data consisting of N data-points and M views can be denoted by an

N ×Dm matrix as X(m) ∈ RN ×Dm . Let the latent factors be denoted by a matrix V =
{V(m) ∈ RN ×Dm }M
m=1 (where rm denotes the number of latent factors of data in the
1 Multi-View Data Completion 11

mth view) with view-specific factor loading matrices W = {W(m) ∈ R rm ×Dm }M

m=1 .
(m)
Hence the predicted data matrix X̂ can be factorized as:

X̂(m) = V(m) W(m) . (1.1)

All the existing methods add an additional constraint on X̂(m) so that the
reconstructed feature values closely approximate the known or observed parts of
the data. To this end, a loss function measuring the within-view approximation error
for each view is defined as:

Loss(m) (m) (m) 2 (m)

data = X̂I (m) − XI (m) 2 = VI (m) W
(m)
− X(m) 2 ,
I (m) 2
(1.2)

(m)
where VI (m) denotes the rows of V(m) indexed by I (m) .

1.3.2 Between-View Relation on Latent Factors

When data for a data-point is missing in a view, there is not enough information
within that view to impute the features for that data-point. One needs to use other
information sources, i.e., the other views where the corresponding features are
known. All the existing methods use some kinds of inter-view or between-view
relationships on latent factors in order to achieve this. We discuss the different types
of relationships below.
Totally Shared Latent Factors According to the most simple setting, CCAbased
[28, 29] assumes that latent factors for all the views are exactly same, i.e., there is a
common latent factor V such that

V(m) = V.

On top of it, CCAbased does not predict a new feature vector but only selects the
nearest neighbour of it from the set of available feature vectors in a view.
Totally Shared and View-Specific Latent Factors Realize that different views
may also capture different view-specific aspects of a data-point along with capturing
the common aspects that are present in all views. Therefore Ashraphijuo et al. [4]
assume that each V(m) contains some view-specific factors along with some global
common factors V and solves the following problem:


M
(m)
arg min [VV(m) ]I (m) W(m) − XI (m) 22 . (1.3)
V,V(m) ,W(m) m=1
12 S. Bhadra

They also assume that the number of common factors and also the number of view-
specific factors present in each view is known or given.
Partially Shared Latent Factors MLFS [23] considers that along with the totally
shared factors, a subset of views may have a set of additional shared factors.
According to Fig. 1.7, the factor V (13) is shared by the first and the third views.
To impose this structure on the learned latent factor matrices, MLFS learns a global
latent factor matrix V ∈ RN ×r and the associated sparse factor loading matrices
W = {W(m) ∈ R r×Dm }M m=1 . In addition to that, a structured-sparsity constraint
is imposed in the factor loading matrices {W(m) }M m=1 of all views, such that some
of the rows in these matrices share the same support for non-zero entries, whereas
some rows are non-zero only for a subset of these matrices. Figure 1.7 summarizes
their basic framework. As a result, it solves the following optimization problem:

Fig. 1.7 Various kind of sharing of latent factors in existing models are depicted in this figure.
CCAbased [29] assumes all latent factors to be shared by all views, LowRank [4] assumes that
along with shared factor each view also has some view-specific factors (V (1) factor is only for the
first view) and MLFS [23] in addition to that assumes some factors are also partially shared by
only a subset of views (V (13) factor is shared by the first and the third views but not by the second
view)
1 Multi-View Data Completion 13


M
(m)
arg min VI (m) W(m) − XI (m) 22
V,W(m) m=1

with row sparsity constraints on W(m) , ∀m = 1, . . . , M. (1.4)

MLFS uses generative model to learn the latent factors and also to predict the
missing features. It uses group-wise automatic relevance determination [32] as the
sparsity inducing prior on {W(m) }M
m=1 , which also helps in inferring r by shrinking
the unnecessary rows in W to near zero.

1.4 Multi-View Kernel Completion

Multi-view kernel completion techniques predict a complete positive semi-definite

(PSD) kernel matrix (K̂(m) ∈ RN ×N ) corresponding to each view where some
rows and columns of the given kernel (K(m) ∈ RN ×N ) are fully missing (Fig. 1.5).
The crucial task is to predict the missing (tth) rows and columns of K̂(m) , for all
(m)
t ∈ Ih = {IN /I (m) }. Existing methods for kernel completion have addressed
completion of multi-view kernel matrices with various assumptions. In this section,
we will discuss details of the following state-of-the-art methods for multi-view
kernel completion.
– Multi-view learning with features and similarities (MLFS) by Lian et al. [23]
– Multi-view kernel completion (MKCsdp , MKCapp , MKCembd(ht) , MKCembd(hm) )
by Bhadra et al. [6]
– The expectation maximization-based method (EMbased ) by Tsuda et al. [31] and
Shao et al. [27]
These methods have various assumptions, i.e., any auxiliary full kernel matrix is
available or not; all kernel functions are linear or non-linear; kernel functions in
different views are homogeneous or heterogeneous in nature, etc. The applica-
tion/assumption of these methods is described in Table 1.1.
Almost all existing state-of-the-art approaches for predicting K̂(m) are based on
learning both between-view and within-view relationships among the kernel values
(m)
(Fig. 1.8). The sub-matrix K̂I (m) I (m) should be approximately equal to the observed
(m)
matrix KI (m) I (m) . However, in some approaches such as [6, 23, 31], approximation
quality of the two parts of the kernel matrices is traded.

1.4.1 Within-View Kernel Relationships

For being a valid kernel the predicted K̂(m) needs to be a positive semi-definite
matrix for each m. The expectation maximization-based method (abbreviated
14 S. Bhadra

Table 1.1 Table shows the assumption made by the state-of-the-art multi-view kernel completion
methods and the algorithm used by them
Methods
MKCsdp MKCapp EMbased MKCembd(hm) MLFS MKCembd(ht)
[6] [6] [27, 31] [6] [23] [6]
Properties of missing values
Some rows and columns of a kernel Yes Yes Yes Yes Yes Yes
matrix are missing entirely
Views are homogeneous Yes Yes Yes Yes Yes Yes
No need of at least one complete Yes Yes x Yes Yes Yes
kernel
Kernels are spectral variant of each x x Yes Yes Yes Yes
other
Views are heterogeneous x x x x Yes Yes
Highly non-linear kernel Yes x x x x Yes
Algorithm
Semi-definite programming Yes x x x x x
Non-convex optimization x Yes x Yes x Yes
Generative model x x x x Yes x
Expectation maximization x x Yes x x x

Fig. 1.8 We assume N data-points with M views, where a few data-points missing from each
individual view, and consequently corresponding rows and columns are missing (denoted by ‘?’)
in kernel matrices (K(m) ). The proposed methods predict the missing kernel rows/columns (e.g.,
the tth column in views 1 and m) with the help of other data-points of the same view (within-
view relationship, blue arrows) and the corresponding data-point in other views (between-view
relationship, green arrows)

henceforth as EMbased ) [31] does not assume any other relationship among data-
points but only guarantees the PSD property of individual kernel matrix. In
optimization problem, it requires explicit positive semi-definiteness constraints:

K̂(m)  0, (1.5)
1 Multi-View Data Completion 15

Fig. 1.9 Within-view and between-view relationships in kernel values assumed in different
models

to guarantee the PSD property of individual kernel matrix. MKC [6] has also
considered no other but only explicit PSD constraint in one of its formulation
denoted as MKCsdp .
But with missing rows and columns these constraints leave the model too
flexible. Hence some methods also predict missing rows/columns by learning some
other relationship k̂ = g(K) among data-points in the same view. Relationship
among data-points of same view is known as within-view relationship. For within-
view relationship learning, some methods rely on the concept of low rank kernel
approximation [23], while others use the concept of local linear embedding [6].
Figure 1.9 pictorially presents the within-view relationship considered in various
proposed methods.
Multi-view learning with features and similarities [23] assume kernels are linear
in nature and hence can have low rank approximation, i.e.,
T T
K̂(m) = g(K(m) ) = U(m) U(m) = AT W(m) W(m) A, (1.6)
16 S. Bhadra

where U(m) ∈ R N ×r with r << N . A and W(m) are denoted, respectively, as

global latent factor matrices and the associated factor loading matrices for the mth
view. A by-product of this approximation is that in optimization, PSD property
is automatically guaranteed without inserting explicit positive semi-definiteness
constraints.
Recently proposed multi-view kernel completion [6] considered retaining the
non-linear property of a kernel by assuming local embedding in feature space
instead of kernel space. It assumes that in each view there exists a sparse embedding
in F , given by a small set of data-points B (m) ⊂ I (m) , called a basis set, that is able
to represent all possible feature representations in that particular view. Hence MKC
reconstructs the feature map of tth data-point φt(m) by a sparse linear combination
of a subset of observed data-points

(m)
 (m) (m)
φ̂t = ait φi , (1.7)
i∈B (m)

where ait ∈ R is the reconstruction weight of the ith feature representation for the
tth data-point. Hence, approximated kernel values can be expressed as

(m) (m) (m)

 (m) (M) (m) (m)
k̂tt = φ̂t , φ̂t = ait aj t φi , φj . (1.8)
i,j ∈I (m)

The above formulation retains the non-linearity of the feature map φ and the
 N
corresponding kernel. A = aij i,j =1 is the matrix of re-constructing weights.
Further, A(m)
B (m)
is the sub-matrix of A containing the rows indexed by B (m) , the
set of basis vectors in kernel feature space in view. Thus the reconstructed kernel
matrix K̂ can be written as
(m)T (m)
K̂(m) = g(K(m) ) = AB (m) KB (m) AB (m) . (1.9)

Note that K̂ is positive semi-definite when KB (m) is positive semi-definite. Thus, a

by-product of this approximation is that in optimization PSD property is automati-
cally guaranteed without inserting explicit positive semi-definiteness constraints.
Intuitively, the reconstruction weights are used to extend the known part of the
kernel to the unknown part, in other words, the unknown part is assumed to reside
within the span of the known part.
To select a sparse set of reconstruction weights, Bhadra et al. [6] regularizes the
reconstruction weights by the 2,1 norm [3] of the reconstruction weight matrix,

 
A(m) 2,1 = (aij )2 . (1.10)
i j
1 Multi-View Data Completion 17

Finally, for the observed part of the kernel, all existing methods add the additional
constraint on K̂(m) that the reconstructed kernel values closely approximate the
known or observed values. To this end, a loss function measuring the within-view
approximation error for each view is defined as

(m) (m)
Losswithin = g(K(m) )I (m) I (m) − KI (m) I (m) 22 . (1.11)

Hence, for individual views the observed part of a kernel is approximated by

g(K(m) )I (m) I (m) by optimizing the parameters (A, W) (Eqs. (1.10) and (1.11)) in

arg min g(K(m) )I (m) I (m) − KI (m) I (m) 22 + λA(m) 2,1 , (1.12)
A,W

where λ is a user-defined hyper-parameter which indicates the weight of regulariza-

tion.
Without the 2,1 regularization, the above approximation loss could be trivially
optimized by choosing A(m) as the identity matrix. The 2,1 regularization will have
the effect of zeroing out some of the diagonal values and introducing non-zeroes to
(m)
the sub-matrix AB , corresponding to the rows indexed by B, where B = {i|aii =
0}.
MKC [6] also claimed that Eq. (1.12) corresponds to a generalized form of the
Nyström method [35] which is a sparse kernel approximation method that has been
successfully applied to efficient kernel learning. Nyström method finds a small
set of vectors (not necessarily linearly independent) spanning the kernel, whereas
MKC method searches for linearly independent basis vectors and optimizes the
reconstruction weights for the data-points.

1.4.2 Between-View Kernel Relationships

For a completely missing row or column of a kernel matrix, there is not enough
information available for completing it within the same view, and hence the
completion needs to be based on other information sources, i.e., the other views
where the corresponding kernel parts are known. All existing methods learn some
kind of inter-view or between-view relationships f (·) to predict missing values in a
view with help of information present in other views, i.e., K̂(m) ≈ f (K̂[1,...,M]/m ).
Various existing methods learn different kind of inter-view or between-view
relationships among kernels to predict missing values depending upon various
assumptions regarding the available information. This gives us between-view loss
as:

Lossbetween (K̂, f (K̂[1,...,M]/m )) = K̂(m) − f (K̂[1,...,M]/m )22 .

(m)
(1.13)
18 S. Bhadra

The MKCapp [6] learns linear relationships among values of each element of ker-
nel matrices (kij ) based on learning a convex combination of the kernels, extending
the multiple kernel learning [2, 12] techniques to kernel completion. The second
technique discussed in EMbased [27, 31] learns relationship among eigen structure
of kernels of various views. In case kernels of different views are heterogeneous
and do not share the similar eigen-spectra, the third technique discussed in MLFS
[23] and MKC [6] learns the linear relationship among the latent low-dimensional
embedding of different views based on learning reconstruction weights so that
they share information among all views [23] or selected set of similar views [6].
Figure 1.9 presents summary of the between-view relationship considered in various
proposed method.
Between-View Learning of Kernel Values In multi-view kernel completion
perhaps the simplest situation arises when the kernels of the different views are
similar. In that case to learn between-view relationships MKCsdp and MKCapp
[6] express the individual kernel matrix corresponding to each view as a convex
combination of kernel matrices of the other views. Hence the proposed model learns
kernel weights S = (sml )Mm,l=1 between all possible pairs of kernels (m, l) such that


M
K̂(m) ≈ f (·) = sml K̂(l) , (1.14)
l=1,l =m

where the kernel weights are confined to a convex combination as:

⎧ ⎫
⎨ 
M ⎬
S = S|sml ≥ 0, sml =1 . (1.15)
⎩ ⎭
l=1,l =m

The kernel weights then can flexibly pick up a subset of relevant views to the current
view m.
Between-View Learning Assuming Rotated Eigen Structure of a Known Kernel
In practical applications, the kernels arising in a multi-view setup might be very
heterogeneous in their distribution. In such cases, it might not be realistic to find a
convex combination of other kernels that are closely similar to the kernel of a given
view. In particular, when the K̂m is assumed to be approximated by the parametric
model which is derived from the spectral variants of K(O) , which is an N × N fully
observed auxiliary kernel matrix [27, 31] the missing values in K̂m is completed by
expressing the kernel matrix as the spectral variants of K(O) , i.e.,


N
(m)
K̂(m) ≈ f (·) = si Mi , where Mi = vi viT , (1.16)
i=1
1 Multi-View Data Completion 19

(m)
where si ≥ 0 and the eigen decomposition of K(O) is denoted as K(O) =
N T
i=1 λi vi vi with λi ’s and vi ’s as eigen values and eigen vectors, respectively. Here
all incomplete kernels considered to have eigen vectors same as that of K(O) , but
(m)
their own set of eigen values (si ’s).
Between-View Learning of Reconstruction Weights When the eigen vectors of
a kernel matrix are rotated from the eigen vectors of kernels of other views, Bhadra
et al. [6] propose an alternative approach MKCembd(hm) , where instead of the
kernel values or eigen vectors, they assume that the basis sets and the reconstruction
weights (Eq. (1.9)) have between-view dependencies as follows:

A(1) = . . . = A(M) . (1.17)

The reconstructed kernel is thus given by

(m)
K̂(m) ≈ f (·) = AT KB (m) A. (1.18)

However, assuming that kernel functions in all the views have similar eigen-
vectors is also unrealistic for many real-world data-sets with heterogeneous sources
and kernels applied to them. On the contrary, it is quite possible that only for
a subset of views the eigen-vectors of approximated kernel are linearly related.
MKCembd(ht) [6] thus allow the views to have different reconstruction weights, but
a parametrized relationship among them, i.e., the reconstruction weights in a view
can be approximated by a convex combination of the reconstruction weights of other
views:


M
A (m)
≈ sml A(l) , (1.19)
l=1,l =m

where the coefficients sml are defined as in Eq. (1.14).

The reconstructed kernel is thus given by
⎛ ⎞ ⎛ ⎞

M
(l)T

M
K̂(m) ≈ f (·) = ⎝ sml AB (m) ⎠ KB (m) ⎝ sml AB (m) ⎠ .
(m) (l)
(1.20)
l=1,l =m l=1,l =m

This also allows the model to find an appropriate set of related kernels from the
set of available incomplete kernels, for each missing entry.
20 S. Bhadra

1.4.3 Optimization Methods

Almost all state-of-the-art multi-view kernel completion methods optimize their

parameters by minimizing both within-view and between-view loss, i.e., using
Eqs. (1.11) and (1.13)

M
(m) [1,...,M]/m (m)
arg min Lossbetween (K̂, f (K̂ )) + λ2 Losswithin (1.21)
A,W m=1

Like their assumptions and formulation various methods also use various opti-
mization techniques to solve their optimization formulation.
– In multi-view setting, MLFS [23] proposed a generative model-based method
which approximates the similarity matrix for each view as a linear kernel in some
low-dimensional space.
– MKC [6] solves their non-convex problem with the help of solving sequence of
convex problems using proximal gradient method.
– On the other hand, EMbased [27, 31] have proposed an expectation maximization-
based method to complete an incomplete kernel matrix for a view, with the help
of a complete kernel matrix from another view.

1.5 Discussion

This section discusses the applicability of various methods depending upon char-
acteristic of data-set. The applicability of methods on data or kernel completion
depends upon the number and type of missing features and also the degree
of heterogeneity in information of different views. Degree of heterogeneity of
information can be captured in eigen-spectrum of the kernel matrix [6].
Multi-View Data Completion The imputation in feature matrix is more appro-
priate if there are less number of missing features for each data-point (incomplete
view) or less number of features per view (missing view).
The very first multi-view matrix completion method, CCAbased uses a heuristics-
based method for predicting missing data-points. Assumption of having only
common latent factors is also not true for real-world data. This technique is
applicable if views are homogeneous. When eigen-spectra of kernel matrices (in
this case linear kernel) for all views are similar (the left most plot of Fig. 1.10) then
views are considered to be homogeneous and hence can be explained with shared
factors alone. In real data, each view mostly has some view-specific complementary
information. Hence each view cannot be explained only by shared factors, otherwise
prediction accuracy of CCAbased is not expected to be good enough in real-world
data.
Another random document with
no related content on Scribd:
3844 Howe Samuel 1K July
W 23
Aug
7186 Hoyt A D 3K
29
17 July
3237 Hudson W
E 12
31 Sept
8797 Hughes Wm
K 15
Sept
9652 Humphrey —— C 3L
24
July
3484 Hunkey E B 1L
17
Aug
4703 Henly D 8G
4
16 Aug
5355 Ingols L 64
H 11
Sept
9389 Ingerson P 7 I
20
17 Oct
11489 Jackson A J
I 26
Oct
10619 Jackson R 7B
10
Oct
10710 Jackson R W 7D
11
19 Feb
12602 Jerdan J 65
F 6
Aug
7385 Johnson B 7K 64
30
19 Aug
5849 Jones Wm
E 16
Oct
10243 Jory G F 8F
3
11586 Kellar J 19 Oct
 J 28
11 Sept
8237 Kelley L
D 9
17 July
3313 Kennedy W
G 14
Aug
6169 Kilpatrick C 3C
19
Aug
5366 Land C 6 I
11
17 Sept
8350 Lamber W
K 10
19 Nov
11707 Levitt H
A 1
16 Sept
7967 Lincoln A
I 6
Oct
10931 Littlefield C Cav 1F
14
Aug
6340 Lord Geo H 3B
21
13 Aug
5549 Ludovice F
F 13
April
490 Lowell B 4G
12
Sept
9426 Macon L 8A
21
16 April
709 Malcolm H M
A 24
Aug
6606 Marshall B F 1H
23
19 Nov
12122 Maston A
D 22
32 Oct
10392 Mathews James
F 14
12011 Maxwell J 8E Nov
14
July
3679 McFarland G 3G
21
Sept
9538 McGinley J 7A
22
June
2200 McKinney G 3 I
19
Nov
12084 McFarland E S 8 I
18
July
4391 Metcalf Oliver 8H
31
McFarland W, 19 Mch
12768
Cor K 13
7 Aug
5200 Melgar J
- 10
Aug
5614 Messer C R 7F
14
Sept
9399 Miller C J Cav 1B
21
June
2002 Miller J O 2D
15
1 Sept
7573 Mills M
- 2
Moore Charles July
2808 8B
W 3
18 Oct
11042 Moore G
D 17
Aug
7273 Moore J D Cav 1B
30
Aug
6940 Moore W C 7A
26
8118 Moyes F 32 Sept
 F 8
Aug
7046 Newton C 9K
27
May
1507 Nickerson D 4F
31
Sept
8020 Nolton H 7B
6
16 June
2131 O’Brien W
A 18
19 Aug
6325 Opease S
- 21
8 Mch
143 Osborn A J
- 24
10 Nov
10866 Owens O H
- 6
July
3710 Parker A Cav 1E
21
Parsons James 16 Sept
7979
W D 6
14 Sept
9362 Patrick F
F 20
Peabody F S, June
2272 5 I
S’t 20
Jan
12543 Pequette P 4G 65
28
May
1486 Perkins D Cav 1 I 64
31
Aug
5197 Perkins T 1H
10
Aug
6911 Peters H 4E
26
Nov
12056 Phillbrook F Art 1A
17
2064 Phelps W H Cav 1H June
16
July
3436 Pinkham U W Art IA
17
1 May
1361 Pottle A E Cav
- 25
Aug
5698 Pratt A M “ 1L
15
16 Sept
8441 Pulerman G
D 11
19 Jan
12410 Prescott C 65
H 7
31 Sept
7785 Richardson C 64
L 4
Aug
6762 Richardson J K 8G
24
Richardson W, Oct
10465 C 1B 64
Cor 7
Aug
5522 Ricker Wm, Cor C 1D
13
Sept
8480 Ridlon N 7D
11
May
900 Riseck R 3 I
5
19 July
3921 Roberts H
K 25
Aug
5236 Rowe L 1A
10
Mch
166 Rosmer Frank 4C
26
Aug
5796 Ruet H 2H
15
8557 Russell G A Cav 1E Sept
 12
Aug
5450 Sampson E 1F
12
Aug
4532 Sawyer Enos Art 1H
2
31 July
3182 Sawyer John
K 11
Oct
11462 Shorey S Cav 1K
20
June
2243 Simmons G F 6K
20
July
3159 Smith W 9K
11
July
3331 Smith W A 6F
14
June
1782 Snowdale F 4C
10
19 Sept
9974 Snower S C
A 28
36 June
1998 Springer H W
A 15
20 Aug
4596 Steward G
H 3
19 Oct
11562 St Peter F
F 27
19 Aug
7001 Swaney P
F 27
Mch
199 Swan H B, Cor 3F
28
June
1936 Swan F 3F
14
Sept
8682 Thompson F 9E
13
10455 Thompson John 3E Oct
7
April
621 Thorn E 9 I
19
Oct
10928 Toothache J 7G
14
May
1106 Turner C C 4E
15
32 Aug
5090 Tufts J
C 8
Nov
11875 Taylor G 9C
16
32 Dec
12322 Tuttle D L
F 20
32 Nov
12196 Tuttle L S, Cor
F 30
Thorndie W B, 19 Mch
12706 65
Cor I 2
32 Aug
6245 Valley F 64
K 19
32 July
3335 Venill C
G 15
Aug
7226 Walker A B, Cor 1K
29
July
3894 Walker M C 5 I
24
Sept
7722 Wall A Cav 1K
4
20 Aug
5942 Walsh Thomas
H 17
Aug
6750 Watson B 7K
24
10558 Webber Oliver 3A Oct
 9
Whiteman A M, Aug
4559 5 I
Cor 2
June
1648 Whitcomb T O 4F
5
32 Aug
6251 Whittier J K P
C 19
20 Oct
10445 Willard W
B 7
Sept
7711 Williams C 6G
3
32 Aug
6900 Wilson George
C 26
16 July
3639 Wilson G W
H 20
19 July
3132 Willey D H
E 10
July
3860 Winslow E I 4B
24
Aug
5512 Winslow N L 4K
13
32 Aug
6372 Wyman A
C 21
16 June
2095 Wyman J
A 17
Jan
12470 Wyer R 3K 65
16
Nov
12043 Wright C 1G 64
16
Mar
178 Young E W, S’t 3H
26
Aug
6369 Young J 3H
21
8140 Young J, Cor 8 I Sept
8
Total
233.

MARYLAND.
1 May
850 Allen W H 64
H 3
2 May
1028 Anderson Wm
C 11
1 May
1379 Aikens A Cav
I 26
6 May
1928 Adams Jas T
H 14
2 Oct
10288 Abbott D E
D 4
1 Dec
2325 Archer H 64
I 24
8 Mch
112 Babb Samuel
I 23
2 April
288 Berlin Jas Cav
F 1
2 April
472 Beltz W W
H 9
1 May
1086 Bowers A
I 14
Brown 2 May
1455
Augustus G 29
2 May
1487 Braddock Wm
D 30
1549 Buck H Cav 1 June
 B 2
9 June
1644 Buckley Geo
B 5
1 June
2404 Bennett C B
D 24
2 July
3268 Brant D B
H 13
1 Aug
4602 Betson James Bat
A 3
2 Aug
5261 Ball J A
B 10
1 Aug
3525 Brown J C Art
B 23
2 Aug
6540 Brown E R
C 13
2 Sept
7727 Brown E
D 3
1 Sept
8975 Buckley A M
B 17
1 Sept
1184 Beale R Cav
D 19
Buckner 2 Nov
11761
George K 3
8 Oct
11620 Bell J R
D 28
7 Jan
12373 Bloom J, Cor 65
F 1
8 Feb
12679 Book C
G 19
2 Mch
54 Carpenter Wm Cav 64
I 17
9 April
304 Cook Lewis
E 1
 469 Coombs E A 9 April
I 9
2 April
524 Carter Wm
C 13
9 April
728 Cary W H
F 25
6 May
1357 Carl J M
E 25
2 May
1371 Cabbage C H
H 25
2 June
2012 Cullin John
D 15
1 July
4182 Crasby M
G 28
2 Aug
4620 Carter John
C 3
1 Aug
5036 Carr Wm Cav
D 8
9 Aug
5063 Childs G A
I 8
6 Aug
5826 Crisle J
G 16
Cole’s Sept
8008 Crouse W A -E
C 9
Sept
8035 Conway Wm E -E
6
4 Sept
8266 Crabb H
E 9
1 Sept
8357 Coon H S
E 10
1 Sept
8618 Crouse J A Cav
A 13
10600 Collins D 1 Sept
 C 10
1 Jan
12395 Callahan P 65
F 4
8 Mar
181 Duff Chas, Cor 64
A 27
Dunn John, 9 May
1410
Cor H 27
9 June
2396 Davis Thomas
- 24
35 July
3912 Drew C
B 24
2 July
4138 Dennis Benj
A 28
1 July
4211 Davis G Cav
F 29
2 Aug
6510 Dickwall Wm
F 22
1 Sept
8199 Deller F
E 8
42 Aug
6788 Dennissen T
I 25
4 Sept
8428 Ellis C
D 12
7 Oct
10410 Eli W
C 6
2 July
3849 Fecker L
I 24
9 May
1321 Fairbanks J E
C 23
2 June
2559 Francis J, Cor
K 27
2 June
2600 Flage F J
H 28
2824 Farrass Jas 7 July
G 2
2 Aug
6016 Frantz F
H 17
2 Aug
7404 Fink L
H 31
9 Sept
9290 Frederick J E
I 19
8 Mar
12752 Freare W 65
A 10
9 May
1271 Gordon A B 64
E 22
1 June
2138 Gerard Fred Cav
B 18
2 July
3013 Green Thomas
D 7
2 July
3789 Gregg F
I 22
1 Aug
6072 Gilson J E, S’t Cav 64
C 18
2 Aug
6731 Ganon J W
K 24
1 Mar
12735 Goff John 65
I 6
2 April
1767 Honck J, Cor 64
H 27
9 May
826 Hickley John
G 1
1 June
1625 Howell L H Cav
M 4
2 June
1720 Hoop H
I 8
2657 Hickley J S 2 June
 H 23
1 June
2494 Hidderick H
I 26
2 July
2978 Hite J E
I 7
2 July
3864 Hering P, S’t
C 24
1 Aug
4767 Hank Thomas Bat
D 5
1 Aug
5292 Hilligar ——
E 11
8 Aug
5408 Hood John
C 12
2 Aug
5917 Holmes L
H 17
8 Aug
6484 Hour S
E 22
1 Aug
6504 Harris J E
A 22
9 Sept
7434 Hazel J
C 1
1 Sept
8165 Himick F Cav
E 8
7 Sept
8398 Hall J
D 10
9 Sept
9932 Holden J R
C 28
2 Oct
11109 Hakaion F
K 18
2 Jan
12422 Hoover J Cav 65
C 9
2 July
2895 Isaac Henry 64
H 4
 93 Jones David Bat 1 Mar
A 22
2 April
669 Jenkins M
A 22
2 April
460 Keplinger J
H 9
7 April
544 Keefe Lewis
F 14
9 Aug
7242 Kirby J
F 29
1 May
1019 Laird Corbin Cav
F 11
2 May
1056 Lees W H
C 13
2 July
3913 Louis J, S’t
B 24
2 Oct
11385 Little D Cav
K 24
1 Dec
12361 Lebud J Cav
D 30
1 Feb
12667 Lambert W 65
I 17
McCarle 1 Mar
206 Cav 64
James B 28
2 April
471 Moland B
F 9
9 May
896 Myers Noah
G 5
1 May
1190 McGuigen S K Bat
D 18
1 May
1307 Myers L S
B 23
1797 Moore Frank 9 June
 A 10
6 June
1898 Moffitt Thomas
- 13
2 June
2059 Martz G H
H 16
1 July
3429 Machler C S Bat
A 17
2 July
3797 McKinsay Jno
I 22
6 July
4051 Miller F
C 27
8 July
4146 Mathers F
G 28
Macomber 1 Aug
4881
John C B 6
2 Aug
5170 Marvin J
H 9
1 Aug
6757 Moon J J
D 25
1 Aug
7281 McCullough J
I 30
7 Aug
7327 McLamas J
C 30
2 Sept
8043 Markell S
H 6
4 Oct
10150 Munroe J, Cor
H 1
1 Oct
10861 Markin W
F 13
8 Oct
11547 Mathews J
- 27
1 Feb
12608 McMiller J A 65
E 7
 91 Nice Jacob Cav 5 Mar 64
M 21
9 April
371 Nace Harrison
H 15
1 Sept
9752 Norris N
- 25
2 Mar
153 Pool Hanson
H 25
1 Sept
7590 Porter G
I 2
7 Sept
7981 Pindiville M
H 6
2 Aug
5069 Papple D, Cor
H 8
9 Mar
252 Rusk John
E 30
2 May
918 Russell A P
C 6
9 June
1606 Rodk Simon
E 4
9 June
1901 Robinson J 64
- 13
Rynedollar 1 June
2850
Wm C D 23
1 Aug
6599 Reed Thos P Art
B 23
9 Mar
155 Seberger F
F 25
9 April
317 Scarboro Rob’t
I 2
1 April
478 Suffecol S
I 9
718 Sinder John 2 April
 H 24
9 May
899 Snooks W
E 5
9 May
1205 Spence Levi
D 19
1 May
1272 Scarlett Jas
D 22
9 June
1926 Smith Ed, S’t
I 14
9 June
2004 Stafford John
G 15
9 June
2361 Shipley W
G 23
1 June
2489 Schineder J Bat
B 26
1 Aug
5797 Smith John Cav
B 15
2 Aug
6751 Shelley B
F 24
Shiver G H, 1 Aug
6816
Cor C 25
1 Aug
6919 Stull G E Cav
D 26
2 Sept
7580 Shilling Wm
K 2
7 Sept
7833 Stolz ——
K 4
1 Sept
8296 Smitzer J
D 9
6 Sept
8716 Segar Chas
F 14
2 Sept
9309 Snyder F
K 20
9451 Stratten J A Art 1 Sept
C 21
1 Oct
10215 Shafer J N Cav
A 22
1 Oct
11159 Samon L W
I 19
1 Oct
11160 Speaker H
F 19
4 Nov
12195 Spaulding J
C 29
1 Feb
12704 Smith G C 65
I 26
9 Mar
149 Tyson J T 64
D 25
9 May
1022 Tysen J T
I 11
1 April
677 Turner Wm F Cav
D 22
1 May
1029 Turner A Cav
B 11
9 May
1356 Tindle E, Cor
G 25
9 May
1377 Turner C
E 26
13 Sept
7872 Thompson J
I 5
Thompson 2 Sept
8689
John S 14
2 Sept
9246 Tucker ——
D 19
11 Sept
9335 Tindell Wm
B 20
11450 Tilton J Cav 1 Oct
 F 25
9 June
1583 Ulrich Daniel
I 3
2 May
1305 Veach Jesse
H 23
1 Sept
8269 Viscounts A J Art
E 9
9 Mar
78 Wise John
D 20
9 Mar
21 White Wm
C 7
1 April
553 Widdons D
E 14
Webster 9 April
537
Sam’l, Cor G 17
Wharton 2 May
1171
Samuel F 17
9 June
2275 Worthen Wm
C 20
4 Aug
4748 West M
D 5
Weaver 1 Sept
9409
George B 2
13 Sept
11578 Witman D
D 28
1 Nov
12147 Wolfe H
B 24
9 April
455 Yieldhan R
C 9
Zeck Wm J, 7 May
1060
Cor E 13
9 July
3223 Zimmerman C
E 12