Textbook Learning Representation For Multi View Data Analysis Models and Applications Zhengming Ding Ebook All Chapter PDF

Learning Representation for Multi-View
Data Analysis: Models and Applications

Zhengming Ding
Visit to download the full and correct content document:
https://textbookfull.com/product/learning-representation-for-multi-view-data-analysis-
models-and-applications-zhengming-ding/
More products digital (pdf, epub, mobi) instant
download maybe you interests ...
Linking and Mining Heterogeneous and Multi-view Data

Deepak P
https://textbookfull.com/product/linking-and-mining-
heterogeneous-and-multi-view-data-deepak-p/
Data Analysis in the Cloud : Models, Techniques and

Applications 1st Edition Marozzo
https://textbookfull.com/product/data-analysis-in-the-cloud-
models-techniques-and-applications-1st-edition-marozzo/
Advanced R Statistical Programming and Data Models:

Analysis, Machine Learning, and Visualization 1st
Edition Matt Wiley
https://textbookfull.com/product/advanced-r-statistical-
programming-and-data-models-analysis-machine-learning-and-
visualization-1st-edition-matt-wiley/
Machine Learning and Big Data Analytics Paradigms:

Analysis, Applications and Challenges Aboul Ella
Hassanien
https://textbookfull.com/product/machine-learning-and-big-data-
analytics-paradigms-analysis-applications-and-challenges-aboul-
ella-hassanien/
Time Series Analysis Methods and Applications for
Flight Data Zhang
https://textbookfull.com/product/time-series-analysis-methods-
and-applications-for-flight-data-zhang/
Statistical Modeling for Degradation Data 1st Edition

Ding-Geng (Din) Chen
https://textbookfull.com/product/statistical-modeling-for-
degradation-data-1st-edition-ding-geng-din-chen/
Practical Machine Learning for Streaming Data with

Python: Design, Develop, and Validate Online Learning
Models 1st Edition Sayan Putatunda
https://textbookfull.com/product/practical-machine-learning-for-
streaming-data-with-python-design-develop-and-validate-online-
learning-models-1st-edition-sayan-putatunda/
Electrolyzed Water in Food Fundamentals and

Applications Tian Ding
https://textbookfull.com/product/electrolyzed-water-in-food-
fundamentals-and-applications-tian-ding/
Practical Machine Learning for Data Analysis Using

Python 1st Edition Abdulhamit Subasi
https://textbookfull.com/product/practical-machine-learning-for-
data-analysis-using-python-1st-edition-abdulhamit-subasi/
Advanced Information and Knowledge Processing
Zhengming Ding
Handong Zhao
Yun Fu
Learning
Representation for
Multi-View Data
Analysis
Models and Applications
Advanced Information and Knowledge
Processing
Series editors
Lakhmi C. Jain
Bournemouth University, Poole, UK, and
University of South Australia, Adelaide, Australia
Xindong Wu
University of Vermont
Information systems and intelligent knowledge processing are playing an increasing
role in business, science and technology. Recently, advanced information systems
have evolved to facilitate the co-evolution of human and information networks
within communities. These advanced information systems use various paradigms
including artificial intelligence, knowledge management, and neural science as well
as conventional information processing paradigms. The aim of this series is to
publish books on new designs and applications of advanced information and
knowledge processing paradigms in areas including but not limited to aviation,
business, security, education, engineering, health, management, and science. Books
in the series should have a strong focus on information processing—preferably
combined with, or extended by, new results from adjacent sciences. Proposals for
research monographs, reference books, coherently integrated multi-author edited
books, and handbooks will be considered for the series and each proposal will be
reviewed by the Series Editors, with additional reviews from the editorial board and
independent reviewers where appropriate. Titles published within the Advanced
Information and Knowledge Processing series are included in Thomson Reuters’
Book Citation Index.
More information about this series at http://www.springer.com/series/4738

Zhengming Ding Handong Zhao
•
Yun Fu
Learning Representation
for Multi-View Data Analysis
Models and Applications
123
Zhengming Ding Yun Fu
Indiana University-Purdue Northeastern University
University Indianapolis Boston, MA, USA
Indianapolis, IN, USA
Handong Zhao
Adobe Research
San Jose, CA, USA
ISSN 1610-3947 ISSN 2197-8441 (electronic)

Advanced Information and Knowledge Processing
ISBN 978-3-030-00733-1 ISBN 978-3-030-00734-8 (eBook)
https://doi.org/10.1007/978-3-030-00734-8
Library of Congress Control Number: 2018961715
© Springer Nature Switzerland AG 2019

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part
of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission
or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar
methodology now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publication does not imply, even in the absence of a specific statement, that such names are exempt from
the relevant protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authors or the editors give a warranty, express or implied, with respect to the material contained herein or
for any errors or omissions that may have been made. The publisher remains neutral with regard to
jurisdictional claims in published maps and institutional affiliations.
This Springer imprint is published by the registered company Springer Nature Switzerland AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
Preface
This book equips readers to handle complex multi-view data representation,

centered around several major visual applications, sharing many tips and insights
through a unified learning framework. This framework is able to model most
existing multi-view learning and domain adaptation, enriching readers’ under-
standing from their similarity and differences based on data organization and
problem settings, as well as the research goal.
A comprehensive review exhaustively provides the key recent research on
multi-view data analysis, i.e., multi-view clustering, multi-view classification,
zero-shot learning, and domain adaption. More practical challenges in multi-view
data analysis are discussed including incomplete, unbalanced, and large-scale
multi-view learning. Learning representation for multi-view data analysis covers a
wide range of applications in the research fields of big data, human-centered com-
puting, pattern recognition, digital marketing, Web mining, and computer vision.
This book consists of ten chapters. Chapter 1 introduces the background and
unified model of multi-view data representations. Part I, which includes Chaps. 2–4,
introduces the unsupervised learning for multi-view data analysis. Chapter 2 pre-
sents the unsupervised representation learning methods for two multi-view sce-
narios. One is considering various data sources as multiple views. The other is
considering different splits of one source data as multiple views. Chapter 3
addresses the more challenging and practical incomplete multi-view clustering
problem. Chapter 4 introduces a novel outlier detection problem in multi-view
setting and correspondingly proposes a multi-view outlier detection framework.
Part II, which includes Chaps. 5 and 6, presents the multi-view data analysis for
supervised multi-view classification. Chapter 5 presents two multi-view classifi-
cation models—one is dual low-rank decomposition multi-view subspace and the
other is cross-view auto-encoder. Chapter 6 shows an adaptive latent semantic
representation model in a sparse dictionary learning scheme for zero-shot learning
(a special case of multi-view classification problem). Part III, which includes Chaps.
7–10, presents the multi-view data analysis for domain adaptation. Chapter 7 lists
the missing modality transfer learning model to solve the problem when target
modality is not available in the training stage. Chapter 8 discusses the multi-source
v
vi Preface
transfer learning problem when all the sources are incomplete. Chapter 9 proposes
three deep domain adaptation models to address the challenge where target data has
limited or no label. Following this, Chap. 10 provides a deep domain generalization
model aiming to deal with the target domain that is not available in the training
stage while only with multiple related sources at hand.
In particular, this book can be used by these audiences in the background of
computer science, information systems, data science, statistics, and mathematics.
Other potential audiences can be attracted from broad fields of science and engi-
neering since this topic has potential applications in many disciplines.
We would like to thank our collaborators Ming Shao, Hongfu Liu, and Shuyang
Wang. We would also like to thank editor Helen Desmond from Springer for the
help and support.
Indianapolis, IN, USA Zhengming Ding

San Jose, CA, USA Handong Zhao
Boston, MA, USA Yun Fu
September 2018
Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 What Are Multi-view Data and Problem? . . . . . . . . . . . . . . . . . 1
1.2 A Unified Perspective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Organization of the Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Part I Unsupervised Multi-view Learning

2 Multi-view Clustering with Complete Information . . . . . . . . . . . . . 9
2.1 Deep Multi-view Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.1.2 Deep Semi-NMF Formulation . . . . . . . . . . . . . . . . . . . 11
2.1.3 Experiments on Face Benchmarks . . . . . . . . . . . . . . . . 16
2.1.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.2 Ensemble Subspace Clustering . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.2.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.2.2 Ensemble Formulation with Sparse and Block-Wise
Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.2.3 Experiments on Face, Object, Motion Benchmarks . . . . 34
2.2.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3 Multi-view Clustering with Partial Information . . . . . . . . . . . . . . . 51
3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
3.2 Incomplete Multi-view Clustering . . . . . . . . . . . . . . . . . . . . . . 53
3.2.1 Incomplete Case Formulation . . . . . . . . . . . . . . . . . . . 53
3.2.2 Complete Graph Laplacian . . . . . . . . . . . . . . . . . . . . . 54
3.2.3 Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.2.4 Complexity Analysis . . . . . . . . . . . . . . . . . . . . . . . . . 57
vii
viii Contents
3.3 Experiment on Synthetic and Real-World Data . . . . . . . . . . . . . 58

3.3.1 Experimental Result . . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.3.2 Convergence Study . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
3.3.3 Parameter Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
4 Multi-view Outlier Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
4.2 Related Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
4.3 Multi-view Outlier Detection Method . . . . . . . . . . . . . . . . . . . . 70
4.3.1 The Proposed Consensus Based Algorithm . . . . . . . . . 70
4.3.2 Outlier Measurement Criterion . . . . . . . . . . . . . . . . . . 72
4.4 Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
4.4.1 Algorithm Derivation . . . . . . . . . . . . . . . . . . . . . . . . . 73
4.5 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
4.5.1 Synthetic Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
4.5.2 Real-World Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
4.5.3 Analytical Experiments . . . . . . . . . . . . . . . . . . . . . . . . 85
4.5.4 Application on Saliency Detection . . . . . . . . . . . . . . . . 90
4.5.5 Application on Face Reconstruction . . . . . . . . . . . . . . . 91
4.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
Part II Supervised Multi-view Classification

5 Multi-view Transformation Learning . . . . . . . . . . . . . . . . . . . . . . . 99
5.1 Dual Low-Rank Decomposition for Multi-view Learning . . . . . 99
5.1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
5.1.2 Robust Multi-view Subspace Learning . . . . . . . . . . . . . 101
5.1.3 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
5.2 Coupled Marginalized Auto-encoders for Cross-domain
Multi-view Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
5.2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
5.2.2 The Proposed Algorithm . . . . . . . . . . . . . . . . . . . . . . . 113
5.2.3 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
5.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
6 Zero-Shot Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
6.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
6.2 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
6.3 The Proposed Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
Contents ix
6.3.1 Learning Latent Semantic Dictionary . . . . . . . . . . . . . . 131

6.3.2 Adaptive Graph Guided Latent Semantics . . . . . . . . . . 132
6.3.3 Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
6.3.4 ZSL with Fast Inference . . . . . . . . . . . . . . . . . . . . . . . 135
6.4 Experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
6.4.1 Dataset & Experimental Setting . . . . . . . . . . . . . . . . . . 136
6.4.2 Zero-Shot Classification . . . . . . . . . . . . . . . . . . . . . . . 137
6.4.3 Zero-Shot Retrieval . . . . . . . . . . . . . . . . . . . . . . . . . . 138
6.4.4 Empirical Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
6.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
Part III Transfer Learning

7 Missing Modality Transfer Learning . . . . . . . . . . . . . . . . . . . . . . . 147
7.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
7.1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
7.2 Transfer Learning via Latent Low-Rank Constraint . . . . . . . . . . 150
7.2.1 Conference Version Revisit . . . . . . . . . . . . . . . . . . . . . 150
7.2.2 Transfer Learning with Dictionary Constraint . . . . . . . . 151
7.2.3 Low-Rank Transfer with Latent Factor . . . . . . . . . . . . 152
7.3 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
7.3.1 Datasets and Experiments Setting . . . . . . . . . . . . . . . . 163
7.3.2 Convergence and Property in Two Directions . . . . . . . 164
7.3.3 Recognition Results . . . . . . . . . . . . . . . . . . . . . . . . . . 166
7.3.4 Parameter Property and Training Time . . . . . . . . . . . . 170
7.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
8 Multi-source Transfer Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
8.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
8.2 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
8.3 Incomplete Multi-source Transfer Learning . . . . . . . . . . . . . . . 178
8.3.1 Effective Incomplete Multi-source Alignment . . . . . . . . 179
8.3.2 Cross-Domain Knowledge Transfer . . . . . . . . . . . . . . . 180
8.3.3 Cross-Source Knowledge Alignment . . . . . . . . . . . . . . 183
8.3.4 Solving Objective Function . . . . . . . . . . . . . . . . . . . . . 185
8.3.6 Generalization Bound Analysis . . . . . . . . . . . . . . . . . . 189
8.4 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
8.4.1 Synthetic Experiment . . . . . . . . . . . . . . . . . . . . . . . . . 190
8.4.2 Real-world Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . 191
8.4.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
x Contents
8.4.4 Property Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196

8.4.5 Incomplete Single Source Comparison . . . . . . . . . . . . . 199
8.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
9 Deep Domain Adaptation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
9.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
9.2 Stacked Low-Rank Coding . . . . . . . . . . . . . . . . . . . . . . . . . . . 204
9.2.1 Single-Layer Low-Rank Coding . . . . . . . . . . . . . . . . . 205
9.2.2 Optimization Solution . . . . . . . . . . . . . . . . . . . . . . . . . 208
9.2.4 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . 211
9.3 Deep Low-Rank Coding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
9.3.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
9.3.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
9.3.3 Deep Transfer Low-Rank Coding . . . . . . . . . . . . . . . . 218
9.3.4 Non-linear Representation . . . . . . . . . . . . . . . . . . . . . . 224
9.4 Spectral Bisection Tree Guided Deep Adaptive Exemplar
Autoencoder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
9.4.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
9.4.2 Data Composition via Spectral Bisection Tree . . . . . . . 236
9.4.3 Deep Adaptive Exemplar Autoencoder . . . . . . . . . . . . 237
9.4.5 Datasets and Experimental Setting . . . . . . . . . . . . . . . . 244
9.4.6 Results and Discussion . . . . . . . . . . . . . . . . . . . . . . . . 244
9.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246
10 Deep Domain Generalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
10.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
10.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
10.3 Deep Generalized Transfer Learning . . . . . . . . . . . . . . . . . . . . 254
10.3.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254
10.3.2 Deep Neural Networks Revisit . . . . . . . . . . . . . . . . . . 255
10.3.3 Deep Generalized Transfer Learning . . . . . . . . . . . . . . 255
10.3.4 Model Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
10.4 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260
10.4.1 Datasets and Experimental Setting . . . . . . . . . . . . . . . . 260
10.4.2 Comparison Experiments . . . . . . . . . . . . . . . . . . . . . . 261
10.4.3 Self-evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264
10.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
Chapter 1
Introduction
Multi-view data generated from various view-points or multiple sensors are com-
monly seen in real-world applications. For example, the popular commercial depth
sensor Kinect uses both visible light and near infrared sensors for depth estimation;
autopilot uses both visual and radar sensors to produce real-time 3D information
on the road; face analysis algorithms prefer face images from different views for
high-fidelity reconstruction and recognition. However, such data with large view
divergence would lead to an enormous challenge: data across various views have a
large divergence preventing them from a fair comparison. Generally, different views
tend to be treated as different domains from different distributions. Thus, there is an
urgent need to mitigate the view divergence when facing specific problems by either
fusing the knowledge across multiple views or adapting knowledge from some views
to others. Since there are different terms regarding “multi-view” data analysis and
its aliasing, we first give a formal definition and narrow down our research focus to
differentiate it from other related works but in different lines.
1.1 What Are Multi-view Data and Problem?
Definition 1 (Multi-view Data) (Fig. 1.1): Assume we have a set of data X =

{X 1 , X 2 , . . . , X v } from v views, e.g., face poses, camera views and types of features.
In this book, we are especially interested in two cases upon data correspondence:
First, the samples across v views are correspondent (i.e., sample-wise relationship)
in multi-view data, falling in the conventional multi-view learning; Second, the
samples across different views have no data correspondence, falling in the transfer
learning scenario, where discriminant knowledge are transferred.
First, multi-view learning aims to to merge the knowledge from different views
to either uncover common knowledge, or employ the complementary knowledge in
© Springer Nature Switzerland AG 2019 1

Z. Ding et al., Learning Representation for Multi-View Data Analysis,
Advanced Information and Knowledge Processing,
https://doi.org/10.1007/978-3-030-00734-8_1
2 1 Introduction
Fig. 1.1 Different scenarios of multi-view data analytics. a Different types of features from single
image; b different sources to represent information; c–e images from different viewpoints
specific views to assist learning tasks. For example, in vision, multiple features
extracted from the same object by various visual descriptors, e.g., LBP, SIFT and
HOG are very discriminant in recognition tasks. Another example is multi-modal data
captured, represented, and stored in varied formats, e.g., near-infrared and visible
face, and image and text. For multi-view learning, the goal is to fuse the knowledge
from multiple views to facilitate common learning tasks, e.g., clustering and classi-
fication. The key challenge is exploring data correspondence across multiple views.
The mappings among different views are able to couple view-specific knowledge
while additional labels would help formulate supervised regularizers. The general
setting of multi-view clustering is to group n data samples in v different views (e.g.,
v types of features, sensors, or modalities) by fusing the knowledge across different
views to seek a consistent clustering result. The general setting of multi-view clas-
sification is that it needs to build a model with given v views of training data. In
the test stage, we would have two different scenarios. First, one view will be used
to recognize other views with the learned model. In this case, the label information
across training and test data is different; Second, specifically for multi-features based
learning, is that v-view training data is used to seek a model by fusing the cross-view
knowledge, which is also used as gallery data to recognize v-view probe data.
Second, domain adaptation attempts to transfer knowledge from labeled source
domains to facilitate the learning burden in the target domains with sparsely or
no labeled samples. For example, in surveillance, faces are captured by long wave
infrared sensor in night-time, but recognition model is trained on regular face images
collected under visible light. Conventional domain adaptation methods consider seek-
ing domain-invariant representation for the data or modifying classifiers to fight off
the marginal or conditional distribution mismatch across source and target domains.
The goal of domain adaptation is to transfer knowledge from well-labeled sources
to unlabeled targets, which accounts for the more general settings that some source
views are labeled while target views are unlabeled. The general setting of domain
adaptation is that we build a model on both labeled source data and unlabeled target
data. Then we use the model to predict the unlabeled target data, either the same
1.1 What Are Multi-view Data and Problem? 3
data in the training stage or different data. Thus, we have corresponding transductive
domain adaptation and inductive domain adaption.
There are different strategies to deal with multi-view data, e.g., translation, fusion,
alignment, co-learning and representation learning. This book will focus on represen-
tation learning and fusion. The following chapters would discuss multi-view data ana-
lytic algorithms along with our proposed unified model Sect. 1.2 from three aspects.
Furthermore, we will discuss the challenging situation where test data are sampled
from unknown categories, e.g., zero-shot learning, and more challenging tasks with
incomplete data, e.g., missing modality transfer learning, incomplete multi-source
adaptation and domain generalization.
1.2 A Unified Perspective
Due to the distribution divergence across different views, view-invariant feature

learning is a widely-used and promising technique to address the multi-view chal-
lenges. Generally, multiple view-specific linear or non-linear mapping functions
would be sought to transform the original multi-view data into a new common space
by identifying dedicated alignment strategies with various loss functions. Specifi-
cally, we could formulate them into a common objective including two parts: (1)
multi-view alignment term; (2) feature learning regularizer, namely:

v
v
min A ( f i (X i ), f j (X j )) + λ R( f k (X k )),
f 1 (·),..., f v (·)
i=1,i< j k=1
where f i (·) is a feature learning function for view i, either linear, non-linear mapping,
or deep network.
The first common term A (·) is a pairwise symmetric alignment function across
multiple views to either fuse the knowledge among multiple views or transfer knowl-
edge across different views. Due to different problem settings, multi-view learning
and domain adaptation would explore various strategies to define the loss functions.
While multi-view learning employs data correspondence (i.e., sample-wise relation-
ship w/ or w/o labels) to seek common representation, domain adaptation employs
domain- or class-wise relationship during the model learning for discriminant domain
invariant feature.
The second common term R(·) is the feature learning regularizer by incorporat-
ing either the labeled information or the intrinsic structure of the data, or both during
the mapping learning. To name a few, logistic regression, Softmax regression, graph
regularizers are usually incorporated to carry the label and manifold information.
When we turn to deep learning, this term is mostly Softmax regression. For a part of
multi-view learning algorithms, they would merge feature learning regularizer into
the alignment term. Generally, the formulation of the second term is very similar
4 1 Introduction
between multi-view learning and domain adaptation within our research concentra-
tion.
Along the unified model, we will cover both shallow structure learning and deep
learning approaches for multi-view data analysis, e.g., subspace learning, matrix
factorization, low-rank modeling, deep auto-encoder, deep neural networks, deep
convolutional neural networks. For example, multi-view clustering models will be
explored including multi-view matrix factorization, multi-view subspace learning,
multi-view deep structure learning in unsupervised setting.
1.3 Organization of the Book
The rest of this book is organized as follows. The first two parts are for multi-view
data analysis with sample-wise correspondence; and the third part is for multi-view
data analysis with class-wise correspondence.
Part I focuses on developing unsupervised multi-view clustering (MVC) models.
It consists of the following three chapters. Chapter 2 explores complementary infor-
mation across views to benefit the clustering problem and presents a deep matrix
factorization framework for MVC, where semi-nonnegative matrix factorization is
adopted to learn the hierarchical semantics of multi-view data in a layer-wise fashion.
To maximize the mutual information from each view, we enforce the non-negative
representation of each view in the final layer to be the same. Furthermore, to respect
the intrinsic geometric structure in each view data, graph regularizers are introduced
to couple the output representation of deep structures.
Chapter 3 considers an underlying problem hidden behind the emerging multi-
view techniques: What if one/more view data fail? Thus, we propose an unsuper-
vised method which well handles the incomplete multi-view data by transforming
the original and incomplete data to a new and complete representation in a latent
space. Different from the existing efforts that simply project data from each view
into a common subspace, a novel graph Laplacian term with a good probabilistic
interpretation is proposed to couple the incomplete multi-view samples. In such a
way, a compact global structure over the entire heterogeneous data is well preserved,
leading to a strong grouping discriminability.
Chapter 4 presents a multi-view outlier detection algorithm based on clustering
techniques to identify two different types of data outliers with abnormal behaviors.
We first give the definition of both types of outliers in multi-view setting. Then we
propose a multi-view outlier detection method with a novel consensus regularizer
on the latent representations. Specifically, we explicitly characterize each kind of
outliers by the intrinsic cluster assignment labels and sample-specific errors. We
experimentally show that this practice generalizes well when the number of views are
greater than two. Last but the least, we make a thorough discussion on the connection
and difference between the proposed consensus-regularization and the state-of-the-
art pairwise-regularization.
1.3 Organization of the Book 5
Part II proposes to solve multi-view classification problems including zero-shot

learning (a special problem of multi-view learning). This part includes the following
two chapters.
Chapter 5 presents two multi-view transformation learning algorithms. First, we
develop a Robust Multi-view Subspace Learning algorithm (RMSL) through dual
low-rank decompositions, which desires to seek a low-dimensional view-invariant
subspace for multi-view data. Generally, one sample lies in two kinds of structures,
one is class structure and the other is view structure, which are intertwined with
one another in the original feature space. To address this, Through dual low-rank
decompositions, RMSL aims to disassemble two intertwined structures from each
other in the low-dimensional subspace. Second, we propose a Coupled Marginal-
ized Denoising Auto-encoders framework, whose core idea is to build two types of
marginalized denoising auto-encoders for effective feature extraction. Specifically,
the intermediate dataset is treated as one of two views in one domain, therefore, one
domain has two views while the other domain only has one view.
Chapter 6 targets at precisely recognizing unseen categories through a shared
visual-semantic function, which is built on the seen categories and expected to well
adapt to unseen categories. We tackle this issue by exploiting the intrinsic relationship
in the semantic manifold and enhancing the transferability of visual-semantic func-
tion. Specifically, we propose an Adaptive Latent Semantic Representation (ALSR)
model in a sparse dictionary learning scheme, where a generic semantic dictionary
is learned to connect the latent semantic space with visual feature space. To build
a fast inference model, we explore a non-linear network to approximate the latent
sparse semantic representation, which lies in the semantic manifold space.
Part III discusses the transfer learning scenarios when the multi-view data are
with class-wise correspondence. This part includes the following four chapters.
Chapter 7 defines Missing Modality Problem in transfer learning, since we always
confront such a problem that no target data are achievable, especially when data
are multi-modal. Under this situation, the target modality is blind in the training
stage, while only the source modality can be obtained. To this end, we propose a
novel transfer learning framework by extending conventional transfer learning into
two directions to handle the Missing Modality Problem. By borrowing an auxiliary
database with the same complete modalities, our model can learn appropriate low-
dimensional subspaces from cross-modality direction and cross-database one.
Chapter 8 attempts to utilize incomplete multiple sources for effective knowledge
transfer to facilitate the learning task in target domain. Nowadays, it is common
to see multiple sources available for knowledge transfer, each of which, however,
may not include complete classes information of the target domain. Naively merging
multiple sources together would lead to inferior results due to the large divergence
among multiple sources. The core idea is to seek an appropriate domain-free subspace
where relevant knowledge for target from multiple sources is coupled and reinforced
to compensate for any missing data in other sources. Specifically, IMTL is designed to
minimize the marginal and conditional distribution discrepancy from two directions:
cross-domain transfer and cross-source transfer.
6 1 Introduction
Chapter 9 develops three novel deep domain adaptation approaches. First, we

propose a Deep Low-Rank Coding framework (DLRC) for transfer learning. The core
idea of DLRC is to jointly learn a deep structure of feature representation and transfer
knowledge via an iterative structured low-rank constraint, which aims to deal with
the mismatch between source and target domains layer by layer. Second, we propose
a novel Deep Transfer Low-rank Coding (DTLC) framework to uncover more shared
knowledge across source and target in a multi-layer manner. Specifically, we extend
traditional low-rank coding with one dictionary to multi-layer dictionaries by jointly
building multiple latent common dictionaries shared by two domains. Third, we
propose a novel deep model called “Deep Adaptive Exemplar AutoEncoder”, where
we build a spectral bisection tree to generate source-target data compositions as
the training pairs fed to autoencoders, and impose a low-rank coding regularizer to
ensure the transferability of the learned hidden layer.
Chapter 10 explores to fight off the challenge through capturing knowledge from
multiple source domains and generalizing to the unseen target domains. In reality,
we would always confront such cases in reality that the target data are totally blind in
the training stage, which is extremely challenging since we have no prior knowledge
of the target. However, existing domain generalization research efforts all employ
shallow structures, so it is difficult for them to well uncover the rich information
within the complex data. To this end, we desire to explore deep structure learning in
domain generalization to uncover more effective knowledge across multiple sources.
Part I
Unsupervised Multi-view Learning
Chapter 2
Multi-view Clustering with Complete
Information
Abstract Multi-view Clustering (MVC) has garnered more attention recently since
many real-world data are comprised of different representations or views. The key
is to explore complementary information to benefit the clustering problem. In this
chapter, we consider the conventional complete-view scenario. Specifically, in the
first section, we present a deep matrix factorization framework for MVC, where
semi-nonnegative matrix factorization is adopted to learn the hierarchical semantics
of multi-view data in a layer-wise fashion. In the second section, we make an exten-
sion and consider the different sampled feature sets as multi-view data. We propose
a novel graph-based method, Ensemble Subspace Segmentation under Block-wise
constraints (ESSB), which is jointly formulated in the ensemble learning framework.
2.1 Deep Multi-view Clustering1
2.1.1 Overview
Traditional clustering aims to identify groups of “similar behavior” in single view

data (Von Luxburg 2007; Liu et al. 2015; Steinwart 2015; Tao et al. 2016; Liu et al.
2016; Li et al. 2017). As the real-world data are always captured from multiple sources
or represented by several distinct feature sets (Cai et al. 2013a; Ding and Fu 2014;
Gao et al. 2015; Zhao and Fu 2015; Wang et al. 2016), MVC is intensively studied
recently by leveraging the heterogeneous data to achieve the same goal. Different
features characterize different information from the data set. For example, an image
can be described by different characteristics, e.g., color, texture, shape and so on.
These multiple types of features can provide useful information from different views.
MVC aims to integrate multiple feature sets together, and uncover the consistent
latent information from different views. Extensive research efforts have been made
1 Thischapter is reprinted with permission from AAAI. “Multi-view Clustering via Deep Matrix
Factorization”. 31st AAAI Conference on Artificial Intelligence, pp. 2921–2927, 2017.
© Springer Nature Switzerland AG 2019 9
Z. Ding et al., Learning Representation for Multi-View Data Analysis,
Advanced Information and Knowledge Processing,
https://doi.org/10.1007/978-3-030-00734-8_2
10 2 Multi-view Clustering with Complete Information
in developing effective MVC methods (Cai et al. 2013a; Gao et al. 2015; Xu et al.
2016; Zhao et al. 2016). Along this line, Kumar et al. developed co-regularized Multi-
view spectral clustering to do clustering on different views simultaneously with a
co-regularization constraint (Kumar et al. 2011). Gao et al. proposed to perform
clustering on the subspace representation of each view simultaneously guided by a
common cluster structure for the consistence across different views (Gao et al. 2015).
A good survey can be found in Xu et al. (2013).
Recently, lots of research activities on MVC have achieved promising performance
based on Non-negative Matrix Factorization (NMF) and its variants, because the non-
negativity constraints allow for better interpretability (Guan et al. 2012; Trigeorgis
et al. 2014). The general idea is to seek a common latent factor through non-negative
matrix factorization among Multi-view data (Liu et al. 2013; Zhang et al. 2014, 2015).
Semi Non-negative Matrix Factorization (Semi-NMF), as one of the most popular
variants of NMF, was proposed to extend NMF by relaxing the factorized basis matrix
to be real values. This practice allows Semi-NMF to have a wider application in the
real world than NMF. Apart from exploring Semi-NMF in MVC application for the
first time, our method has another distinction from the existing NMF-based MVC
methods: we adopt a deep structure to conduct Semi-NMF hierarchically as shown in
Fig. 2.1. As illustrated, through the deep Semi-NMF structure, we push data samples
from the same class closer layer by layer. We borrow the idea from deep learning
(Bengio 2009), thus this practice has such a flavor. Note that the proposed method
is different from the existing deep auto-encoder based MVC approaches (Andrew
et al. 2013; Wang et al. 2015), though all of us are of deep structure. One major
difference is that Andrew et al. (2013), Wang et al. (2015) are based on Canonical
Correlation Analysis (CCA), which is limited to 2-view case, while our method has
no such limitation.
Fig. 2.1 Framework of our proposed method. Same shape denotes the same class. For demonstra-
tion purposes, we only show the two-view case, where two deep matrix factorization structures
are proposed to capture rich information behind each view in a layer-wise fashion. With the deep
structure, samples from the same class but different views gather close to each other to generate
more discriminative representation
2.1 Deep Multi-view Clustering 11
To sum up, in this section we propose a deep MVC algorithm through graph reg-
ularized semi-nonnegative matrix factorization. The key is to build a deep structure
through semi-nonnegative matrix factorization to seek a common feature represen-
tation with more consistent knowledge to facilitate clustering. To the best of our
knowledge, this is the first attempt applying semi-nonnegative matrix factorization
to MVC in a deep structure. We summarize our major contributions as follows:
• Deep Semi-NMF structure is built to capture the hidden information by leveraging
benefits of strong interpretability from Semi-NMF and effective feature learning
from deep structure. Through this deep matrix factorization structure, we dis-
semble unimportant factors layer by layer and generate an effective consensus
representation in the final layer for MVC.
• To respect the intrinsic geometric relationship among data samples, we introduce
graph regularizers to guide the shared representation learning in each view. This
practice makes the consensus representation in the final layer preserve most shared
structures across multiple graphs. It can be considered as a fusion scheme to boost
the final MVC performance.
2.1.2 Deep Semi-NMF Formulation
2.1.2.1 Overview of Semi-NMF
As a variant of NMF, Ding et al. (2010) extended the application of traditional

NMF from non-negative input to a mix-sign input, while still preserving the strong
interpretability at the same time. Its objective function can be expressed as:
min X − Z H 2F , (2.1)

Z ,H ≥0
where X ∈ Rd×n denotes the input data with n samples, each sample is of d dimen-
sional feature. In the discussion on equivalence of semi-NMF and K-means clustering
(Ding et al. 2010), Z ∈ Rd×K can be considered as the cluster centroid matrix,2 and
H ∈ R K ×n , H ≥ 0 is the “soft” cluster assignment matrix in latent space.3 Similar
to the traditional NMF, the compact representation H uncovers the hidden semantics
by simulating the part-based representation in human brain, i.e., psychological and
physiological interpretation.
While in reality, natural data may contain different modalities (or factors), e.g.,
expression, illumination, pose in face datasets (Samaria and Harter 1994; Georghi-
ades et al. 2001). Single NMF is not strong enough to eliminate the effect of
2 For a neat presentation, we do not follow the notation style in Ding et al. (2010), and remove the
mix-sign notation “±” on X and Z , which does not affect the rigorousness.
3 In some literatures (Ding et al. 2010; Zhao et al. 2015), Semi-NMF is also called the soft version
of K-means clustering.
those undesirable factors and extract the intrinsic class information. To solve this,
Trigeorgis et al. (2014) showed that a deep model based on Semi-NMF has a promis-
ing result in data representation. The multi-layer decomposition process can be
expressed as
X ≈ Z 1 H1+
X ≈ Z 1 Z 2 H2+
.. (2.2)
.
X ≈ Z 1 . . . Z m Hm+
where Z i denotes the ith layer basis matrix, Hi+ is the ith layer representation matrix.
Trigeorgis et al. (2014) proved that each hidden representations layer is able to
identify the different attributes. Inspired by this work, we propose a MVC method
based on deep matrix factorization technique.
In the MVC setting, let us denote X = {X (1) , . . . , X (v) , . . . , X (V ) } as the data
sample set. V represents the number of views. X (v) ∈ Rdv ×n , where dv denotes the
dimensionality of the v-view data and n is the number of data samples. Then we
formulate our model as:

V

min (α (v) )γ X (v) −Z 1(v) Z 2(v) . . . Z m(v) Hm 2F + βtr(Hm L (v) HmT )
Z i(v) , Hi(v) v=1
Hm , α (v) (2.3)

V
s.t. Hi(v) ≥ 0, Hm ≥ 0, α (v) = 1, α (v) ≥ 0,
v=1
where X (v) is the given data for vth view. Z i(v) , i ∈ {1, 2, . . . , m} is the ith layer map-
ping for view v. m is the number of layers. Hm is the consensus latent representation
for all views. α (v) is the weighting coefficient for the vth view. γ is the parameter to
control the weights distribution. L (v) is the graph Laplacian of the graph for view v,
where each graph is constructed in k-nearest neighbor (k-NN) fashion. Theweight
matrix of the graph for view v is A(v) and L (v) = A(v) − D (v) , where Dii(v) = j Ai(v) j
(He and Niyogi 2003; Ding and Fu 2016).
Remark 1 Due to the homology of Multi-view data, the final layer representation
Hm(v) for vth view data should be close to each other. Here, we use the consensus
Hm as a constraint to enforce Multi-view data to share the same representation after
multi-layer factorization.
Remark 2 Multiple graphs are constructed to constrain the common representation

learning so that the geometric structure in each view could be well preserved for the
final clustering. Moreover, the novel graph term could fuse the geometric knowledge
from multiple views to make the common representation more consistent.
2.1.2.2 Optimization
To expedite the approximation of the variables in the proposed model, each of the
layers is pre-trained to have an initial approximation of variables Z i(v) and Hi(v)
for the ith layer in vth view. The effectiveness of pre-training has been proven
before Hinton and Salakhutdinov (2006) on deep autoencoder networks. Similar
to Trigeorgis et al. (2014), we decompose the input data matrix X (v) ≈ Z 1(v) H1(v)
to perform the pre-training, where Z 1(v) ∈ Rdv × p1 and H1(v) ∈ R p1 ×n . Then the vth
view feature matrix H1(v) is decomposed as H1(v) ≈ Z 2(v) H2(v) , where Z 2(v) ∈ R p1 × p2
and H2(v) ∈ R p2 ×n . p1 and p2 are the dimensionalities for layer 1 and layer 2,
respectively.4 Continue to do so until we have pre-trained all layers. Follow-
ing this, the weights of each layer is fine-tuned by alternating minimizations of
the proposed objective function Eq. (2.3). First, we denote the cost function as

V
C = (α (v) )γ X (v) − Z 1(v) Z 2(v) . . . Z m(v) Hm 2F + βtr(Hm L (v) HmT ) .
v=1
Update rule for weight matrix Z(v) i . We minimize the objective value with
(v)
respect to Z i by fixing the rest of variables in vth view for the ith layer. By setting
∂C /∂ Z i(v) = 0, we give the solutions as
Z i(v) = (Φ T Φ)−1 Φ T X (v) H̃i(v) T ( H̃i(v) H̃i(v) T )−1

(2.4)
Z i(v) = Φ † X (v) H̃i(v) † ,
where Φ = [Z 1(v) . . . Z i−1

(v)
], H̃i(v) denotes the reconstruction (or the learned latent
feature) of the ith layer’s feature matrix in vth view, and notation † represents the
Moore–Penrose pseudo-inverse.
Update rule for weight matrix Hi(v) (i < m). Following Ding et al. (2010), the
update rule for Hi(v) (i < m) is formulated as follows:

[Φ T X (v) ]pos + [Φ T Φ Hi(v) ]neg
Hi(v) = Hi(v) , (2.5)
[Φ T X (v) ]neg + [Φ T Φ Hi(v) ]pos
where [M]pos denotes a matrix that all the negative elements are replaced by 0.
Similarly, [M]neg denotes one that has all the positive elements replaced by 0. That is,
pos |Mk j | + Mk j neg |Mk j | − Mk j

∀k, j [M]k j = , [M]k j = . (2.6)
2 2
Update rule for weight matrix Hm (i.e., Hi(v) (i = m)). Since Hm involves the
graph term, the updating rule and convergence property have never been investigated
4 For the ease of presentation, we denote the dimensionalities (layer size) from layer 1 to layer m
as [ p1 . . . pm ] in the experiments.
before. We give the updating rule first, followed by the proof of its convergence
property.
[Φ T X (v) ]pos +[Φ T Φ Hm ]neg +Gu (Hm , A)
Hm =Hm (2.7)
[Φ T X (v) ]neg +[Φ T Φ Hm ]pos +Gd (Hm , A)
where Gu (Hm , A) = β([Hm A(v) ]pos + [Hm D (v) ]neg ) and Gd (Hm , A) = β([Hm
A(v) ]neg + [Hm D (v) ]pos ).
Theorem 2.1 The limited solution of the update rule in Eq. (2.7) satisfies the KKT
condition.
Proof We introduce the Lagrangian function

V
L (Hm ) = (α (v) )γ X (v) − Z 1(v) Z 2(v) . . . Z m(v) Hm 2F
v=1
(2.8)
(v)

+ βtr(Hm L HmT ) − ηHm ,
where the Lagrangian multiplier η enforces nonnegative constraints, Hm ≥ 0. The

zero gradient condition gives ∂L (Hm )/∂ Hm = 2Φ T (Φ Hm − X (v) ) + 2Hm (D (v) −
A(v) ) − η = 0. From the complementary slackness condition, we obtain

2Φ T (Φ Hm − X (v) ) + 2Hm (D (v) − A(v) ) kl
(Hm )kl
(2.9)
= ηkl (Hm )kl = 0.
This is a fixed point equation that the solution must satisfy at convergence.
The limiting solution of Eq. (2.7) satisfies the fixed point equation. At conver-
gence, Hm(∞) = Hm(t+1) = Hm(t) = Hm , i.e.,
(Hm )kl = (Hm )kl

[Φ T X (v) ]kl + [Φ T Φ Hm ]kl + [Gu (Hm(v) , A)]kl
pos neg (2.10)
.
[Φ T X (v) ]neg + [Φ T Φ Hm ]pos + [Gd (Hm(v) , A)]kl
Note that Φ T X (v) = [Φ T X (v) ]pos − [Φ T X (v) ]neg ; Φ T Φ Hm = [Φ T Φ Hm ]pos −

[Φ T Φ Hm ]neg ; Hm D (v) = [Hm D (v) ]pos − [Hm D (v) ]neg ; Hm A(v) = [Hm A(v) ]pos −
[Hm A(v) ]neg . Thus Eq. (2.10) reduces to
T
2Φ (Φ Hm −X (v) )+2Hm (D (v) −A(v) ) kl (Hm )2kl = 0. (2.11)
Equation (2.11) is identical to Eq. (2.9). Both equations require that at least one of
the two factors is equal to zero. The first factors in both equations are identical. For
the second factor (Hm )kl or (Hm2 )kl , if (Hm )kl = 0 then (Hm2 )kl = 0, and vice versa.
Therefore if Eq. (2.9) holds, Eq. (2.11) also holds and vice versa.
Update rule for weight α (v) . Similar to (Cai et al. 2013b), for the ease of rep-
resentation, let us denote R (v) = X (v) − Z 1(v) Z 2(v) . . . Z m(v) Hm 2F + βtr(Hm L (v) HmT ).
The objective in Eq. (2.3) with respect to α (v) is written as

V
V
min (α (v) )γ R (v) , s.t. α (v) = 1, α (v) ≥ 0. (2.12)
α (v)
v=1 v=1
The Lagrange function of Eq. (2.12) is written as

V V
min (α (v) )γ R (v) − λ( α (v) − 1), (2.13)
α (v)
v=1 v=1
where λ is the Lagrange multiplier. By taking the derivative of Eq. (2.13) with respect
to α(v), and setting it to zero, we obtain
1
λ γ −1
α (v) = . (2.14)
γ R (v)

V
Then we replace α (v) in Eq. (2.14) into α (v) = 1, and obtain
v=1
1
(v) γ R (v) 1−γ
α = V .
(v)
1−γ
1 (2.15)
γR
v=1
It is interesting to see that with only one parameter γ , we could control the different
weights for different views. When γ approaches ∞, we get equal weights. When γ
is close to 1, the weight of the view whose R (v) value is the smallest is assigned to
1, and the others are assigned to 0.
Until now, we have all the update rules done. We repeat the updates iteratively
until convergence. The entire algorithm is outlined in Algorithm 2.1. After obtaining
the optimized Hm , standard spectral clustering (Ng et al. 2001) is performed on the
graph built on Hm via k-NN algorithm.
2.1.2.3 Time Complexity
Our deep matrix factorization model is composed of two stages, i.e., pre-training and
fine-tuning, so we analyze them separately. To simplify the analysis, we assume the
dimensions in all the layers (i.e., layer size) are the same, denoting p. The original
feature dimensions for all the views are the same, denoting d. V is the number of
views. m is the number of layers.
Algorithm 2.1: Optimization Solution of Problem (2.3)

Input: Multi-view data X (v) , tuning parameters γ , β, the layer size di , the number of classes
k.
1 Initialize:
2 for all layers in each view do
(v) (v) (v)
3 (Z i , Hi ) ← SemiNMF(Hi−1 , di )
4 end
5 while not converged do
6 for all layers in each view do
˜ Hm if i = m
7 Hi(v) ← (v) ˜(v)
Z i+1 Hi+1 otherwise
8 Φ ← i−1τ =1 Z τ .
(v)
9 Z i ← Φ † X (v) H̃i .
(v) Update via Eq. (2.5) if i = m
10 Hi ←
Update via Eq. (2.7) otherwise
11 end
12 end
(v) (v)
Output: Weighted matrices Z i and feature matrices Hi (i = m) and Hm in the final layer.
In pre-training stage, the Semi-NMF process and graph construction are the time
consuming
parts. The complexity is of order O V mt p (dnp + np 2 + pd 2 + pn 2 +
dn ) , where t p is the number of iterations to achieve convergence in Semi-NMF
2
optimization
process. Normally,
p < d, thus the computational cost is T pr e. =
O V mt p (dnp + pd 2 + dn 2 ) for the pre-training stage. Similarly, in the fine-tuning

stage, the time complexity is of order T f ine. = O V mt f (dnp + pd 2 + pn 2 ) , where
t f is the number of iterations in this fine-tuning stage. To sum up, the overall com-
putational cost is Ttotal = T pr e. + T f ine. .
2.1.3 Experiments on Face Benchmarks
We choose three face image/video benchmarks in our experiments, as face contains

good structural information, which is beneficial to manifesting the strengths of deep
NMF structure. A brief introduction of datasets and preprocessing steps is as follows.
Yale consists of 165 images of 15 subjects in raw pixel. Each subject has 11
images, with different conditions, e.g., facial expressions, illuminations, with/without
glasses, lighting conditions, etc. Extended Yale B consists of 38 subjects of face
images. Each subject has 64 faces images under various lighting conditions and
poses. In this work, the first 10 subjects, 640 images data are used for experiment.
Notting-Hill is a well-known video face benchmark (Zhang et al. 2009), which is
generated from movie “Notting Hill”. There are 5 major casts, including 4660 faces
in 76 tracks.
For these datasets, we follow the preprocessing strategy (Cao et al. 2015). Firstly
all the images are resized into 48 × 48 and then three kinds of features are extracted,
i.e., intensity, LBP (Ahonen et al. 2006) and Gabor (Feichtinger and Strohmer 1998).
Specifically, LBP is a 59-dimension histogram over 9 × 10 pixel patches generated
from cropped images. The scale parameter λ in Gabor wavelets is fixed as 4 at four
orientations θ = {0◦ , 45◦ , 90◦ , 135◦ } with a cropped image of size 25 × 30 pixels.
For the comparison baselines, we have the following. (1) BestSV performs stan-
dard spectral clustering (Ng et al. 2001) on the features in each view. We report the best
performance. (2) ConcatFea concatenates all the features, and then performs stan-
dard spectral clustering. (3) ConcatPCA concatenates all the features, then projects
the original features into a low-dimensional subspace via PCA. Spectral clustering
is applied on the projected feature representation. (4) Co-Reg (SPC) (Kumar et al.
2011) co-regularizes the clustering hypotheses to enforce the memberships from
different views admit with each other. (5) Co-Training (SPC) (Kumar and Daume
III 2011) borrows the idea of co-training strategy to alternatively modify the graph
structure of each view using other views’ information. (6) Min-D(isagreement) (de
Sa 2005) builds a bipartite graph which derives from the “minimizing-disagreement”
idea. (7) MultiNMF (Liu et al. 2013) applies NMF to project each view data to the
common latent subspace. This method can be roughly considered as one-layer ver-
sion of our proposed method. (8) NaMSC (Cao et al. 2015) firstly applies (Hu et
al. 2014) to each view data, then combines the learned representations and feeds to
the spectral clustering. (9) DiMSC (Cao et al. 2015) investigates the complementary
information of representations of Multi-view data by introducing a diversity term.
This work is also one of the most recent approaches in MVC. We do not make the
comparison with deep auto-encoder based methods (Andrew et al. 2013, Wang et
al. 2015), because these CCA-based methods cannot fully utilize more than 2 view
data, leading to an unfair comparison.
To make a comprehensive evaluation, we use six different evaluation metrics
including normalized mutual information (NMI), accuracy (ACC), adjusted
rand index (AR), F-score, Precision and Recall. For details about the metrics,
readers could refer to Kumar and Daume III (2011), Cao et al. (2015). For all the
metrics, higher value denotes better performance. Different measurements favor dif-
ferent properties, thus a comprehensive view can be acquired from the diverse results.
For each experiment, we repeat 10 times and report the mean values along with stan-
dard deviations.
2.1.3.1 Result
Tables 2.1 and 2.2 tabulate the results on datasets Yale and Extended YaleB. Our
method outperforms all the other competitors. For the dataset Yale, we raise the
performance bar by around 7.57% in NMI, 5.08% in ACC, 8.22% in AR, 6.56% in
F-score, 10.13% in Precision and 4.61% in Recall. On average, we improve the state-
of-the-art DiMSC by more than 7%. The possible reason why our method improves
a lot is that both image data in Yale and Extended YaleB contain multiple factors, i.e.,
Table 2.1 Results of 6 different metrics (mean ± standard deviation) on dataset Yale
Method NMI ACC AR F-score Precision Recall
BestSV 0.654 ± 0.616 ± 0.440 ± 0.475 ± 0.457 ± 0.495 ±
0.009 0.030 0.011 0.011 0.011 0.010
ConcatFea 0.641 ± 0.544 ± 0.392 ± 0.431 ± 0.415 ± 0.448 ±
0.006 0.038 0.009 0.008 0.007 0.008
ConcatPCA 0.665 ± 0.578 ± 0.396 ± 0.434 ± 0.419 ± 0.450 ±
0.037 0.038 0.011 0.011 0.012 0.009
Co-Reg 0.648 ± 0.564 ± 0.436 ± 0.466 ± 0.455 ± 0.491 ±
0.002 0.000 0.002 0.000 0.004 0.003
Co-Train 0.672 ± 0.630 ± 0.452 ± 0.487 ± 0.470 ± 0.505 ±
0.006 0.001 0.010 0.009 0.010 0.007
Min-D 0.645 ± 0.615 ± 0.433 ± 0.470 ± 0.446 ± 0.496 ±
0.005 0.043 0.006 0.006 0.005 0.006
MultiNMF 0.690 ± 0.673 ± 0.495 ± 0.527 ± 0.512 ± 0.543 ±
0.001 0.001 0.001 0.000 0.000 0.000
NaMSC 0.671 ± 0.636 ± 0.475 ± 0.508 ± 0.492 ± 0.524 ±
0.011 0.000 0.004 0.007 0.003 0.004
DiMSC 0.727 ± 0.709 ± 0.535 ± 0.564 ± 0.543 ± 0.586 ±
0.010 0.003 0.001 0.002 0.001 0.003
Ours 0.782 ± 0.745 ± 0.579 ± 0.601 ± 0.598 ± 0.613 ±
0.010 0.011 0.002 0.002 0.001 0.002
pose, expression, illumination, etc. The existing MVC methods only involve one layer
of representation, e.g., one layer factor decomposition in MultiNMF or the practice of
self-representation (i.e., coefficient matrix Z in NaMSC and DiMSC Cao et al. 2015).
However, our proposed approach can extract the meaningful representation layer by
layer. Through the deep representation, we eliminate the influence of undesirable
factors, and keep the core information (i.e., class/id information) in the final layer.
Table 2.3 lists the performance on video data Notting-Hill. This dataset is more
challenging than the previous two image datasets, since the illumination conditions
vary dramatically and the source of lighting is arbitrary. Moreover, there is no fixed
expression pattern in the Notting-Hill movie, on the contrary to datasets Yale and
Extended YaleB. We observe from the tables that our method reports the superior
results in five metrics. The only outlier is NMI, but our performance is slightly
worse than DiMSC by only 0.25%. Therefore, we safely draw the conclusion that our
proposed method generally achieves better clustering performance in the challenging
video dataset Notting-Hill.
2.1.3.2 Analysis
In this subsection, the robustness and stability of the proposed model is evaluated.
The convergence property is firstly studied in terms of objective value and NMI
Table 2.2 Results of 6 different metrics (mean ± standard deviation) on dataset Extended YaleB
Method NMI ACC AR F-score Precision Recall
BestSV 0.360 ± 0.366 ± 0.225 ± 0.303 ± 0.296 ± 0.310 ±
0.016 0.059 0.018 0.011 0.010 0.012
ConcatFea 0.147 ± 0.224 ± 0.064 ± 0.159 ± 0.155 ± 0.162 ±
0.005 0.012 0.003 0.002 0.002 0.002
ConcatPCA 0.152 ± 0.232 ± 0.069 ± 0.161 ± 0.158 ± 0.164 ±
0.003 0.005 0.002 0.002 0.001 0.002
Co-Reg 0.151 ± 0.224 ± 0.066 ± 0.160 ± 0.157 ± 0.162 ±
0.001 0.000 0.001 0.000 0.001 0.000
Co-Train 0.302 ± 0.186 ± 0.043 ± 0.140 ± 0.137 ± 0.143 ±
0.007 0.001 0.001 0.001 0.001 0.002
Min-D 0.186 ± 0.242 ± 0.088 ± 0.181 ± 0.174 ± 0.189 ±
0.003 0.018 0.001 0.001 0.001 0.002
MultiNMF 0.377 ± 0.428 ± 0.231 ± 0.329 ± 0.298 ± 0.372 ±
0.006 0.002 0.001 0.001 0.001 0.002
NaMSC 0.594 ± 0.581 ± 0.380 ± 0.446 ± 0.411 ± 0.486 ±
0.004 0.013 0.002 0.004 0.002 0.001
DiMSC 0.635 ± 0.615 ± 0.453 ± 0.504 ± 0.481 ± 0.534 ±
0.002 0.003 0.000 0.006 0.002 0.001
Ours 0.649 ± 0.763 ± 0.512 ± 0.564 ± 0.525 ± 0.610 ±
0.002 0.001 0.002 0.001 0.001 0.001
performance. Then the analytical experiments on three key model parameters β, γ ,

and layer size are conducted.
Convergence analysis. In Theorem 2.1, we theoretically show that the most
complex updating for Hm satisfies KKT conditions. To experimentally show the
convergence property of the whole model, we compute the objective value of Eq. (2.3)
in each iteration. The corresponding parameters γ , β and layer size are set as 0.5,
0.1 and [100, 50], respectively. The objective value curve is plotted in red in Fig. 2.2.
We observe that the objective value decreases steadily, and then gradually meets the
convergence after around 100 iterations. The average NMI (in blue) has two stages
before converging: from #1 to #14, the NMI increases dramatically; then from #15 to
#30, it slightly bumps and reaches the best at around the convergence point. For the
sake of safety, the maximum number of iterations is set to 150 for all the experiments.
Parameter analysis. In the proposed method, we have four sets of parameters
i.e., balancing parameters β and γ , layer size pi and the number of nearest neighbors
k when constructing k-NN graph. Selecting k in the k-NN graph construction algo-
rithms is an open problem (He and Niyogi 2003). Due to the limited page length, we
only include the first three parameter analysis experiments in this section. However,
we find that k = 5 usually achieves relatively good results.
Figure 2.3 shows the influence of NMI result with respect to the parameter γ under
three different layer size settings, i.e., {[100 50], [500 50], [500 200]}. Parameter
Another random document with
no related content on Scribd:
Flip scrambled to her feet and Madame Perceval tried them against
her. "How are they?" Flip asked eagerly.
"Perfect. Couldn't be better. Put on your things and we'll go out and
try them."
As Flip snapped the skis onto her boots Madame Perceval said,
"Now don't expect miracles, Philippa. The skis don't make as much
difference as all that. Just go very slowly and do as I say."
Madame Perceval was right. Flip was not able, all of a sudden, to ski
like an angel because of the new skis. But she no longer fell quite so
frequently, or had such a desperate struggle to get to her feet again.
"Better, much better!" Madame Perceval cried as Flip slid down a
tiny incline and stopped without falling. "Now turn around."
Flip raised her leg and the long ski no longer tumbled her
ignominiously onto the snow. She snapped her other leg around and
there she was, all in one piece and erect.
"Bravo!" Madame cried. "Now herring-bone up the little hill and come
down again."
Her tongue sticking out with eagerness, Flip did as Madame
Perceval told her.
"Good," the art teacher said. "Good, Philippa. More spring in your
knees if you can. How about that bad knee? Does it bother you?"
"Not much." Flip shook her head. "Oh, Madame, do you think I can
learn?"
"I know you can. Just don't stick your tongue out so far. You might
bite it off in one of your tumbles."
"Do you think Fräulein Hauser will take me back in the skiing class?"
"Wait! Wait!" Paul cried, waving his ski sticks in wild excitement. "I
have a much better idea."
Madame laughed and ducked as one of the sticks went flying. "All
right, Paul. Calm down and tell us this magnificent idea." But Flip
could see that she was pleased because Paul sounded excited and
happy, and the dark look had fled from his face.
"Well, Flip was telling me about this ski meet you have at school and
how everybody can go in for it and there's a prize for form, and a
long race, and a short race, and a prize for the girl who's made most
progress and all sorts of things. And I think it would be wonderful if
we could teach Flip and she could enter the ski meet and win and
surprise everybody."
Madame Perceval started to laugh but then she looked at Flip and
Paul and their eager excited faces, and she said slowly, "It would be
rather a tall order teaching Flip just on week-ends. She needs lots of
practice."
"I could slip out in the morning before Call Over," Flip cried. "If I
make my bed before breakfast and hurry breakfast I'd have almost
an hour and nobody'd see me then."
"And think how surprised that Fräulein Hauser would be," Paul cried.
"And the girls would be so surprised," Flip shouted. "Erna and Jackie
and all of them. Oh, Madame, do you think I could learn? I'd work
terribly hard. I'd practice and practice."
"If you keep on improving the way you've improved this afternoon,"
Madame Perceval told her, "I'm sure you could."
"Come on, Aunt Colette," Paul cajoled.
Madame Perceval looked at them for a moment longer. Then she
smiled and said, "Why not?"
4
Flip finished her still life of a plaster head of Diana, a wine bottle, a
loaf of bread, and a wine glass, early during the next art class.
"That's good, Flip," Madame Perceval said. "Really very good,
though your perspective is wobbly—everything's going up hill at
quite an alarming angle and poor Diana looks as though she were
about to fall on her ear. But the color and texture is excellent. That's
really bread, and the transparency of your glass is a great
improvement over your last still life. That's good work, Flip."
Flip blushed with pleasure, partly at the praise, and partly because
Madame was calling her Flip. Several of the girls looked up at the
name and Gloria actually winked at her.
"You have time to start something else," Madame was saying.
"Here's a clean sheet of paper and a piece of charcoal. Just draw
anything you like. Either from something in the room or from your
imagination."
For the past two days Flip had been thinking of three things, Paul,
skiing, and Madame's daughter. She had not had another
opportunity to ask Paul about Denise, how old she was, or whether
she was alive or dead. Somehow Flip felt that she must be dead and
that perhaps that accounted for the sadness in Madame Perceval's
eyes. She wondered what Madame's daughter would look like and,
almost without volition, her hand holding the charcoal moved across
the paper and she began to draw a girl, a girl about her own age
sitting on a rock and looking out across the valley to the mountains.
The likeness was stronger than she could possibly have guessed.
She was trying, more or less, to draw a girl who looked like Madame
and who had short hair like hers. But the girl who appeared on the
paper did not look like Madame and Flip felt discouraged because
she knew the perspective was wrong again and the mountains were
too small and far away and the girl's feet weren't right. She sighed
and tried to erase the mountains and the feet and correct them.
Madame Perceval stood behind her and looked over her shoulder
down at the paper. Flip almost jumped as the art teacher's strong
fingers dug into her arm.
"What are you doing?" Madame Perceval's voice was calm and low,
but Flip felt the strain in it.
"Just—just a girl looking at the mountains," she stammered. "The—
the feet aren't right."
"I'll show you," Madame Perceval said; but instead of explaining
what was wrong, and then telling Flip what to do to correct it, as she
usually did, she took the charcoal and swiftly put the feet in again
herself; and then she took the thumbtacks out of Flip's board and
took the paper and walked over to the cupboard with it and Flip saw
that her hands were trembling.
In a moment she came back with a fresh piece of paper. "Why don't
you try drawing one of the girls in the class?" Madame suggested,
and her voice was natural again. "Erna, you've finished, haven't you?
Will you sit still and let Flip sketch you?"
"Yes, Madame. How do you want me to sit, Pi—Philippa—uh—Flip?"
Madame Perceval smiled as Erna stumbled over Flip's name, and
Flip said, "Oh, the way you are now looking over the back of your
chair is fine, if you're comfortable."
She took up the charcoal and sketched quickly and then she laughed
because the girl on her paper was so out of proportion and funny-
looking and at the same time she was Erna. In trying to get a
likeness Flip had over-accentuated and the braces on Erna's teeth
were ridiculous and her chin jutted out and the barette pulled the hair
back far too tightly from the forehead.
"What are you laughing at?" Erna demanded.
Flip looked at her drawing and thought,—oh, dear, now Erna will be
mad.
But Madame Perceval had come over and was laughing, too, and
showing the paper with Erna on it to the class, and everybody was
laughing.
"I think you have a flair for caricature, Flip," Madame said.
And Jackie bounced up and down on her chair, crying, "Draw me,
Flip, draw me!"
"Hold still, then, Jackie," Madame said, handing Flip another sheet of
paper.
Flip's hand holding the charcoal made Jackie's curly hair fly wildly
about the paper; the enormous, long lashed black eyes took up half
the page, and the mouth was a tiny bud above the pointed little chin.
Erna had been watching and as Flip laid down the charcoal for a
moment she grabbed the paper and held it up, shouting,
"Look at Jackie! She looks just like a cat!"
"Draw me! Draw me!" All the girls were shouting at Flip until Madame
Perceval stopped them, saying, "Not now, girls. The bell just rang.
You can get Flip to draw you any time. I know she'd like to, wouldn't
you, Flip?"
"Oh, yes, Madame!"
So they besieged Flip in the Common Room with requests for
caricatures to send home, and Flip went to her locker, her face bright
with happiness, to get her sketch book and pencils.
"Don't make my nose too big!" "Should I take my glasses off, Flip?"
"Oh, Pill, don't put in my freckles!" 'Flip' and 'Pill' came
indiscriminately, and somehow quite suddenly and surprisingly Flip
knew that she no longer minded the 'Pill' because it sounded
friendly; it was being said to her, not at her.
—I'm liking school, she thought.—I'm liking it. Now it will sound
better when I tell Paul I like it.
Only Esmée Bodet was discontented with her picture. "I don't look
like that!" she said, and tore the page across, tossing the pieces in
the waste paper basket.
"She looks exactly like that," Erna said in Flip's ear. "Come on up on
the billiard table and let's play jacks." The entire school had a jacks
craze on. Even the seniors were playing though Esmée turned up
her nose and said it was a child's game, and continued to play very
bad bridge.
"Oh, jacks! Let me play too!" Gloria cried, clambering up and sitting
cross-legged on the green felt of the billiard table; and Flip realized
that one reason Gloria never lacked for partners, or a place in the
Common Room games, was that she never hesitated to ask.
"Come on, Jackie," Erna called. "Climb up."
Flip was quite good at jacks and Gloria bounced up and down
impatiently. "Come on, Pill, miss can't you? I want a turn." And she
gave Flip's elbow a jog, but Flip caught the ball and laughed
triumphantly.
"Good for you, Flip," Erna cried. "You can't play if you're going to
cheat, Glo."
"It's Erna's turn next, anyhow," Jackie said. "By the way, Pill, I think
it's a dirty shame Hauser made you drop skiing."
"Me too." Erna nodded so violently that her hair came out of the
barette and she had to fasten it again.
Flip thought of the progress she had already made on her skis, and
smiled to herself. Then she shrugged, "Well, if she thinks I'm too
impossible to teach, I guess that's that."
"The old minge, the mangy old minge," Gloria muttered. "I say, Pill.
What're you going to be when you get out of this place, an artist?"
Flip nodded. "I'd like to be. The way my father is. I'd like to paint
portraits and do illustrations for children's books." She reached wildly
for the jacks' ball, which was this time an old golf ball Gloria's mother
had sent, but it bounced off the table and Erna scrambled after it.
"At last," she said, bringing it back and collecting the jacks. "I'm
going to be a doctor like my father. I think it must be wonderful to cut
people up and put them back together again." Underneath her joking
words Flip could tell that she was serious.
"The trouble is that you can't always put them back together again,"
Jackie said.
"I will." Erna swept up her jacks with a confident gesture. "If people
have their legs and things blown off I'll discover a way to put them
back or give them new ones off dead people."
Flip started to tell Erna that Paul wanted to be a doctor too, but
Gloria, who didn't mind when she herself talked about glass eyes or
false teeth, put her hands over her ears. "Oh, stop! Stop!"
"Well, dead people can give their eyes so blind people can see,"
Erna said, "so I don't see why they shouldn't give their legs and
things, too."
Gloria clapped her hand over Erna's mouth. "You go talk about your
old operations somewhere else."
"Who asked you to play jacks anyhow?" Erna mumbled from behind
Gloria's hand. "Let go and let me play. I'm on fivesies, eggs in the
basket."
"Foursies."
"Fivesies."
"It's fivesies," Flip corroborated. "Are you going to be a movie
actress, Jackie?"
Jackie laughed and waved her arms. "My father says I'll be an
actress over his dead body. I haven't thought about it much. Maybe
I'll just be a wife like my mother. She says that's a career in itself,
only lots of people forget it."
"Love," Gloria sighed, "that's what I'm cut out for."
"Do you believe in love at first sight?" Flip asked and blushed.
"I believe in love." Gloria placed her hand dramatically over her
heart. "It's love that makes the world go round."
"Have you seen Maggie Campbell's brother?" Jackie asked. "He's
the handsomest man I ever saw. Maggie's going to give me a
snapshot of him for Christmas."
Flip sat with her legs stuck out in front of her on the old hotel billiard
table, because her stiff knee kept her from sitting cross-legged or on
her heels, and watched, and listened, and occasionally said a word,
and she felt so excited that she could feel the excitement like hunger
in the pit of her stomach. She was excited because for the first time
she felt on the inside, and underneath the new warm sense of being
one of them was the glorious secret knowledge of Paul—and
tomorrow she would see him again.
5
The first thing Paul asked Flip the next day was, "Have you been
practicing your skiing?"
Flip nodded. "Every morning."
"How's it going?"
"Better."
"Well, come on and let's go. Is Aunt Colette coming over?"
"I don't know."
"Well, come on, Flip," Paul said impatiently. "I want to see how much
you've improved."
They went out, Ariel rushing madly about them, digging up the snow,
running and jumping against them, until Paul had to send him in.
Paul was visibly impressed with Flip's progress, and when Madame
Perceval appeared on skis, Paul flew over to her in great excitement.
"Flip's a natural born skier, Aunt Colette!" he cried. "She's
magnificent!"
Madame Perceval smiled at Paul and held out her hand to Flip.
"Let's see what you've accomplished, little one."
She, too, was impressed. "You must have been working hard!" she
said. "We'll have you doing Christianas and all sorts of things in no
time."
"Oh, Madame, do you really think so?"
"Just keep up the practicing, Flip, as you've been doing, and I'm sure
of it."
"She'll be quite a shock to everybody at the ski meet, won't she?"
Paul asked.
Madame laughed. "She certainly will."
And Flip went to bed that night to dream of soaring through the air on
her skis, watched by admiring throngs of girls; of executing perfect
Christianas and the delicate loops of telemarks; and when she woke
up in the morning her mind was still a happy jumble of snow
conditions, stems, and langlaufs.
Flip had thought as she slipped out the ski room door after breakfast
each morning that the girls would become curious about her hurried
breakfasts and ask what she was doing; but they were used to her
disappearances and absences and were too hungry and sleepy and
hurried in the cold dark of the mornings to pay much attention to
anything besides getting themselves out of their warm beds and then
eating as much hot chocolate and porridge and rolls and jam as
possible.
Flip was out practicing intently one Saturday morning when she
noticed someone watching her. She looked up, fearful that she was
being discovered, but it was no one from the school. It was a man
with a dark, wild face, and the look in his eyes frightened her; but he
waved and grinned at her cheerfully and moved away. He wore
climbing boots and carried a stick and he struck off up the mountain,
walking very rapidly. She watched after him until he was lost in the
trees, wondering what a strange man was doing on the grounds of a
girls' school. Then she thought he might be a new gardener or
perhaps someone to help with flooding the hockey field for ice
skating, though that was not to be done till the Christmas holidays.
Oh, well, she thought, there's never anybody around who isn't meant
to be around, so I guess it's all right.
And she kept on working at the skiing until time to get the mail before
Call Over.
Most of the girls were already at the desk in the Hall when she
arrived, flushed from her early morning exercise; and Signorina, who
was on duty, was giving out the mail. Since she had begun noticing
other people besides herself, Flip had learned a lot from the mail.
Hardly a day went by that Jackie did not have a letter from her
mother. Erna always came rushing eagerly to the desk but seldom
received anything. Gloria frequently didn't even bother to come and if
she had a letter someone took it to her. Esmée had already begun to
get letters from boys and read them aloud to anyone who would
listen. Solvei's letters came as regularly as Jackie's, and Sally
received hers every Wednesday and Saturday.
"Philippa Hunter," Signorina called.
Flip took the letter from her father and opened it eagerly.
"My darling baby," he said, beginning the letter as he had not done in
years, "here I am in a hospital in Shanghai, but don't be worried
because it's nothing serious—jaundice—but it's a great nuisance
especially because the doctor says I won't possibly be able to get to
you for your Christmas holidays. Flippet, Flippet, don't be too terribly
disappointed and don't weep that sweet face into a pulp. Eunice will
be delighted to have you for your holidays, and she is in Nice, and
the weather will be wonderful, and I know she'll do everything she
can to make you happy. Your letters have sounded so much more
contented recently and I feel that you are growing up and that you try
to enjoy yourself without your yellow old father. I expect to be in
Germany and Switzerland shortly after New Year and I promise you
that nothing will interfere with our Easter."
Flip's disappointment was so acute and overwhelming that she
thought for a moment she was going to be sick. She turned and ran
until she reached the bathroom and then she shut herself in and
leaned against the door and she felt all hollow inside herself, from
the top of her head down to her toes, and there was no room in this
cold vacuum for tears.
After a few moments she heard a knock. She clenched her fists and
held her breath but whoever it was did not go away, and the knock
came again. If it's Miss Tulip I'll kick her, she thought in fury.
Then Erna's voice came. "Flip."
"What?" Flip said, sounding hard and forbidding.
"Flip, it's just me. Erna."
"Oh."
"Did you—was it—was there bad news in your letter?"
"No. It's all right." Flip's voice was stifled.
"Well, look, Flip," Erna said. "I just meant ... Percy's taking Call Over
this morning and you know how strict she is ... and the bell's about to
ring...."
Flip opened the door and came out. "Thanks, Erna."
"Oh, that's all right," Erna said uncomfortably. "I'm sorry if it was bad
news in your letter."
"It's just that my father's sick in China and I can't be with him for the
Christmas holidays," Flip started to explain in a controlled voice.
Then she burst out, "and I have to spend the holidays with Eunice—
she's a friend of my father's—and I don't like her and if she marries
my father I'll—I'll want to kill her."
"Ach, that's awful," Erna said. "I'm awful sorry, Flip. It certainly is
awful."
"Well—" Flip's voice trailed off; then she spoke briskly. "We'd better
get down to Call Over."
6
The next day she told Paul about the letter and for the first time
since she had received it she started to cry. Ariel, distressed at her
unhappiness, jumped up at her, almost knocking her over, and licked
excitedly at her face.
"That Eunice," Paul said, frowning heavily and pushing Ariel away
from Flip and sending him over to the hearth. Then he jumped up.
"Put on your skis and go on out and start practicing," he
commanded. "I'll be out in a minute." And he half-shoved Flip out the
door.
Flip went out obediently and put on her skis and started working on
her turns. In just a few minutes Paul came flying out of the lodge,
shouting, "Flip! Flip!"
He rushed up, panting, and gasped, "My father says you may stay
here with us for Christmas if your father says it's all right! And Aunt
Colette is going to be with us because my mother can't come." His
face was radiant with pleasure.
Flip sat down in the snow, her feet going every which way.
"And you can work on your skiing every day. And I'm sure Aunt
Colette can take us up to Gstaad to ski, and to Caux too, so you'll be
familiar with Gstaad and all the runs for the ski meet and maybe you
will become such a good skier that we can do a double jump! Papa
said he'd write your father right away this afternoon. Oh, Flip, it will
be wonderful to have you here all the time instead of just on Sunday
afternoons!"
"Oh, Paul!" Flip cried and scrambled to her feet. "Oh, Paul! Next to
being with father it's the most wonderful thing in the world. I know
he'll let me!"
"Well," Paul said, giving her a quick, shy hug. "What a relief. Come
on. Let's get to work on your skiing."
Flip had been skiing conscientiously for about an hour under Paul's
tutelage when Madame Perceval came out and called them.
"Come on in to tea, children!"
They skied over to her, Flip with almost as great ease and
confidence as Paul, shouting, "Hello, Madame!" "Hello, Aunt
Colette!"
"So," Madame said, raising Flip's chin and looking into her eyes.
"You're happy about your holidays now?"
"Oh, yes, Madame!"
"I was wondering what had happened to upset you, my problem
child. You seemed so much happier and then gloom descended. But
you did have some reason this time. It's hard to be away from your
father at Christmas time."
"And it would have been awful to be with Eunice," Flip said. "Eunice
always makes me feel—well, even clumsier and gawkier and
tongue-tieder and everything than I am. But oh, Madame, I'll love
being here, and I'll try to help and not be a bother."
"Hurry up, Flip, take off your skis," Paul called impatiently. "Papa
went over to Lausanne to the dentist yesterday and brought us back
cakes from Nyffeneggers."
When they had finished tea Madame said, "How about skiing back to
school with me, Flip? Feel up to it?"
"Yes, Madame, I think so."
"You haven't skied any distance at all, yet, and I think it would be
good for you. Not afraid of skiing in the dark? I'll keep right beside
you."
"I'm not afraid, Madame."
They pushed off, Flip feeling excited and happy as she turned
around to wave good-bye to Paul, who was standing in the lighted
doorway. And Flip thought how beautiful the night was with the stars
just coming out; and the pine trees' noble arms bowed with snow;
and the shadows of the ruined chateau looming behind them; and
the warmth and comfort of the lodge, the golden light pouring out the
open door and Paul standing there waving good-bye.
"Yes," Madame Perceval said, as if in answer to her thoughts. "It's
beautiful, isn't it? In the spring the fields are as white as they are
now, with narcissi, not snow.... Shall we go?"
They started off down the mountain side, Madame calling Flip from
time to time to check her speed or give her instructions. Now at last
Flip had the feeling of being a bird, of having wings. And as she
pushed through the cold night air she felt that it was as solid and
entire an element as water. A bird must know this solidity; but as she
felt the air against her body the only thing within her own knowledge
with which she could compare it was water, and she felt as she
broke through it that she must be leaving a wake of air behind her,
as a boat does, cutting through water.
Madame let her go faster and faster, and, exhilarated by the speed
and the beauty, she would have gone flying past the school gates if
Madame had not checked her. They turned through the gates
together and moved slowly down the white driveway.
"That was good skiing, Flip," Madame said. "I'm really very proud of
you."
Flip dropped her head in quick confusion, then looked up with eyes
that shone in the starlight. "I love it, Madame, I just love it!"
"You know," Madame told her, "We're not going to be able to enter
you in the beginner's class at the ski meet. You'll have to go in the
intermediate. If you go on improving at this rate you'd be disqualified
from the beginner's class. And with all the skiing you'll be able to do
during the holidays I don't think there's any question but you'll go on
improving. I want to work with you on your left stem turn. Your right is
fine, but the left is the only place where your weak knee seems to
bother you. Don't worry, though. I think a little extra practice and the
left stem will be as good as the right."
They went indoors and Flip put her skis on the rack, stroking them
lovingly. The smell of the ski room, of hot wax and melted snow, and
damp wool from the ski clothes, was almost as pleasant to her now
as the smell of the art Studio.
"Madame," she said softly, "thank you so much for the skis."
"The girl who left them was rolling in money," Madame spoke shortly,
"and I suspect it was black market money. They're in far better hands
now—or rather on far better feet." She laughed. "Run along upstairs
to the Common Room. There's about half an hour before dinner. We
made better time than I expected."
7
Flip ran up the stairs and across the Hall, almost bumping into Miss
Tulip.
"Really, Philippa Hunter!" Miss Tulip exclaimed in annoyance. "Will
you kindly remember that you are supposed to walk, not run. You
used to be such a nice, quiet girl and you're turning into a regular
little hoyden." And Miss Tulip shut herself up in the cage of the
faculty elevator and pressed the button.
Instead of being crushed by Miss Tulip's irritation Flip had to
surpress a laugh as she watched the elevator rise and saw the
matron's feet in their long, narrow white shoes slowly disappearing
up the elevator shaft. Then, completely forgetting her admonition,
she ran on down the corridor and into the Common Room.
She had just started a letter to her father when the big glass door
was opened and Martha Downs and Kaatje van Leyden came in. A
sudden hush came over the Common Room because the senior girls
had studies and a special living room of their own on the second
floor, and seldom came downstairs unless it was to lecture one of the
girls for some misdeed that affected the two school teams, the Odds
and the Evens, or that came under the jurisdiction of the Student
Government. Martha and Kaatje walked towards Flip now and she
knew that everybody was wondering, "Now what has Pill done?"
But Martha smiled in a friendly way and said, "Hi, Philippa."
"Hi," Flip said, standing up awkwardly.
"I hear you're good at drawing people."
"Oh—just sort of caricatures," Flip mumbled.
Erna, who had been listening curiously, broke in, "She's wonderful,
Martha! I'll show you the ones she did of Jackie and Gloria and me in
the dormitory last night."
Erna had forgotten that they weren't supposed to have books or
drawing materials in the dormitory at night, but Martha and Kaatje
kindly ignored this and looked at the slips of paper Erna held out.
They both laughed.
"Why, you're a genius, Philippa," Kaatje cried.
And Martha said, "We came down to see if you'd do us."
"Oh, I'd love to," Flip said. "Right now?"
"How long does it take you?"
"About a second," Erna told them. "Here's a chair, Martha, and one
for you, Kaatje. Run get your sketch book, Flip."
Flip got her pad and a couple of sharp pencils out of her locker. "Just
stay the way you are, please," she said to Martha. "That's fine."
It wasn't quite as easy to draw Martha as it had been the girls she
saw constantly in the Common Room and the class room, or as easy
as the faculty, whose caricatures, sketched hurriedly at the end of
study halls had thrown the girls into fits of laughter; but she managed
to get a passable exaggeration of Martha's almost Hollywood beauty
onto the paper, and the Head Girl was very pleased.
While Flip was drawing Kaatje, Martha said, "My mother writes me
you're going to be spending the holidays in Nice with Mrs. Jackman,
Philippa. We're going to be there for a week, so maybe we'll see
you."
Flip shook her head, glancing up briefly from her sketch of Kaatje.
"I'm not going to be with Mrs. Jackman. I'm staying up the mountain
with Paul Laurens."
"Percy's nephew?" Martha asked in surprise. "How did you get to
know him?"
"She has tea with him every Sunday afternoon." Erna, who had
evidently appointed herself as Flip's spokesman told the seniors.
"She's just come back from there now, haven't you, Flip?"
Flip nodded, tore off her page, and gave it to Kaatje.
"Thanks simply ages, Philippa," Kaatje said. "You'll probably be
besieged by every girl in school."
"I don't mind," Flip said. "It's what I love to do. If those aren't right or
if you want any more I'd love to try again."
"We may take you up on that." Martha smiled at her. "Sorry you
aren't going to be in Nice for the holidays."
"Flip, you're made," Erna said when the older girls had left. "If Martha
and Kaatje like your pictures there won't be a girl in school who won't
want one. I bet you'll get artist's cramp or something."
"It's all right with me." Flip grinned happily.
"And it's wonderful about the holidays. When did that happen?"
"This afternoon. And Madame's going to be there, too."
"Percy?" Erna looked dubious. "I'm not sure I'd like that. She's so
strict."
"She's not a bit strict when you're not at school. She's—oh, she's so
much fun and she doesn't act a bit like a teacher. And Paul says
she'll take us on all kinds of trips on the holidays, to Gstaad, and
we'll come down from Caux on a bobsled, and we'll go to Montreux
and places to the movies and all sorts of things."
"It's too bad you can't ski," Erna said; and Flip turned away to hide a
grin.
8
Flip was out skiing by herself before breakfast several mornings
later when she saw the strange man again. At first she did not notice
him, and then she became vaguely aware through her concentration
on her skiing that someone was watching her, and she swung
around and there he was leaning against a tree. This time he did not
smile and wave and move away up the mountain. He just stood
there watching her and she stared nervously back. He was very thin
and his cheeks were sunken and his jaw dark as though he needed
to shave. He wore shabby ski clothes and a small beret and his eyes
were very dark and brilliant. She stood, leaning lightly on her ski
sticks, looking back at him and wishing he would go away when
suddenly he came stumbling across the snow towards her. She
started to push away on her skis but he made a sudden leap at her
and she fell headlong. She started to scream but he clapped his
hand across her mouth.
CHAPTER FIVE
The Stranger
"DON'T be afraid. Don't be afraid. I won't hurt you," he kept saying,
and he righted her and stood her up again, keeping a firm grip on her
arm. She could feel each of his fingers pressing through her sweater
and ski jacket and they hurt as they dug into her arm.
"Let go!" she gasped. "Let me go!"
"It's all right," he repeated. "I won't hurt you. Don't be afraid."
"But you are hurting me! Let go!"
Slowly his fingers relaxed, though he did not release her. "I didn't
mean to knock you down like that. I lost my balance and fell against
you. I'm very tired and hungry. Have you any food?"
She shook her head.
"Just a cracker or a piece of chocolate? School girls always have
something to eat in their pockets."
She shook her head again. "I haven't anything. What are you doing
here?"
"I'm the—uh—I'm the new janitor. I'm going to keep the furnace
going so you'll be warm enough all winter. I live—uh—I live up the
mountain and I didn't have a chance to eat breakfast this morning
because I overslept. Are you sure you haven't even a crust of
bread?"
"I haven't anything. Won't the cook give you something in the
kitchen?"
"She's in a bad mood this morning. What are you doing out here all
alone? Shouldn't you be in the school?"
"Not till Call Over at a quarter to nine."
"But why are you here all alone?" the man asked her, and she was
afraid of the hungry look in his dark eyes.
"I'm skiing."
"But why do you ski here all alone every morning?" he persisted.
"I like it."
Now at last he let go her arm. "Well, I'm off up the mountain," he
said, and without another word or a backwards glance he struck off
across the snow.
The thought of him troubled her until she went in to get the mail
before Call Over. Then she had a letter that made her so angry that
she forgot all about him. The letter was from Eunice, and it ran,
"My dear Philippa, I am glad to hear from your father that at last you
are getting along better at school. But I must admit that I am rather
hurt that you choose to spend the holidays with some strange boy
you have just met rather than with me. However, you have always
been an odd child so I suppose I shouldn't be surprised. I do want to
say, though, Philippa dear, that I know your poor father would be
happier if you came to Nice, and I assure you that I would see that
you had a pleasant vacation. As I said in my letter to you last week,
there will be a number of charming young people nearby, and I am
sure it would do you good to know them. Just remember that all you
have to do if you change your mind is to let me know, and don't
forget that you have your father's peace of mind to think of as well as
your own choice. It is very hard on him to be laid up in the hospital,
poor darling, and I shouldn't think you'd want in any way to add to his
worries. I'm afraid this will make you angry, Philippa dear, but do
remember that I'm just thinking of your best interests and that I'm
very fond of you and devoted to your father. Affectionately, Eunice."
Quivering with rage she tore the letter into as small pieces as
possible. Madame Perceval, on duty behind the desk, finished
distributing the mail and asked with a smile, "What's the cause of
your fury, Flip?"
"It's that Eunice again," Flip said. "A woman who's always after my
father. She thinks I ought to spend the holidays with her and I'm
afraid she'll try to convince father that I ought to, too. There isn't time
for that, is there?"
"No, Flip, there isn't. Anyhow, Mlle. Dragonet had a cable from your
father this morning giving his permission for you to stay with Paul.
She supplemented Georges' cable by one of her own saying that
she thought it far better for you to stay with her nephew than for you
to make the difficult trip to Nice. So I don't think you need worry."
"Thank goodness," Flip said. "I think I'd die if I couldn't spend the
holidays with Paul. I just wish Eunice hadn't written the letter and
tried to spoil things for me."
"Just forget it and enjoy yourself," Madame Perceval advised.
"I will," Flip said, and she ran upstairs to throw the scraps of Eunice's
letter in the classroom waste paper basket. Erna was there before
her, sitting glumly at her desk.
"What's the matter, Erna?" Flip asked shyly.
"I can't spend the holidays with Jackie," Erna answered and put her
head down on her arms.
Flip perched awkwardly on her desk and put her feet on the chair.
"Oh, Erna, why not?"
"My mother wrote Mlle. Dragonet and said she wanted me home for
Christmas. She doesn't want me home at all. She sent me away to
school because she didn't want me home."
"Oh, Erna," Flip said, her voice warm with sympathy.
"Both my brothers were killed in the war," Erna said in a muffled
voice. "And I know mutti wishes it had been me. She always liked my
brothers better. I was the baby and so much younger and I always
got in the way."
"Oh, no, Erna," Flip protested. "Your mother wouldn't feel like that."
"She does," Erna said. "If my father would be home and be all funny
and nice the way he used to be before the war when I was tiny it
would be all right. But he's always at the hospital. He says the only
thing he can do to help people's souls is to try to give them strong,

Textbook Learning Representation For Multi View Data Analysis Models and Applications Zhengming Ding Ebook All Chapter PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Textbook Learning Representation For Multi View Data Analysis Models and Applications Zhengming Ding Ebook All Chapter PDF

Uploaded by

Copyright:

Available Formats

Learning Representation for Multi-View

Data Analysis: Models and Applications

Linking and Mining Heterogeneous and Multi-view Data

Data Analysis in the Cloud : Models, Techniques and

Advanced R Statistical Programming and Data Models:

Machine Learning and Big Data Analytics Paradigms:

Statistical Modeling for Degradation Data 1st Edition

Practical Machine Learning for Streaming Data with

Electrolyzed Water in Food Fundamentals and

Practical Machine Learning for Data Analysis Using

More information about this series at http://www.springer.com/series/4738

ISSN 1610-3947 ISSN 2197-8441 (electronic)

Library of Congress Control Number: 2018961715

© Springer Nature Switzerland AG 2019

This book equips readers to handle complex multi-view data representation,

Indianapolis, IN, USA Zhengming Ding

Part I Unsupervised Multi-view Learning

3.3 Experiment on Synthetic and Real-World Data . . . . . . . . . . . . . 58

Part II Supervised Multi-view Classiﬁcation

6.3.1 Learning Latent Semantic Dictionary . . . . . . . . . . . . . . 131

Part III Transfer Learning

8.4.4 Property Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196

1.1 What Are Multi-view Data and Problem?

Definition 1 (Multi-view Data) (Fig. 1.1): Assume we have a set of data X =

© Springer Nature Switzerland AG 2019 1

1.2 A Unified Perspective

Due to the distribution divergence across different views, view-invariant feature

1.3 Organization of the Book

Part II proposes to solve multi-view classification problems including zero-shot

Chapter 9 develops three novel deep domain adaptation approaches. First, we

2.1 Deep Multi-view Clustering1

Traditional clustering aims to identify groups of “similar behavior” in single view

2.1.2 Deep Semi-NMF Formulation

2.1.2.1 Overview of Semi-NMF

As a variant of NMF, Ding et al. (2010) extended the application of traditional

min X − Z H 2F , (2.1)

Remark 2 Multiple graphs are constructed to constrain the common representation

Z i(v) = (Φ T Φ)−1 Φ T X (v) H̃i(v) T ( H̃i(v) H̃i(v) T )−1

where Φ = [Z 1(v) . . . Z i−1

pos |Mk j | + Mk j neg |Mk j | − Mk j

Proof We introduce the Lagrangian function

where the Lagrangian multiplier η enforces nonnegative constraints, Hm ≥ 0. The

(Hm )kl = (Hm )kl 

Note that Φ T X (v) = [Φ T X (v) ]pos − [Φ T X (v) ]neg ; Φ T Φ Hm = [Φ T Φ Hm ]pos −

The Lagrange function of Eq. (2.12) is written as

2.1.2.3 Time Complexity

Algorithm 2.1: Optimization Solution of Problem (2.3)

2.1.3 Experiments on Face Benchmarks

We choose three face image/video benchmarks in our experiments, as face contains

performance. Then the analytical experiments on three key model parameters β, γ ,

You might also like

(Hm )kl = (Hm )kl