Optimization and Control for Systems

in the Big Data Era Theory and

Applications 1st Edition Tsan-Ming
Choi
Visit to download the full and correct content document:
https://textbookfull.com/product/optimization-and-control-for-systems-in-the-big-data-e
ra-theory-and-applications-1st-edition-tsan-ming-choi/
More products digital (pdf, epub, mobi) instant
download maybe you interests ...

Information Systems for the Fashion and Apparel

Industry 1st Edition Tsan-Ming Jason Choi

https://textbookfull.com/product/information-systems-for-the-
fashion-and-apparel-industry-1st-edition-tsan-ming-jason-choi/

Luxury Fashion Retail Management 1st Edition Tsan-Ming

Choi

https://textbookfull.com/product/luxury-fashion-retail-
management-1st-edition-tsan-ming-choi/

Artificial Intelligence for Fashion Industry in the Big

Data Era Sébastien Thomassey

https://textbookfull.com/product/artificial-intelligence-for-
fashion-industry-in-the-big-data-era-sebastien-thomassey/

Data Deduplication for Data Optimization for Storage

and Network Systems 1st Edition Daehee Kim

https://textbookfull.com/product/data-deduplication-for-data-
optimization-for-storage-and-network-systems-1st-edition-daehee-
kim/
Dissipative Systems Analysis and Control: Theory and
Applications Bernard Brogliato

https://textbookfull.com/product/dissipative-systems-analysis-
and-control-theory-and-applications-bernard-brogliato/

Advanced Optimization for Motion Control Systems 1st

Edition Jun Ma

https://textbookfull.com/product/advanced-optimization-for-
motion-control-systems-1st-edition-jun-ma/

Probabilistic data structures and algorithms for big

data applications Gakhov

https://textbookfull.com/product/probabilistic-data-structures-
and-algorithms-for-big-data-applications-gakhov/

Big Data Analytics Systems Algorithms Applications

C.S.R. Prabhu

https://textbookfull.com/product/big-data-analytics-systems-
algorithms-applications-c-s-r-prabhu/

Advances in Mobile Cloud Computing and Big Data in the

5G Era 1st Edition Constandinos X. Mavromoustakis

https://textbookfull.com/product/advances-in-mobile-cloud-
computing-and-big-data-in-the-5g-era-1st-edition-constandinos-x-
mavromoustakis/
International Series in
Operations Research & Management Science

Tsan-Ming Choi
Jianjun Gao
James H. Lambert
Chi-Kong Ng
Jun Wang Editors

Optimization
and Control for
Systems in the
Big-Data Era
Theory and Applications
International Series in Operations Research
& Management Science

Volume 252

Series Editor
Camille C. Price
Stephen F. Austin State University, TX, USA

Associate Series Editor

Joe Zhu
Worcester Polytechnic Institute, MA, USA

Founding Series Editor

Frederick S. Hillier
Stanford University, CA, USA

More information about this series at http://www.springer.com/series/6161

Tsan-Ming Choi • Jianjun Gao
James H. Lambert • Chi-Kong Ng
Jun Wang
Editors

Optimization and Control for

Systems in the Big-Data Era
Theory and Applications

123
Editors
Tsan-Ming Choi Jianjun Gao
Institute of Textiles and Clothing School of Information Management
The Hong Kong Polytechnic University and Engineering
Hung Hom, Kowloon, Hong Kong Shanghai University of Finance
and Economics
James H. Lambert Shanghai, People’s Republic of China
Department of Systems and Information
Engineering Chi-Kong Ng
University of Virginia Department of Systems Engineering and
Charlottesville, VA, USA Engineering Management
The Chinese University of Hong Kong
Jun Wang Shatin, N.T., Hong Kong
Department of Management Science and
Engineering, Business School
Qingdao University
Shandong, People’s Republic of China

ISSN 0884-8289 ISSN 2214-7934 (electronic)

International Series in Operations Research & Management Science
ISBN 978-3-319-53516-6 ISBN 978-3-319-53518-0 (eBook)
DOI 10.1007/978-3-319-53518-0

Library of Congress Control Number: 2017937015

© Springer International Publishing AG 2017

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology
now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this book
are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or
the editors give a warranty, express or implied, with respect to the material contained herein or for any
errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional
claims in published maps and institutional affiliations.

Printed on acid-free paper

This Springer imprint is published by Springer Nature

The registered company is Springer International Publishing AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
Preface

Nowadays, for both business operations and engineering applications, there are huge
amounts of data that can overwhelm computing resources of large-scale systems.
These “big data” provide new opportunities to improve decision making and address
risk for individuals as well as organizations. For example, the presence of market
and sales data will yield better inventory planning for retail companies; massive
and timely financial data will help improve portfolio management; the security
holes of the Internet and the availability of data affect cryptography and privacy.
Undoubtedly, utilizing big data smartly can enhance decision making. However,
how to use and incorporate data into the decision making framework to yield a
scientifically sound optimal decision is a challenging topic.
Motivated by the importance of big data and the respective challenges in
optimization and control, we have compiled and developed this edited volume
on scientific innovations and reviews in optimization, control, and resilience
management in the big data era.
This book includes several important parts, namely, (1) Reviews on Optimization
and Control Theories, (2) Reviews on Optimization and Control Applications,
(3) Financial Optimization Analysis, (4) Operations Analysis, and (5) Concluding
Remarks. All the featured papers are peer-refereed, and the specific topics covered
include the following:
– Optimization and control for systems in the big data era: an introduction
– Dual control in big data era
– Time inconsistency and self-control optimization
– Quadratic convex reformulations for integer and mixed-integer quadratic pro-
grams
– Measurements of financial contagion
– Asset-liability management in continuous time
– Modern cryptography from the World War II era to the big data era
– Supply risk in the new business era
– A parameterized method for optimal multi-period portfolio selection

v
vi Preface

– Sparse and multiple risk measures approach for data-driven portfolio optimiza-
tion
– Multistage optioned portfolio selection
– Multi-period portfolio selection with stochastic investment horizon
– A new model and method for order selection problems in flow-shop production
– Quick response fashion supply chains in the big data era
– Optimization and control for systems in the big data era: concluding remarks and
future research.
We would like to take this opportunity to express our hearty thanks to Matthew
Amboy and John Wolfe of Springer for their kindest support. We are indebted to
all the reviewers who have provided timely review reports on the manuscripts. We
are grateful for all the authors who have contributed their important and interesting
research to this book.
This book is dedicated to our mentor Professor Duan Li, the Patrick Huen
Wing Ming Professor of Systems Engineering and Engineering Management at The
Chinese University of Hong Kong, to honor his great achievements in both systems
control and optimization and celebrate his 65th birthday in July 2017. In the bottom
of our hearts, he is always a distinguished scholar, a kind gentleman, an excellent
professor, and an outstanding teacher. We are very proud of being his students. As
a remark, the royalty received by the editorial team from this book project is 100%
fully donated to Department of Systems Engineering and Engineering Management,
The Chinese University of Hong Kong.
Hung Hom, Kowloon, Hong Kong Tsan-Ming Choi, PhD
Shanghai, People’s Republic of China Jianjun Gao, PhD
Charlottesville, VA, USA James H. Lambert, PhD
Shatin, N.T., Hong Kong Chi-Kong Ng, PhD
Shandong, People’s Republic of China Jun Wang, PhD
November 2016
Contents

1 Optimization and Control for Systems in the Big Data Era: An

Introduction .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 1
Tsan-Ming Choi, Jianjun Gao, James H. Lambert, Chi-Kong Ng,
and Jun Wang

Part I Reviews on Optimization and Control Theories

2 Dual Control in Big Data Era: An Overview . . . . . . . .. . . . . . . . . . . . . . . . . . . . 9
Peilin Fu
3 Time Inconsistency and Self-Control Optimization Problems:
Progress and Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 33
Yun Shi and Xiangyu Cui
4 Quadratic Convex Reformulations for Integer
and Mixed-Integer Quadratic Programs.. . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 43
Baiyi Wu and Rujun Jiang

Part II Reviews on Optimization and Control Applications

5 Measurements of Financial Contagion: A Primary Review
from the Perspective of Structural Break .. . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 61
Xi Pei and Shushang Zhu
6 Asset-Liability Management in Continuous-Time:
Cointegration and Exponential Utility . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 85
Mei Choi Chiu
7 A Review of Modern Cryptography: From the World War II
Era to the Big-Data Era .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 101
Bojun Lu

vii
viii Contents

8 Modeling Supply Risk in the New Business Era: Supply Chain

Competition and Cooperation.. . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 121
Xiang Li, Yongjian Li, and Linghua Zhao

Part III Financial Optimization Analysis

9 A Parameterized Method for Optimal Multi-Period
Mean-Variance Portfolio Selection with Liability . . .. . . . . . . . . . . . . . . . . . . . 147
Xun Li, Zhongfei Li, Xianping Wu, and Haixiang Yao
10 Sparse and Multiple Risk Measures Approach for Data Driven
Mean-CVaR Portfolio Optimization Model. . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 167
Jianjun Gao and Weiping Wu
11 Multistage Optioned Portfolio Selection: Mean-Variance
Model and Target Tracking Model . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 185
Jianfeng Liang
12 Multi-Period Portfolio Selection with Stochastic
Investment Horizon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 217
Lan Yi

Part IV Operations Analysis

13 A New Model and Method for Order Selection Problems
in Flow-Shop Production . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 245
Jun Wang, Xiaoxia Zhuang, and Baiyi Wu
14 Quick Response Fashion Supply Chains in the Big Data Era . . . . . . . . . 253
Tsan-Ming Choi

Part V Concluding Remarks

15 Optimization and Control for Systems in the Big Data Era:
Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 271
Tsan-Ming Choi, Jianjun Gao, James H. Lambert, Chi-Kong Ng,
and Jun Wang

Index . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 277
Contributors

Mei Choi Chiu Department of Mathematics and Information Technology, The

Education University of Hong Kong, Tai Po, Hong Kong
Tsan-Ming Choi Institute of Textiles and Clothing, The Hong Kong Polytechnic
University, Hung Hom, Kowloon, Hong Kong
Xiangyu Cui School of Statistics and Management, Shanghai University of
Finance and Economics, Shanghai, People’s Republic of China
Peilin Fu Department of Applied Engineering, School of Engineering and
Computing, National University, San Diego, CA, USA
Jianjun Gao School of Information Management and Engineering, Shanghai
University of Finance and Economics, Shanghai, People’s Republic of China
Rujun Jiang Department of Systems Engineering and Engineering Management,
The Chinese University of Hong Kong, Shatin, N.T., Hong Kong
James H. Lambert Department of Systems and Information Engineering, Univer-
sity of Virginia, Charlottesville, VA, USA
Xiang Li College of Economic and Social Development, Nankai University,
Tianjin, People’s Republic of China
Xun Li Department of Applied Mathematics, The Hong Kong Polytechnic Univer-
sity, Hong Kong, China
Yongjian Li Business School, Nankai University, Tianjin, People’s Republic of
China
Zhongfei Li Department of Finance and Investment, Sun Yat-Sen Business School,
Sun Yat-Sen University, Guangzhou, China
Jianfeng Liang Department of Finance, Lingnan (University) College, Sun Yat-
sen University, Guangzhou, People’s Republic of China

ix
x Contributors

Bojun Lu Portfolio Management Department, Foresea Life Insurance Co., Ltd.,

Shenzhen, China
Chi-Kong Ng Department of Systems Engineering and Engineering Management,
The Chinese University of Hong Kong, Shatin, N.T., Hong Kong
Xi Pei Department of Finance and Investment, Sun Yat-Sen Business School, Sun
Yat-Sen University, Guangzhou, China
Yun Shi School of Management, Shanghai University, Shanghai, People’s Repub-
lic of China
Jun Wang Department of Management Science and Engineering, Business School,
Qingdao University, Shandong, People’s Republic of China
Baiyi Wu School of Finance, Guangdong University of Foreign Studies,
Guangzhou, People’s Republic of China
Weiping Wu Department of Automation, Shanghai Jiao Tong University, Shang-
hai, People’s Republic of China
Xianping Wu School of Mathematical Sciences, South China Normal University,
Guangzhou, China
Haixiang Yao School of Finance, Guangdong University of Foreign Studies,
Guangzhou, China
Lan Yi Management School, Jinan University, Guangzhou, People’s Republic of
China
Linghua Zhao College of Economic and Social Development, Nankai University,
Tianjin, People’s Republic of China
Shushang Zhu Department of Finance and Investment, Sun Yat-Sen Business
School, Sun Yat-Sen University, Guangzhou, China
Xiaoxia Zhuang Department of Management Science and Engineering, Business
School, Qingdao University, Shandong, People’s Republic of China
Chapter 1
Optimization and Control for Systems in the Big
Data Era: An Introduction

Tsan-Ming Choi, Jianjun Gao, James H. Lambert, Chi-Kong Ng,

and Jun Wang

Abstract The big data era is characterized by the presence of many Vs in terms of
data and data usage. In this introductory chapter, we first discuss some challenges
in optimization and control for systems in the presence of massive amount of data.
We then introduce the papers featured in this book.

Keywords Introduction • Big data • Optimization • Control

T.-M. Choi ()

Institute of Textiles and Clothing, The Hong Kong Polytechnic University,
Hung Hom, Kowloon, Hong Kong
e-mail: jason.choi@polyu.edu.hk
J. Gao
School of Information Management and Engineering, Shanghai University of Finance
and Economics, Shanghai, People’s Republic of China
e-mail: gao.jianjun@shufe.edu.cn
J.H. Lambert
Department of Systems and Information Engineering, University of Virginia,
Charlottesville, VA, USA
e-mail: lambert@virginia.edu
C.-K. Ng
Department of Systems Engineering and Engineering Management, The Chinese University
of Hong Kong, Shatin, N.T., Hong Kong
e-mail: ckng@se.cuhk.edu.hk
J. Wang
Department of Management Science and Engineering, Business School, Qingdao University,
Shandong, People’s Republic of China
e-mail: jwang@qdu.edu.cn

© Springer International Publishing AG 2017 1

T.-M. Choi et al. (eds.), Optimization and Control for Systems in the Big-Data Era,
International Series in Operations Research & Management Science 252,
DOI 10.1007/978-3-319-53518-0_1
2 T.-M. Choi et al.

1.1 Optimization and Control in Big Data Era

We are now in the big data era. The popularity of social media, mobile devices,
cloud storage and application services, etc., all means that the amount of data which
can be collected and used by organizations and companies is increasing everyday
(Chan et al. 2016; Wang et al. 2016).
Traditionally, the term “big data” is associated with many Vs (Choi et al.
2016), such as volume (the amount of data), velocity (speed of data collection and
processing), variety (the structured and unstructured data; complex data), veracity
(the data accuracy and uncertainty) and value (the value associated with data).
Undoubtedly, the presence of a massive amount of data means that we have to
rethink about the strengths and weaknesses of the existing optimization and control
methods.
In the recent literature, big data related optimization and control problems have
been examined. For instance, Facchinei and Scutari (2015) propose a decomposition
framework for achieving parallel optimization for a class of nonconvex problems
with a massive amount of data. The authors demonstrate that their proposed method
outperforms other existing methods. Daneshmand et al. (2015) develop a novel
hybrid and parallel decomposition scheme for solving convex and nonconvex big
data optimization problems. For very big problems, the authors show that their
proposed decomposition scheme works well compared to other random or determin-
istic schemes. Bhattacharya et al. (2016) explore how an evolutionary optimization
based algorithm can handle optimization problems with a high volume dataset. The
authors claim that they have successfully applied the proposed algorithm in real
world financial portfolio management. Richtarik and Takac (2016) explore how
parallel randomized block coordinate descent methods can be used for developing
a big data optimization algorithm. The authors show that their proposed algorithm
can solve a class of large scale problems efficiently. Most recently, Boone et al.
(2016) develop a framework for exploring service parts performance optimization
problems with big data. They propose how, where and why big data applications
can be applied in their proposed framework. For more recent developments of big
data optimization, refer to the books by Emrouznejad (2016) and Japkowicz and
Stefanowski (2016).
In this introductory chapter, we briefly review the papers featured in this book.
According to the sectioning of the book, we present the papers in four sections: Sect.
1.2 reports the papers in the “Reviews on Theories” section, Sect. 1.3 examines the
papers in the “Reviews on Applications” section, Sect. 1.4 introduces the “Financial
Optimization Analysis” papers and Sect. 1.5 describes the papers in “Operations
Analysis”.
1 Optimization and Control for Systems in the Big Data Era: An Introduction 3

1.2 Reviews on Theories

For the review on optimization and control theories, this book features three
papers. First, Fu examines the dual control theory in the big data era. The author
first presents an overview of dual control and its probable applications. She
explores different dual and non-dual controllers and highlights their complexity and
limitations. In particular, she focuses on a class of discrete-time LQG problems
with unknown parameters. She shows the optimal dual control, active open-loop
feedback control via variance minimization (Li et al. 2002) and optimal nominal
dual control for this class of problems. Finally, the author also discusses the
probable usage of dual control in economic systems, information retrieval as well
as mechanical engineering.
Shi and Cui review the time inconsistency and self control optimization prob-
lems, which are commonly found in financial optimization and conflict decision
making (Cui et al. 2012). The authors examine different approaches which can
effectively deal with the time inconsistency challenge in decision making. More-
over, they report the recent progress in the area and mention the challenges of time
inconsistency optimization in the big data era.
Wu and Jiang examine the recent developments in the quadratic convex refor-
mulation method. In fact, the quadratic convex reformulation method is commonly
used to derive efficient equivalent reformulations for mixed-integer quadratically
constrained quadratic optimization problems. The authors comment that even
though the proposed problems can be solved by a standard mixed-integer quadratic
solver using the branch-and-bound method, the solver’s performance is far from
satisfactory. They thus argue that the quadratic convex reformulation approach
provides a systematic way to solve the above-mentioned challenging optimization
problems. The authors also review some recent extensions of the quadratic con-
vex reformulation method for problems such as the challenging semi-continuous
quadratic optimization problems.

1.3 Reviews on Applications

For the review on applications of optimization and control methods in the big data
era, there are four related papers in this book.
Pei and Zhu explore measurements of financial contagion. The authors propose
that the financial contagion is a timely issue which is closely related to financial
systemic risk. In their paper, they first clarify and summarize various critical
concepts and measurements of financial contagion and then highlight their common
features and differences. Since the structural break is known to be especially crucial,
the authors review and discuss the respective financial contagion measurements. The
authors conclude that the big data technology may be helpful for advancing risk
management relevant to financial contagion in, e.g., information acquisition as well
as model specification.
4 T.-M. Choi et al.

Dynamic portfolio optimization is a critical application area of advanced opti-

mization models and methods. Chiu reports the summary of recent advances in the
optimal asset-liability management system, which employs the dynamic portfolio
optimization technique, in a continuous-time domain. From the stochastic optimal
control perspective, the author derives a new asset-liability management solution for
the case with insurers possessing a constant absolute risk averse utility function and
Poisson-type insurance liabilities.
Cryptography is a very interesting area for secret communication. Traditionally,
cryptography focuses on confidentiality, integrity and authentication of information.
Lu reports in her paper a review of modern cryptography. The author chooses
two important systems in cryptography, namely the Merkle–Hellman knapsack
cryptography system and the subset-sum problem based cryptography system in
the review. She examines the encryption and decryption processes of the above two
systems.
In supply risk analysis, Li et al. conduct a systematic review on supply risk
modeling. From the operations management perspective, the authors present various
recent developments in supply risk modeling which include vertical supply chain
interaction, horizontal supply chain competition and network competition problems.
The authors present analytical models for each scenario. They also discuss future
research directions in the big data era.

1.4 Financial Optimization Analysis

In addition to reviews on optimization and control theories and applications, this

book also features many technical papers with novel insights and new findings. In
the area of financial optimization, there are four related papers.
First, Li et al. develop a parameterized method for achieving optimal multi-
period mean-variance portfolio selection with liability considerations. The authors
note that the financial market generates a massive amount of data which are related
to portfolio management. To effectively select the optimal portfolio requires very
careful planning and the use of an efficient method. The authors hence propose a
new method to help derive the analytical optimal portfolio strategies and efficient
frontiers accurately. The authors demonstrate the applicability of their work by
presenting a numerical example.
Gao and Wu explore the data driven mean-CVaR (DDMC) portfolio optimization
problem. The authors consider the case when the out of sample performance of the
DDMC portfolio optimization model is unstable, which occurs in practice owing to
the availability of historical data. To deal with this challenge, the authors propose
a novel method by adding a penalty on the sparsity of the portfolio weight and
combine the variance term in the DDMC model. The authors run a numerical
analysis and confirm that the out-of-sample performance fragility is being well
mitigated using the new method.
1 Optimization and Control for Systems in the Big Data Era: An Introduction 5

Options are critical in the financial market. Liang conducts an analysis on the
multi-stage optioned portfolio selection problem. The author introduces the mean-
variance models first and then establishes the target tracking model for the optioned
portfolio selection problem in both dynamic and static settings. She proposes
two different solution schemes, namely the stochastic programming approach with
optimality condition scheme, and the stochastic control with dynamic programming
scheme. She also reveals the closed form relationship between the mean-variance
model and the target tracking model.
Yi studies the multi-period portfolio selection problem with a stochastic invest-
ment horizon. She argues that the problem is commonly seen in the real world as
an investor may suddenly terminate an investment (which leads to the stochastic
investment horizon) owing to various factors. The author finds that the formulated
problem is non-separable in the sense of dynamic programming. She thus employs
an embedding technique to derive the optimal policy.

1.5 Operations Analysis

Finally, in this book, two papers on operations analysis are featured. First, Wang
et al. present a new model and a novel method to solve the order selection
problems in flow-shop operations. The authors notice that traditional order selection
models separate the production scheduling process and the order selection problem,
and the performance of order selection solely depends on production scheduling.
The authors hence propose a new model which simultaneously considers both
production scheduling and order selection. By computational experiments, the
authors demonstrate that the proposed new model is much better than the traditional
ones.
Quick response is a well-established strategy in supply chain management (Choi
et al. 2003, 2004, 2006). Under quick response, companies can better utilize market
information to improve their forecasts. In the big data era, it is easier for companies
to collect and use a large amount of data. Motivated by the presence of massive
amount of data and the importance of quick response in supply chain operations,
Choi explores quick response in fashion supply chains. He focuses his attention on
how the number of observations affects the expected values of quick response for
the centralized supply chain system as well as the individual supply chain agents in
the decentralized setting. He proves that quick response is a beneficial practice for
the centralized supply chain system as well as the fashion retailer, and the benefit
is increasing in the number of observations. If the number of observations goes
to infinity, resembling the presence of “big data”, the expected values of quick
response will go to the steady states and the analytical expressions are found.
However, there exist cases in which the fashion supplier suffers from a loss after
adopting quick response. As such, the author derives a wholesale pricing markdown
contract to achieve win–win coordination, which means both the supplier and the
fashion retailer are better off (i.e. “win–win”) and the supply chain is simultaneously
globally optimized (i.e. “coordinated”).
6 T.-M. Choi et al.

References

M. Bhattacharya, R. Islam, J. Abawajy, Evolutionary optimization: a big data perspective. J.

Network Comput. Appl. 59, 416–426 (2016)
C.A. Boone, B.T. Hazen, J.B. Skipper, R.E. Overstreet. A framework for investigating optimization
of service parts performance with big data. Ann. Oper. Res. (2016), in press
H.K. Chan, T.M. Choi, X. Yue, Big data analytics: risk and operations management for industrial
applications. IEEE Trans. Ind. Inf. 12, 1214–1218 (2016)
T.M. Choi, D. Li, H. Yan, Optimal two-stage ordering policy with Bayesian information updating.
J. Oper. Res. Soc. 54(8), 846–859 (2003)
T.M. Choi, D. Li, H. Yan, Optimal single ordering policy with multiple delivery modes and
Bayesian information updates. Comput. Oper. Res. 31, 1965–1984 (2004)
T.M. Choi, D. Li, H. Yan, Quick response policy with Bayesian information updates. Eur. J. Oper.
Res. 170, 788–808 (2006)
T.M. Choi, H.K. Chan, X. Yue, Recent development in big data analytics for
business operations and risk management. IEEE Trans. Cybern. 47, 81–92 (2016).
doi:10.1109/TCYB.2015.2507599
X.Y. Cui, D. Li, S.Y. Wang, S.S. Zhu, Better than dynamic mean-variance: Time inconsistency and
free cash flow stream. Math. Financ. 22(2), 346–378 (2012)
A. Daneshmand, F. Facchinei, V. Kungurtsev, G. Scutari, Hybrid random/deterministic parallel
algorithms for convex and nonconvex big data optimization. IEEE Trans. Signal Process.
63(15), 3914–3929 (2015)
A. Emrouznejad (ed.), Big Data Optimization: Recent Developments and Challenges, Series on
Studies in Big Data (Springer, Switzerland, 2016)
F. Facchinei, G. Scutari, Parallel selective algorithms for nonconvex big data optimization. IEEE
Trans. Signal Process. 63(7), 1874–1889 (2015)
N. Japkowicz, J. Stefanowski, Big Data Analysis: New Algorithms for a New Society, Series on
Studies in Big Data (Springer, Switzerland, 2016)
D. Li, F.C. Qian, P.L. Fu, Variance minimization approach for a class of dual control problems.
IEEE Trans. Autom. Control 47(12), 2010–2020 (2002)
P. Richtarik, M. Takac, Parallel coordinate descent methods for big data optimization. Math.
Program. Ser. A 156, 433–484 (2016)
X. Wang, Y. He, Learning from uncertainty for big data. IEEE Syst. Man Cybernet. Mag. 2(2),
26–32 (2016)
 Part I
Reviews on Optimization
and Control Theories
Chapter 2
Dual Control in Big Data Era: An Overview

Peilin Fu

Abstract This paper provides an overview of dual control and its applications
in the big data era. Different non-dual and dual controllers as well as their
attributes, complexity, and limitations are analyzed. As an example, dual control
of a class of discrete-time LQG problems with unknown parameters in both the
state and observation equations is discussed in depth. Optimal dual control, open-
loop feedback control, active open-loop feedback control via variance minimization
approach, and optimal nominal dual control are demonstrated for this type of
problems. The optimal nominal dual control, taking into account the effect of
future learning, is the best possible (partial) closed-loop feedback control that can
be achieved. Applications of dual control in economic systems, manufacturing
processes, information retrieval, robotics, etc. are also introduced.

Keywords Dual control • Stochastic control • Dynamic programming • LQG

control problem

2.1 Introduction

Most real-world processes are very complex and are not well understood. As such,
the control of systems whose dynamics are not completely known is a problem of
major theoretical and practical importance. Feldbaum, in his seminal work in 1960s
(Feldbaum 1965), pointed out that, when implementing the optimal control strategy
for stochastic systems with parameter uncertainty, the controller usually pursues
two often conflicting objectives: to drive the system toward a desired state, and to
perform active learning to reduce the system’s uncertainty. Such a control scheme,
which affects not only the states of the system but also the quality of estimation,
is known as dual control. In 2000, IEEE Control Systems Society listed the dual
control as one of the 25 most prominent subjects in the last century which had
significantly impacted the development of control theory.

P. Fu ()
Department of Applied Engineering, School of Engineering and Computing,
National University, San Diego, CA, USA

© Springer International Publishing AG 2017 9

T.-M. Choi et al. (eds.), Optimization and Control for Systems in the Big-Data Era,
International Series in Operations Research & Management Science 252,
DOI 10.1007/978-3-319-53518-0_2
10 P. Fu

Except for a few ideal situations, the optimal dual control cannot be achieved
both analytically and numerically. Feldbaum showed that the optimal dual control is
the solution to a functional equation known as Bellman equation based on dynamic
programming. Solving this functional equation is intractable due to the “curse
of dimensionality” inherent in dynamic programming. The two subproblems of
stochastic control, estimation and control in most situations are intercoupled. The
future uncertainties of the parameters are functions of the control signals applied
to the system. The loss function, which has to be minimized with respect to the
control signal, thus contains some information of the future observations through
the statistics of the observations given the present information (Bar-Shalom and
Tse 1974). The efforts in dual control have thus mainly been devoted to developing
certain suboptimal solution schemes, such as the certainty equivalence scheme and
open-loop feedback control, by bypassing this essential feature of coupling between
estimation and control.
The control policies were categorized into the following classes in Bar-Shalom
and Tse (1974) according to their information patterns—the availability of past
observation and the possible usage of information about the future observation:
(1) The Open-loop Policy. In this case no measurement knowledge is available for
the controller.
(2) The Feedback Policy. At every time the current information set is available for
the computation of the control but no knowledge about the future measurements
is available. The open-loop optimal feedback (OLOF) control belongs to the
feedback class. It assumes that no observations will be made in the future, the
control law is obtained by using the observations already acquired.
(3) The Closed-Loop Policy. This policy incorporates with the remaining observa-
tion program, i.e. the knowledge that the loop will stay closed through the end
of the process is fully utilized.
There are two aspects in which the closed-loop policy differs from the feedback
policy (Bar-Shalom and Tse 1974).
(1) Caution: In a stochastic control problem, due to the inherent uncertainties,
the controller has to be “cautious” not to increase the effect of the existing
uncertainties on the cost. However, the closed-loop controller, since it “knows”
that future observations will be available and corrective actions based upon
them will be taken, will exercise less “caution.”
(2) Probing or Active Learning: When the dual effect is present, the control
can “help” in learning (estimation) by decreasing the uncertainty about the
state. Therefore, the closed-loop control, which takes into account the future
observation program and statistics, has the capability of active learning when
the dual effect exists. A feedback controller, even though it “learns” by using
the measurements, does not actively “help” the learning. This learning can be
called, therefore, passive, or accidental, and the corresponding control policy
is passively adaptive, as opposed to the closed-loop control which is actively
adaptive.
2 Dual Control in Big Data Era: An Overview 11

Most resulting suboptimal control laws are of a nature of passive learning, since
the function of future active probing of the control is purposely deprived in order
to achieve analytical attainability in the solution process. A central problem in dual
control, and indeed a key barrier to its development, is to power a control law with
the property of active learning.
Prominent features and fundamental properties of dual control have been exten-
sively studied in the literature (Bar-Shalom 1981; Bar-Shalom and Tse 1974; Tse
et al. 1973). An analysis of various approximations in dual control was given
by Lindoff et al. (1999). Filatov and Unbehauen (2000) developed a bi-criteria
approach to cope with the two conflicting goals in dual control. Surveys on dual
control can be found in Wittenmark (1975c) and Filatov and Unbehauen (2000).
Li, Qian, and Fu in a series of papers studied the dual control of discrete-
time LQG problems with unknown parameters. A variance minimization approach
was proposed for discrete-time LQG problems with parameters uncertainty in the
observation equation (Li et al. 2002). Minimizing a covariance term at the final
stage introduced a feature of active learning for the derived control law. The optimal
degree of active learning was determined for achieving an optimality. Fu et al.
(2002) further applied the variance minimization approach to discrete-time LQG
problems with parameters uncertainty in both the state and observation equations,
an optimal open-loop feedback control law with active learning property was
developed. The same problem was revisited in Li et al. (2008), in which the optimal
nominal dual control was proposed. By exploring the future nominal posterior
probabilities, the control law takes into account the function of future learning, thus
is the best possible closed-loop feedback control that can be achieved. Some of these
results are summarized in Sect. 2.3 as an example of dual control problems.
In Sect. 2.2, classification of controllers is introduced. Different non-dual and
dual controllers as well as their attributes, complexity, and limitations are analyzed.
As an example, dual control of a class of discrete-time LQG problems with
unknown parameters in both the state and observation equations is discussed
in depth in Sect. 2.3. Optimal dual control, open-loop feedback control, active
open-loop feedback control, and optimal nominal dual control are demonstrated.
Section 2.4 provides successful applications of dual control in economic systems,
manufacturing processes, information retrieval etc. in the big data era. The paper
concludes in Sect. 2.5.

2.2 Classification of Controllers

2.2.1 Non-dual Controller

If the performance index only takes into account the previous measurements
and does not assume that future information will be available, then the resulting
controller will be called non-dual in Feldbaum’s terminology. In this situation
the control law does not facilitate the identification. The non-dual controllers can
12 P. Fu

be divided into three classes: certainty equivalence controller, one-step cautious

controller, and open-loop optimal feedback controller.

2.2.1.1 Certainty Equivalence Controller

One widely used non-dual approach is developed using the concept of certainty
equivalence. The certainty equivalence holds if it is possible to first solve the
deterministic problem with known parameters and then obtain the optimal controller
for unknown parameters by substituting the true parameter values with the estimated
values (Wittenmark 1975c). One well-known class of problems for which the cer-
tainty equivalence principle holds is the linear-quadratic-gaussian control problems.
In adaptive control there are very few cases where the certainty equivalence prin-
ciple is applicable. The controller obtained by enforcing the certainty equivalence
principle does not take into consideration the fact that the estimated parameters
are not equal to the true ones and are inaccurate. Although the simplicity of the
control law, it ignores the confidence level of the parameter estimates in deriving
the adaptive control scheme. Such a control scheme would result in a control system
that is extremely sensitive to stochastic variations.
A method based on process parameter estimation was first described by Kalman
(1958) using least squares to determine the unknown parameters in the model.
This type of method works well for constant or slow time-varying parameters.
Different approximation methods (Hasting-James and Sage 1969; Panuska 1968;
Young 1968) have been suggested for the models of maximum likelihood type.
Methods using state space models were given in Jenkins and Roy (1966) and Luxat
and Lees (1973).

2.2.1.2 One-Step Cautious Controller

Minimizing over a single time period leads to the one-step cautious controller. This
controller takes the parameter uncertainties into account, in contrast to the certainty
equivalence controller. However, the controller of this type may generate the turn-
off phenomenon. If the estimates are very poor, the magnitude of the control signal
will become very small. The control is thus unintentionally turned off for some
period of time until the noise excites the system in such a way that better estimates
are achieved. This makes the one-step cautious controller unsuitable for control of
systems with quickly varying parameters.
A one-step minimization where the unknown parameters are modeled by a
stochastic process was discussed in Aoki (1967) and Astrom and Wittenmark
(1971). The unknown parameters can be first estimated using a Kalman filter,
which then give the one-step ahead estimates and covariance matrix based on
the current information set. Using a fundamental lemma in stochastic optimal
control (Astrom 1970), it is possible to find the control law to solve the one-
step minimization problem. The control law clearly shows the influence of the
uncertainties of the estimates. Examples with turn-off were given, for instance, by
Astrom and Wittenmark (1971).
2 Dual Control in Big Data Era: An Overview 13

2.2.1.3 The Open-Loop Feedback Optimal Controller

The open-loop optimal feedback (OLOF) control is derived at distinct time instants
under the assumption that no future measurements will be available. Thus an
open-loop control sequence is determined. The first step in the control sequence
is then used and the performance of the system is measured. Based on the new
information (feedback), a new minimization is performed again. In this open-
loop feedback approach, the fact that the estimated parameter may not be exact
is therefore taken into consideration, but the knowledge of the future observation
program is completely ignored. According to the theory of dual control, introduced
by Feldbaum (1965), the open-loop feedback control is, from the estimation point
of view, passive, since it does not take into account that learning is possible in the
future.
Many suboptimal controllers achieved in the current literature are open-loop
optimal feedback controllers (Florentine 1962; Tse and Athans 1972; Aoki 1967).
Lainiotis, Deshpande, and Upadhyay wrote a series of papers (Deshpande et al.
1973; Lainiotis et al. 1972) on an open-loop feedback optimal approach to the
stochastic control of linear systems with unknown parameters. The controller is
designed to minimize the average performance-to-go conditioned on the present
measurements and past control actions and without any active anticipation of new
measurements. The result is a feedback control law similar to the optimal LQG
one, but averaged over the space of the unknown parameters. The algorithm is
straightforward and easy to implement. It may be generated by computing the
average of the specific controllers for some value of the parameters weighted by
the a posteriori probability densities which are Gaussian (Deshpande et al. 1973).
Casiello and Loparo proved in Casiello and Loparo (1989) that these types of passive
control laws are optimal for certain quadratic functionals.
The OLOF controller might be overly cautious because of the assumption that no
further measurements will be available to correct for erroneous control actions. The
properties of the OLOF controller were further discussed by Bar-Shalom and Sivan
(1969) and Tse and Athans (1972).

2.2.2 Dual Controller

If besides the previous measurements the performance index is also considered to be

dependent on the future observations, a dual controller will be constructed. In this
case, the future uncertainties of the parameters are functions of the control applied
to the system. The control law must compromise between the two conflicting tasks:
control and identification. The dual controllers can be classified as optimal dual
controller and suboptimal dual controller.
14 P. Fu

2.2.2.1 Optimal Dual Controller

There are very few cases where it is possible to obtain an analytical representation
of the optimal dual control law. The imposed assumptions are usually unrealistic. In
Gorman and Zaborszky (1968) and Grammaticos and Horowitz (1970), the problem
of controller synthesis was considered under the assumption that the entire state is
measurable. Moreover, it was assumed that the poles of the system are known while
the zeros are unknown. In both cases, it is possible to find the optimal dual control
by solving a set of differential or difference equations corresponding to the Riccati
equation in the standard linear quadratic case. Sternby (1978) discussed a Markov
chain with four states. The transition probabilities are functions of the control. In
that particular example it is possible to find the analytical expression of the optimal
dual controller.
Some results can be seen in the literature to achieve the optimal dual controller
numerically. Florentine (1962) considered a first-order system where the gain is
fixed but unknown with a given a priori distribution. The problem was solved by
discretizing the state and control. Another numerically solved problem was given by
Jacobs and Langdon (1970). The absolute value of the state can be measured through
the observation while the sign is unknown. Introducing the probability for the state
to be positive, it is possible to derive the corresponding functional equation. A zero-
order system with an unknown gain was considered in Astrom and Wittenmark
(1971) where the gain was assumed to be described by a known stochastic process.
A more general treatment of the problem was given by Griffiths and Loparo (1985).
A variance minimization approach for dual control of discrete-time LQG prob-
lems with parameter uncertainty in the observation equation was proposed by
Li et al. (2002). Minimizing a covariance term at the final stage introduced a feature
of active learning for the derived control low. The optimal degree of active learning
was derived for achieving the optimality.

2.2.2.2 Suboptimal Dual Controller

Since it is difficult to determine the optimal dual controllers, much effort has been
devoted to finding suboptimal solutions with dual properties. The approaches can
be classified as follows (Wittenmark 1975c; Astrom and Wittenmark 1989):
(1) Perturbation signals
Employing a cautious controller can give rise to “turn-off” of the control
if the unknown parameters are strongly time-varying. Several ways have been
suggested to avoid the turn-off phenomenon. The turn-off is due to a lack of
excitement. The perturbation signal, which can be a square-wave or pseudo
random signal, etc., can be used to excite the system in order to get good
estimation (Wieslander and Wittenmark 1971). The addition of the extra signal
will naturally increase the probing loss, but may make it possible to improve
the total performance.
2 Dual Control in Big Data Era: An Overview 15

(2) Constrained one-step-ahead minimization

Another way to avoid turn-off is to minimize the loss function one-step ahead
under certain constraints. The constraints such as limiting the minimum value of
the control signal or limiting the variance of the parameter estimates can prevent
the control signal from being too small and impose extra probing (Hughes and
Jacobs 1974; Alster and Belanger 1974). These controllers have the advantage
that the control signal can be easily computed, but the algorithm will contain
application-dependent parameters that have to be chosen by the user.
(3) Approximations of the loss function
Suboptimal dual controls can also be obtained by extending the loss function
in order to prevent the shortsightedness of the cautious controller (Astrom and
Wittenmark 1989). For state space models, one approach is to make a serial
expansion of the loss function in the Bellman equation (Gorman and Zaborszky
1968). Such an expansion can be done around the certainty equivalence or the
cautious controllers. But due to its computational complexity, this approach has
been limited to situations where the control horizon is rather short, usually less
than 10.
Another way is to try to solve the two-step minimization problem. The
derived suboptimal control has correction terms which depend on the sensitivity
functions of the expected future cost, which can avoid the turn-off. But in most
cases, it is not possible to get an analytical solution.
(4) Modifications of the loss function
Adding terms that are reflecting quality of the parameter estimate in the loss
function can prevent the cautious controller from turning off. Solution proposals
(Alster and Langer 1974; Wittenmark 1975b; Milito et al. 1982) have been
seen in the literature to incorporate certain variance terms of the state or the
innovation process into the objective function in order to force the control to
perform active learning. These solution schemes, however, truncate the time
horizon into shorter time periods of one stage, prompting a concern of possible
myopic behaviors.

2.2.2.3 Optimal Nominal Dual Controller

Although the optimal nominal dual controller is also suboptimal, the author would
like to list it as a separate category to distinguish it from other suboptimal dual
controllers. The reason is that it is the best possible closed-loop dual control if
the optimal dual control cannot be achieved. The optimal nominal dual control
was first proposed by Li, Qian and Fu in Li et al. (2008). They pointed out
that a major difficulty in solving dual control for discrete-time LQG problems
with unknown parameters is that the optimal control cannot be determined when
the future posterior probabilities are unknown, while at the same time the future
posterior probabilities depend on the control applied at the early stages. In order
to break this loop, a possible solution scheme is to derive the relationship between
the posterior probability and the control. A control which satisfies a deterministic
16 P. Fu

version of this relationship is defined as the nominal control. The expected posterior
probabilities when applying the nominal control are called nominal future posterior
probabilities. Applying the nominal future posterior probabilities generated by the
nominal control in the Bellman equation, the effect of future learning can be taken
into account. Since in this situation, all the achievable future information is used in
terms of its expected value, the control law obtained can be considered to be the best
possible closed-loop control law in this sense.

2.3 An Example: LQG Problems with Unknown Parameters

Consider the following class of linear-quadratic stochastic optimal control problems

where there exist parameter uncertainties in both the state and the observation
equations,
( )
X
N1
0 0 0 0
.P/ min E x .N/Q.N/x.N/ C Œx .k/Q.k/x.k/ C u .k/R.k/u.k/ j I
kD0

s:t: x.k C 1/ D A.k; /x.k/ C B.k; /u.k/ C w.k/; k D 0; 1;    ; N  1

y.k/ D C.k; /x.k/ C v.k/; k D 1; 2;    ; N;

where x .k/ 2 Rn is the state, u .k/ 2 Rp is the control, y .k/ 2 Rm is the

measured output, and I 0 is the initial information set that includes information
about the probability distribution of the initial state x.0/, the statistics of the random
sequences fw.k/g and fv.k/g, and the initial probability distribution of the unknown
parameter . fw .k/g 2 Rn and fv .k/g 2 Rm are the two independent Gaussian white
noise sequences with zero mean, and variances w2 and v2 , respectively. The random
initial state x .0/ is assumed to be of Gaussian distribution N .Ox .0/ ; P .0// and is
assumed to be independent of the process and observation noises: The quantities
A .k; / ; B .k; /, and C .k; / are matrices of appropriate dimensions whose values
depend on an unknown parameter . It is assumed that  belongs to a finite set
‚ = f1 ; 2 ; : : : ; s g and is a constant over the entire time horizon. The a priori
probabilities of the parameter  are

qi .0; I 0 / D P. D i j I 0 /; i D 1; 2; : : : ; s:

Furthermore, fQ .k/g and fR .k/g are sequences of positive semidefinite and positive
definite symmetric matrices of appropriate dimensions, respectively. Define the
information set at stage k, k = 0, 1, : : :, N, to be I k ,
˚ 
I k D u .0/ ; : : : ; u .k  1/ ; y .1/ ; : : : ; y .k/ ; I 0 :
2 Dual Control in Big Data Era: An Overview 17

The dual control problem for (P) is to find a closed-loop control law,
 
u .k/ D fk I k ; k D 0; 1; : : : ; N  1;

such that the expected performance index in (P) is minimized.

Notice that two kinds of uncertainty are involved in (p): irreducible uncertainty
caused by Gaussian white noise sequences fw.k/g and fv.k/g, and reducible
uncertainty caused by an unknown parameter . If there is no parameter uncertainty
about , the above problem reduces to the conventional linear-quadratic Gaussian
stochastic control problem which is not a dual control problem since the control does
not have an effect on the system’s uncertainty. The certainty equivalence principle
then can be applied to determine the optimal control. Note that the certainty
equivalence principle may not hold even for some stochastic control problems
with only irreducible uncertainty, for example, linear Gaussian systems with an
exponential performance criterion (Jacobson 1973).

2.3.1 Optimal Dual Control

Define xO i .kjk/ to be the state estimate at stage k when assuming  = i :

˚ 
xO i .kjk/ D E x.k/j D i ; I k :

xO i .kjk/ can be obtained using the Kalman filters as stated in Casiello and Loparo
(1989):

xO i .kjk/ D xO i .kjk  1/ C Fi .k/ Œy.k/  C.k; i /Oxi .kjk  1/ (2.1)

xO i .kjk  1/ D A.k  1; i /Oxi .k  1jk  1/ C B.k  1; i /u.k  1/ (2.2)
0 0
Fi .k/ D Pi .kjk  1/C .k; i /ŒC.k; i /Pi .kjk  1/C .k; i / C v2 .k/1 (2.3)
Pi .kjk  1/ D A.k  1; i /Pi .k  1jk  1/A0 .k  1; i / C w2 (2.4)
Pi .kjk/ D ŒI  Fi .k/C.k; i / Pi .kjk  1/; (2.5)

with the initial condition of xO i .0j0/ = xO .0/ and Pi .0j0/ = P.0/.

Define qi .k; I k / to be the posterior probability of model i at stage k,

qi .k; I k / D P. D i j I k /; k D 0; 1; : : : ; N  1:

The posterior probabilities, qi .k; I k /, i = 1; 2; : : : ; s, can be calculated recursively

based on the observation (Casiello and Loparo 1989) as follows:

Li .k/
qi .k; I k / D Ps q .k  1; I k1 /; k D 1; 2; : : : ; N;
k1 /L .k/ i
(2.6)
q
jD1 j .k  1; I j
18 P. Fu

with the initial condition qi .0; I 0 /, where

1 1
Li .k/ D jPy .kjk  1; i /j 2 expΠyQ .kjk  1; i /0 Py .kjk  1; i /1
2
Qy.kjk  1; i / (2.7)
yQ .kjk  1; i / D y.k/  C.k; i /Oxi .kjk  1/ (2.8)
Py .kjk  1; i / D C.k; i /Pi .kjk  1/C0 .k; i / C v2 .k/: (2.9)

Define for i = 1, 2, : : :, s,
˚ 
Ji .k; I k / D E x0 .k/Q.k/x.k/ C u0 .k/R.k/u.k/ j i ; I k ; k D 0; : : : ; N  1
˚ 
Ji .N; I N / D E x0 .N/Q.N/x.N/ j i ; I N :

Then the following is obvious,

J.k; I k / D Efx0 .k/Q.k/x.k/ C u0 .k/R.k/u.k/ j I k g

X
s
D qi .k; I k /Ji .k; I k / k D 0; 1; : : : ; N  1
iD1

J.N; I N / D Efx0 .N/Q.N/x.N/ j I N g

X
s
D qi .N; I N /Ji .N; I N /:
iD1

By the principle of stochastic dynamic programming, the closed-loop control

that minimizes the performance index in problem .P/ can be obtained by solving
the following recursive relation,
(
X
s
min E qi .0; I 0 /Ji .0; I 0 /
u.0/
iD1
X
s
C min E qi .1; I 1 /Ji .1; I 1 / C : : :
u.1/
iD1
nX
s
C min E qi .k; I k /Ji .k; I k / C : : :
u.k/
iD1
hX
s
C min E qi .N  1; I N1 /Ji .N  1; I N1 /
u.N1/
iD1
 )
X
s i o
1 0
C qi .N; I /Ji .N; I /jI
N N N1
: : : jI k
: : : jI jI : (2.10)
iD1
2 Dual Control in Big Data Era: An Overview 19

In principle, the optimal dual control problem .P/ can be solved via (2.10).
However, the difficulty and complexity in solving .P/ hide deeply behind these
seemingly straightforward equations. In fact, in dual control problems, all of the
posterior probabilities at later stages are affected by previous controls. The curse of
uncertainty of the posterior probabilities in later stages is further compounded by
the required expectation operations. Therefore, to derive the cost-to-go functions in
stochastic dynamic programming from (2.10) is a formidable task, as long as the
posterior probabilities at later stages are previously control-dependent.

2.3.2 Open-Loop Feedback Control

Suppose that future learning will not be performed, the open-loop feedback control
can be obtained by fixing all the posterior probabilities in the later stages at
qi .k; I k /; i D 1; 2; : : : ; s. As a result, the following optimal open-loop feedback
control problem is considered at stage k,
(
X
s
min E qi .k; I k /Ji .k; I k / C : : :
u.k/
iD1
(
X
s
C min E qi .k; I k /Ji .N  2; I N2 /
u.N2/
iD1
X
s
C min E qi .k; I k /Ji .N  1; I N1 /
u.N1/
iD1
 ) )
X
s
C qi .k; I /Ji .N; I /jI
k N N1
jI N2
: : : jI :
k
(2.11)
iD1

A controller that uses observations to update online the estimation of the

uncertain parameter is said to have a learning feature. The learning policies can
be further classified into two types—active learning and passive learning. We can
always expect an improving knowledge about the system’s uncertainty when future
observations are utilized. A controller that takes the future uncertainty reduction
is said to have a property of active learning. In return, a controller with an active
learning property affects the degree of future uncertainty reduction. To power a
control law with a property of active learning is, in general, needed to achieve
an optimality in dual control (Griffiths and Loparo 1985). The open-loop feedback
control law is a passive scheme that does not possess an active learning feature (as it
does not take into account any impact from the future learning) and thus can never
be optimal.
20 P. Fu

2.3.3 Active Open-Loop Feedback Control: Variance

Minimization Approach

A degree of success of active learning can be measured by the variance of the final
state. Therefore minimizing a variance term of the final state will add a feature of
active learning to the derived control law. In this section, we consider a modified
problem .Ma .// in which a variance term at the final stage is attached to the
performance index of .P/,
n X
N1 o
.Ma .// min E x0 .N/Q.N/x.N/ C x0 .k/Q.k/x.k/ C u0 .k/R.k/u.k/ j I 0
kD0
0
CTrŒCov.x.N/ j I /
s:t: x.k C 1/ D A.k; /x.k/ C B.k; /u.k/ C w.k/ k D 0; 1;    ; N  1
y.k/ D C.k; /x.k/ C v.k/; k D 1; 2;    ; N

Parameter  2 Œ0; 1/ is a weighting coefficient of active learning. A larger 

implies that more importance has been placed on active learning.
Problem .Ma .// is difficult to be solved directly, since the recursive equations
of dynamic programming involve certain nonlinear terms of the state estimates that
introduces a nonseparability in the sense of dynamic programming. In order to
overcome this difficulty, problem .Ma .// is embedded into a tractable auxiliary
problem in which the optimal open-loop feedback control can be found. Solving
the auxiliary problem and investigating the relationship between the solution sets
of problem .Ma .// and the auxiliary problem, the optimal control of problem
.Ma .// can be identified.
Define S.N/ D Q.N/ C I, the performance index of .Ma .// can be written as

n X
N1 o
0
J D E x .N/S.N/x.N/ C Œx0 .k/Q.k/x.k/ C u0 .k/R.k/u.k/ j I 0
kD0

E.x.N/ j I / E.x.N/ j I 0 /:
0 0
(2.12)

Let

n X
N1 o
J I D E x0 .N/S.N/x.N/ C Œx0 .k/Q.k/x.k/ C u0 .k/R.k/u.k/ j I 0
kD0
0
J D E.x.N/ j I /:
II

It is easy to see that the performance index in .Ma .//, J, is a concave function of
J I and J II ,
 0
J.J I ; J II / D J I   J II J II : (2.13)
Another random document with
no related content on Scribd:
 A curious incident connected with this time of anxiety was
recorded by Sir John. It is related here as printed in the Journal of
the Society for Psychical Research[57]:—

In the year 1879 my son Robert Drummond Hay resided at Mogador with his
family, where he was at that time Consul. It was in the month of February. I had
lately received good accounts of my son and his family; I was also in perfect
health. About 1 a.m. (I forget the exact day in February), whilst sleeping soundly
[at Tangier], I was woke by hearing distinctly the voice of my daughter-in-law, who
was with her husband at Mogador, saying in a clear but distressed tone of voice,
‘Oh, I wish papa only knew that Robert is ill.’ There was a night-lamp in the room. I
sat up and listened, looking around the room, but there was no one except my
wife, sleeping quietly in bed. I listened for some seconds, expecting to hear
footsteps outside, but complete stillness prevailed, so I lay down again, thanking
God that the voice which woke me was an hallucination. I had hardly closed my
eyes when I heard the same voice and words, upon which I woke Lady Drummond
Hay and told her what had occurred, and I got up and went into my study,
adjoining the bedroom, and noted it in my diary. Next morning I related what had
happened to my daughter, saying that though I did not believe in dreams I felt
anxious for tidings from Mogador. That port, as you will see in the map, is about
300 miles South of Tangier. A few days after this incident a letter arrived from my
daughter-in-law, Mrs. R. Drummond Hay, telling us that my son was seriously ill
with typhoid fever and mentioning the night during which he had been delirious.
Much struck by the coincidence that it was the same night I had heard her voice, I
wrote to tell her what had happened. She replied, the following post, that in her
distress at seeing her husband so dangerously ill, and from being alone in a
distant land, she had made use of the precise words which had startled me from
sleep, and had repeated them. As it may be of interest for you to receive a
corroboration of what I have related, from the persons I have mentioned, who
happen to be with me at this date, they also sign, to affirm the accuracy of all I
have related.
When I resigned, in 1886, I destroyed, unfortunately, a number of my diaries
and amongst them that of 1879, or I should have been able to state the day, and
might have sent you the leaf on which I noted the incident.
 CHAPTER XXIII.

THIRD MISSION TO FAS. 1879-1880.

In the autumn of 1879 Sir John writes, ‘State of Morocco better,

but the Government is such a wretched one that I am always finding
the stone I had rolled up, back again at the bottom! A vigorous tyrant
would be preferable to our good, well-meaning Sultan.’
He decided on undertaking his long-deferred visit to the Moorish
Court in the following spring, with a view of bringing his personal
influence to bear on the Sultan and stimulating him to attempt some
measure of reform. He was not, however, sanguine of success. In
February, 1880, he writes:—

Rain has fallen in abundance, crops look well, and Moors are holding up their
heads again, though the Government at Fas is as bad as bad can be, the new Uzir
having been selected because he is a relative of the Sultan, and not for his fitness.
I expect to have very uphill work at the Court. A clever rascal is better than an
ignorant, corrupt dolt, which latter, I fear, is Uzir Mokhta.

And again in March:—

‘I have faint hope of doing much good when I have to deal with an ignorant
fanatic, and shall endeavour to treat direct with the Sultan, who is intelligent but
stupidly avaricious. I say stupidly, for the system is pursued of killing the goose to
get the golden eggs.’
‘I am overwhelmed with work,’ he writes on March 27, on the eve of his
departure, ‘as we are off on the 3rd. Sultan has sent us handsome tents, led
horses and mules, fifty escort and forty baggage animals. We shall be at Fas, I
suppose, on 13th or 14th. Some of our party will return by the end of the month. I
fear I shall be detained till the middle of May. Moors, like all Orientals, try to wear
out a negotiation by dilatoriness, in the hope that one will accept half, merely to get
rid of the business one may take in hand.
‘This is an Augean stable. I have long flourished my broom on the threshold,
but with little hope of clearing away the muck which has accumulated in centuries.
There is now a dawn of hope that the civilised world will take better interest in the
destiny of this fine country and people, who for centuries in Spain were in the van
of science, literature, and art.
‘Entre nous, I brought about the Conference at Madrid. I fear, however, so
many interests will clash that we shall not be able to get rid of all the foreign
vultures which prey upon Morocco, and that no measures will be adopted for
urging the Sultan to abolish a system which is equivalent to preying on his own
vitals.
‘The French are very active about the railway to the Sudan. As the road will
pass near, or even I believe through, a part of Morocco, we may expect to hear of
troubles which it is to be apprehended will bring about a conflict. Of course
Morocco will be crushed, though I do not suppose either Great Britain, or other
countries having interests in the Mediterranean, will ever allow the Straits to be
held by France or any other Power. It is a disgrace, however, to the civilised world
if this wretched Government is allowed to drag on. Sultan must be required to
introduce reforms. I shall preach and try to rouse him; but I shall have a nest of
hornets about me both at Fas and Tangier.
‘N.B.—No cups of coffee to be drunk that have not been tasted by my host first,
or you will hear of a belly-ache!’

The Mission started on April 2, and on the 25th Sir John writes
from Fas:—

We had a pleasant journey here, and our companions are agreeable, clever
men.
Reception on entering Fas unusually demonstrative. The Sultan most gracious;
but he is surrounded by venal and ignorant Ministers whose only aim is to fill their
own pockets, so my preachings and prayings will, I fear, not result in any radical
reforms.
When my back is turned, Sultan will be deceived, and the progress in cow-tail
fashion will continue.

Some of the petty obstacles which beset his path and the way in
which he overcame them may be gathered from the following
account written by Sir John of an interview with the Uzir Sid Mokhta.

Having learnt that it was not the intention of the Uzir to return my
visit of ceremony on arrival, I sent a message to him to this effect:
‘On what day and at what hour will it be convenient for the Uzir to
return the visit of ceremony I propose to pay him?’
He replied that he could not return the visit of any person,
Mohammedan or Christian, as he was connected by marriage with
the Sultan. To this my reply was, through the interpreter, that I could
not admit such an excuse, for if he was connected in the female line
by marriage with Sultan Hassan, an ancestor of my family
descended in direct line from Queen Arabella of Scotland. Therefore,
on the same pretext of alliance with royalty, I could decline to call
upon him! After an interchange of many messages, this question of
etiquette was referred to the Sultan, who declared that the Uzir was
to return my visit the day after I had called.
This Uzir had also declined to introduce the word ‘Sir’ in his letters
to me, or to put the equivalent in Arabic, giving as an excuse that it
was contrary to the precepts of the Mohammedan religion to address
any Christian by a term which was, as he had been given to
understand, equivalent to ‘Sid,’ meaning Lord or Master.
To this I replied that unless he prefixed the title, which my own
Sovereign had given me and which was made use of in Her
Majesty’s letter of credence to the Sultan, it was out of the question
to expect that I should address him as ‘Sid’ or by any other title. The
Uzir offered to address me verbally, or in writing, by the Spanish
word ‘Caballero.’ I replied that I was not a Spaniard, and therefore
declined to be addressed by a Spanish title; but if from a religious
point of view he persisted in declining to use the English word ‘Sir’ or
a synonymous Arabic title in writing to me, I should, in addressing
him a letter, give him, in addition to that of Uzir, the same title or
preface that he granted to my name.
This question was also referred to the Sultan, who decided that
the Uzir should address me as the ‘Minister of the Queen of Great
Britain,’ without putting my name, and that I should in like manner
address the Uzir without putting his name. I told the Uzir, when we
met, that such a discussion was most puerile, and would be so
considered by statesmen and diplomatists, both in Mohammedan
and Christian countries; that it was not worth the waste of time and
paper, and I should let it drop, accepting the Sultan’s decision.
We then interchanged visits without further question; but a few
days after these visits of etiquette, having occasion to interview the
Uzir on business, I requested him to fix the hour and place of
meeting. He sent word to me that, as the weather was warm, he
would receive me in a ‘kubba’ in the garden of his palace, and
named the hour. I took a ride that morning, and arrived at the Uzir’s
house ten minutes before the time fixed for the interview. The usher
of the Uzir received me, and said that his master had not yet arrived
from the Court. I looked at my watch and told the usher that I had
arrived ten minutes before my time, and would therefore sit down
and await the Uzir. So he led me to a pavilion, at the end of a long
narrow path, where I saw two chairs placed, apparently for the Uzir
and myself. The one, facing the entrance, was a very gorgeous arm-
chair covered with beautiful damask; the other, on the left of it, an
ordinary rush-bottomed wooden chair, evidently intended as a seat
for the British Envoy. I heard at a distance the heavy shuffling steps
of the unwieldy Uzir, waddling towards the spot; so without letting it
appear that I was aware of his arrival, I took possession of the
gorgeous chair, to the dismay of the usher; saying at the same time
in a loud voice, so that the Uzir might hear, ‘What have you done?
You must be very ignorant in matters of ceremonial forms to have
placed such a shabby chair for your master the Uzir by the side of
this handsome chair, which you have prepared for me. Take it away,’
pointing to the rush-bottomed chair, ‘and bring for your master a
proper seat.’
With an hysterical laugh the Uzir, seeing the man hesitate, said,
‘The Bashador is right: go and fetch another chair.’ I rose and
saluted the Uzir, saying, ‘We will converse standing until the other
chair is brought.’
I had an object in all this, for I knew that by pricking the wind-bag
of vanity and fanaticism of the Uzir I should better prepare him to
treat with me upon business and obtain satisfactory results; and thus
it proved.
In a private conversation with the Sultan one day, I alluded in
delicate language to the stupidity and unfitness of Uzir Mokhta. His
Majesty replied, smiling at my remark, that Alarbi Mokhta being the
chief of a powerful tribe, the ‘Jarmai,’ he had placed him in an
influential position, adding, ‘he is yet an unbroken “tsaur” (ox), who in
due time will be tamed by the yoke and will improve.’
The palace built by Mokhta since his appointment as Uzir is
worthy of his ancestors the Moors of Spain, who erected the
splendid monuments of architecture at Granada and Seville.
The arches, the designs in stucco, the wooden ceilings, all carved
and painted in elaborate arabesque, are as beautiful as those of the
ancient palaces I have seen in Andalusia.
The Uzir is said to have spent on this building upwards of
$100,000, money obtained by peculation, extortion, and other
corrupt practices; for, though the Sultan’s Ministers are not paid, the
emoluments derived from his official position by the Uzir far exceed
the salary of our Prime Minister at home.
Though Mokhta did not, and never could, like me, yet, as a proof
that my action and language took the nonsense out of him, the
morning I left the Court, when about to call on the Uzir to take leave,
I met him, and having mentioned my intention, he begged me not to
take the trouble to call, saying that he would have the pleasure of
accompanying me to the gates of the town (a mile distant). This he
did, and we parted apparently the best of friends.
Poison is said to be frequently employed at the Court to get rid of
obnoxious persons of rank. A cup of coffee is a dangerous beverage
when offered by a Moorish host, with whom you may not happen to
be on friendly terms. The effect of the subtle poison which can thus
be administered is rarely immediate; but weeks or months after, the
victim’s hair commences to fall off, and he dies gradually in a state of
emaciation.
On one occasion, after an angry discussion with Mokhta, coffee
was brought. I noticed that he took the cup intended for me, put it to
his lips, making a noise as if sipping,—but which I thought sounded
suspiciously like blowing into the liquid,—and then offered it to me.
Not fancying the bubbled coffee, I declined, saying to the Uzir, ‘I
could not drink before you; pray keep that cup yourself’—helping
myself, while speaking, to the other, which I drank.
The Uzir put down the cup he had offered me, without drinking it.
This, after all, he may have done from tiff, and not because there
was really any poison in it.

After the return of the Mission to Tangier, Sir John writes:—

 We all returned hearty and happy; A. the colour of mahogany, and I of a
saucepan. Our companions were charming. There was not a murmur or a
difference of opinion, except perhaps on politics.
Sultan was perfectly charming in my private interviews, and His Sherifian
Majesty enjoyed my jokes, for, you know, as the bubbles rise in my empty head, I
let them escape.
The Ministers were most attentive, and all the officers at the Court vied one
with another in their attempts to pay us honour and to be hospitable. The Uzir is an
ignorant fanatic. We parted good friends; but I think I left him in a shaky position
vis-à-vis his master. I observed to His Sherifian Majesty casually, ‘In other
countries I have found that Ministers are often wiser and better informed than their
Sovereigns. In Morocco I find Your Majesty far superior to your Uzir.’
The Sultan has agreed that a revision of the Commercial Treaty of 1856 is to
be entered on when his Minister for Foreign Affairs returns, and steps are then to
be taken to improve commerce. Prohibition placed on the exportation of grain
removed. A port in the Wadnun district is to be opened to trade. Steps are to be
taken to reform the system of government, and at the Sultan’s request I gave him
a list of the Governors who are notorious for their tyranny and extortions,
impressing on His Sherifian Majesty that he should endeavour to modify the evils
of the existing system by a judicious choice; but that the most careful selection
would not avail to secure competent and upright Governors so long as the
uncertainty of their tenure tempted them to secure it by corrupt influence for which
extortion must find the means: that this reform requires attention for the
maintenance of order, administration of justice, and the collection of the revenue.
I obtained also a royal order to Bargash, the Minister for Foreign Affairs, to
attend to all appeals of Jews who cannot obtain justice from Governors in the
interior. Then I obtained a settlement of some fifteen claims of British subjects.
A mole is to be built at Tangier, and an order obtained for building, or rather
repairing, the mole at Mogador, which, in consequence of the state of Moorish
finances, had not been executed.
Works are also to be undertaken for the better supply of water for Tangier when
the state of finances permits.
The Sultan is well-disposed, clever, and intelligent; but my work is only a
quarter done, for the difficulty with all Eastern Governments is to obtain the
execution of royal edicts that have been received.
I tried to induce Sultan not to give me a horse, &c.; but His Majesty was irate,
and said that if he did not give me a public mark of his goodwill, his subjects would
be displeased and a wrong inference would be drawn. So I had to accept the
‘white elephant’ and trappings, sword and gun; all very beautiful. The three latter
gifts I have told Lord G. I accept on official grounds, and therefore wish to present
them to the Kensington Museum through the Foreign Office.
The gentlemen who accompanied me received handsome swords of honour; A.
and A., two beautiful dresses; the interpreters and Arab secretary, mules.

In 1880 Sir John was promoted to the rank of Envoy

Extraordinary, and among his friends there was apparently some
expectation that he might be chosen to succeed Sir Henry Layard at
Constantinople; but in reply to a letter to that effect he says:—

My memory fails me and I quite forget if I have acknowledged several pleasant

letters from you, and amongst them one expressing a hope that I might replace
Layard. It is very good of you to say that I should be a fit man for the post, but I
cannot agree with you. Honest and conscientious I am; but I have neither the pen
nor the tongue to conduct affairs upon which the fate of nations will depend.

The following letters from his former colleague at Tangier,

Monsieur Tissot, show that, whatever he might himself have thought
of his fitness for the appointment, he would have been cordially
welcomed by the French Ambassador.

Ambassade de France, Thérapia, Juillet 11, 1880.

Je n’ai vu Sir Henry Layard qu’au moment même de son départ. Je ne puis pas
dire que je le regrette: il me serait difficile de souhaiter un collègue plus agréable
que Mr. Goschen et il est impossible que l’union entre les deux Ambassades soit
plus intime et plus confiante qu’elle ne l’est. Mais Mr. Goschen ne restera pas à
Constantinople et je vous demande qui lui succédera?
Je veux espérer que ce sera vous. Je vous ai toujours donné rendezvous ici,
vous vous le rappelez. Faites en sorte de me tenir parole. Ce serait pour moi une
joie bien vive, mon cher ami, que d’aller vous recevoir à bord de l’‘Antelope.’ Il
n’est pas de jour où je ne vous regrette, et j’espère, encore une fois, que nous
recommencerons ici, côte à côte, nos travaux herculéens du Maroc. L’‘étable’ est
plus vaste et proportionnellement encore plus sale que celle de notre ami Mulai
Hassen (que Dieu le rende victorieux).

The second letter from M. Tissot, written from Pera, is dated

nearly a year later.
 Mon cher Ami,
Je m’empresse de vous remercier de votre affectueuse lettre: votre écriture
‘torrentielle’ trahit toujours la vigueur et l’activité enragée que je vous ai connue.
Excellent signe! 58[‫]ما شاء هللا‬
J’ai reconnu votre amitié à la façon dont vous avez bien voulu me présenter à
mon futur collègue: j’espère vivre avec lui en d’aussi bons termes politiques
qu’avec vous, bien que ce ne soit pas toujours facile sur le terrain de
Constantinople. Je suis sûr, tout au moins, que nous nous querellerons aussi
amicalement qu’avec Goschen qui a emporté de moi, m’assure-t-on, le souvenir
affectueux qu’il m’a laissé. Nous n’avons pas toujours été du même avis, mais
notre intimité n’en a jamais souffert,—au contraire, et notre estime réciproque s’est
accrue de toute la déférence qu’une paire de poings solides inspire à une autre
paire de même trempe.
Comme je ne connais pas encore Dufferin, j’ai le droit de vous dire, mon cher
ami, que c’est vous que je désirais ici et que la nomination a même été une
déception pour moi. Vos instincts ‘rather pugnacious’ trouveraient ici matière à
ample satisfaction: Old Turkey (vous êtes libre à traduire ‘le vieux dindon’) est plus
fanatique et plus réfractaire que jamais à l’influence européenne. De plus, il n’y a
plus de Gouvernement turc: le Sultan, ou pour mieux dire le Khalifa, comme il se
plaît à se désigner lui-même dans les notes qu’il nous adresse, a tout confisqué et
prétend tout faire pour lui-même, sûr moyen de ne rien faire. La Porte n’est plus
qu’un décor de théâtre qui s’ouvre sur le vide.
Goschen a apporté ici plus d’illusions qu’il n’en remporte. Je l’avais prévenu
dès son arrivée.
Nous venons cependant, après une année du plus dur labeur, de résoudre les
deux questions Monténégrine et Grecque. Je vais me reposer sur ce double
succès. Je pars dans huit jours pour Vichy via Paris et je laisserai à mon Chargé
d’Affaires le soin et l’honneur de combattre à côté de Lord Dufferin dans la
question arménienne, ‘confound it!’ Nous avons sué huit jours, Goschen et moi, à
rédiger la fameuse note en faveur de réformes arméniennes. La Porte s’en est
émue comme d’une noisette.
 CHAPTER XXIV.

1881-1884.

The monotonous tale of Moorish apathy is continued, only

diversified by occasional gleams of hope that he had succeeded in
rousing the Sultan to a sense of his position—a hope never fulfilled.
Thus on January 19, 1881, Sir John writes from Tangier:—

Things are looking better in the interior. The Sultan has addressed energetic
letters to all the Governors, rebuking them for the state of misgovernment, and
threatening his dire displeasure if murderers and malefactors are not arrested and
punished.
Entre nous, I have pulled this wire; but His Sultanic Majesty might almost as
well have addressed me a letter on the state of Ireland.
Now I am at work for the revision of our Convention of Commerce, as
instructed by the Foreign Office. Merchants of course expect much, far more than I
can obtain. I am communicating with some who are very sanguine. I have pointed
to the conduct of civilised Governments who reject liberal measures, and ask how
they can expect ignorant folk like Moorish Ministers to introduce free trade.

And on April 7:—

I have worked up the Treaty, but cannot get the torpid Bargash to respond. All
must go to Sultan, and His Majesty will require two months to consider.
The debt of Morocco to British Loan Contractors expires next year.
$30,000,000 have passed through my hand as Commissioner. No pay: nothing
has stuck. Alhamdulillah[59].

Returning from a brief holiday spent in Europe he found the old

state of affairs unchanged:—
 August 30, 1881.
We are quiet in Morocco, but there are rumblings. If the French make any rash
movements towards Morocco territory we might have an outbreak of fanaticism,
and then the direst consequences may ensue.
The Sultan is still marching towards Marákesh, ‘eating up’ his indigestible
subjects, and thus preparing them to welcome the Nazarene.
Bargash has not returned, so nothing has been done about Convention. These
Moors understand the value of vis inertiae. I am sick of them, and some day may
strike work, for counsels are all thrown away on such a corrupt torpid brute as the
present Uzir.

Ravensrock, Oct. 9, 1881.

I think the French will leave Morocco alone. They must bide their time.
The world does not form a favourable opinion of the result of the prospect they
held out of introducing civilisation and prosperity. Instead we have murders,
robbers, and the fanatical feelings of a quiet inoffensive people roused. A
gentleman who resides in Algeria describes the treatment of the Arab population in
the interior there before the insurrection, as being as hard as that of the poor
people in this country.
Hundreds extort money and grind down the Mohammedans. I dare say we
were as bad in India years ago.

Sir John was at this time more than usually despondent of the
future of Morocco, for another of his favourite projects for the
improvement of trade had met with a complete check. Ever since the
Commercial Treaty of 1856 he had continuously urged on the
Moorish Government the importance of allowing the exportation of
grain. All his efforts were, however, without result until 1881, when a
half-hearted trial of the new system was made. Unfortunately in this
year a prolonged drought, with consequent failure of crops, resulted
in a famine, and the superstitious Moors at the Court at once
concluded that the Deity was angered at the innovation, and they
therefore refused to renew the permission to export grain in following
years.
The results of the famine were, in other ways, most distressing.
The peasantry, fired by the prospect of a larger market for their grain,
had greatly extended cultivation, and many square miles of land,
barren before, had been ploughed and sown, and, with the usual
improvidence of a poor and thoughtless race, they had raised money
from usurers on their ungrown crops. Thus with the complete failure
of these crops, nearly the whole population of the immense arable
plains of Morocco were reduced, not only to beggary, but to absolute
starvation.
Once more Sir John proceeded to the Court, and endeavoured to
rouse the Moors to action.

‘We are off to-morrow,’ he writes on March 28, 1882. ‘H.M.S. “Salamis,” takes
us to Dar-el-Baida, then we go to Marákesh by land. Sultan sends escort, &c., &c.
to meet us. The French have preceded us, Italian and Spanish Ministers follow.
These Missions are like locusts eating up the country, and, alas! no rain falls, and
failure of crops and consequent famine are menaced. . . .
‘A fierce stand has been made against modification of the scale of duties by
this stupid people. But I have other fish to fry at the Court. Sysiphus will continue
to roll up his stone to the last.’

Sir John in this letter refers to the system of ‘mona’ levied on the
provinces through which the Missions pass. In times of scarcity the
tax falls very heavily on the unfortunate peasantry. The Kaids order
much more than is required, and they and the dependants of the
Mission often divide the spoil. Sir John always did what he could to
mitigate this evil, fixing the amount of provision required, and
appointing one of his staff to watch the distribution. Any surplus was
sold by auction each day, and the proceeds given to the poor of the
district. The Spanish Minister adopted the same system on his visit
to the Court at this time.
The next letter shows him immersed in business at Marákesh.

Dar-Mulai-Ali, Marákesh, April 21, 1882.

I am so bothered with work that I hate the sight of a pen and ink, even when
writing to those I love.
 It is getting very hot here. In the shade about 80° (Fahrenheit) at 2 p.m.; but in
our rooms, in this thick-walled house, 71°. We are very comfortable.
The Sultan is most gracious and flattering, but he wants to keep me until Dons
and Italian Mission have come and gone. If so, we shall be here for a month more.
The French Minister went away pleased, and thanked me for aiding him in getting
satisfactory answers.
Nothing is decided about the new Convention. I find the greatest opposition to
any free trade suggestions; but I hammer away, though I tell them my preachings
and warnings have the same effect as the efforts of a man who attempts to fill a
bucket which has no bottom, or at least a hole in it.
The Sultan has assembled twenty-five thousand men to go to Sus to open
ports, as we advised. Crops and all vegetation have failed there and in other
Southern provinces. We passed through stunted crops dried up; no rain. It is most
sad; for the Sultan, following my advice, had encouraged agriculture—and in the
grand rich province of Shawía, through which we passed, we did not see two acres
of land uncultivated. Wheat is far dearer than in England; horses and cattle are
dying; misery everywhere. I am trying to dissuade the Sultan from going to Sus,
and urging him to await another year and better harvest. His troops will desert him
and a revolution will follow if he should be compelled to retreat. Nous verrons the
effect of my advice. Everybody, high and low, is begging me to stop the expedition.
I have settled some affairs and others are in progress. This is an Augean
stable, and I am tired of sweeping, as filth accumulates ten times faster than I can
sweep. I tell the Sultan as much.

The Palace of Dar-Mulai-Ali, in which Sir John was lodged in

1882, has attached to it the following story.
Mulai Ali, the late owner, and uncle of Sultan Mulai Hassan, had
been a great favourite of the people of Marákesh. A student, kind
and just, he passed his time with the learned men of the city and in
the great mosque, the Kutubía; the grounds of the mosque and the
beautiful orchard belonging to Mulai Ali being separated only by a
wall, some six feet high, having in it a door of communication.
One Friday, not many months before our arrival, the Sultan, his
Court and the people of Marákesh were startled by hearing the
‘muddin’ cry from the minaret of the Kutubía—after the usual chant
at midday of ‘God is great and Mohammed is his prophet,’—‘Long
life to our Sultan Mulai Ali.’ The ‘muddin’ was seized and brought
before the Sultan, and though put to the torture, all he would admit
was, that God had inspired him and not man—and he was thrown
into prison. Mulai Ali was also questioned by his royal nephew, but
denied any knowledge of what had taken place—except that he had
heard the cry. A few days after the occurrence Mulai Ali died
suddenly—the Court said of apoplexy, but the people whispered
poison, and the Sultan confiscated his property.
The house was really beautiful, with a great marble court and
splashing fountain in the centre. The garden stood right under the
shadow of the Kutubía tower, and therefore it was considered a
special compliment that a Christian Envoy should be allowed to live
there, though of course the door of communication between the
mosque and Mulai Ali’s property was kept closed.
On the day of the public audience, Miss Hay rode with her father
to the gate of the palace. There, as on former occasions, she was
about to dismount, for, as a rule, no mounted person but the Sultan
is allowed within the palace court. But one of the chief ushers came
forward, and said, ‘Sidna (our Lord) desires you will remain on
horseback.’ He then led her into the great court, which was lined with
foot-soldiers, and ordering some of the men to stand back, placed
her in front of them; himself, with several subordinate officials,
standing as a guard before her.
Sir John, with the other members of the Mission, had taken up a
position in the centre of the court.
Then from the soldiers the cry arose of ‘Long life to Sidna,’ which
was caught up and echoed and re-echoed throughout the whole
town of Marákesh. The large green gate, the private entrance to the
palace, opened, and the Sultan, riding a milk-white horse, with his
guard, umbrella-bearer and attendants, entered and rode up to the
Envoy. After the interview, he passed near where Miss Hay was
stationed, turned his horse so as to face her, and caused it to curvet
and rear slightly, while himself saluting her. She acknowledged the
salute, and the Sultan proceeded on his way to the palace.
 This act of courtesy raised much comment among the Moors, as it
was the first time that the presence of a lady at a public audience
had been acknowledged by any Sultan. Next day the Uzir’s wife, a
clever and very pretty woman, congratulated Lady Hay and her
daughter on having won such a high mark of favour from His
Sherifian Majesty.
On the return journey, Sir John writes from the camp at Kasba
Jedída, four hours’ march from Rabát, on May 30:—

All wondrous well. Actually cold to-day, with a fresh wind blowing from the
Atlantic: 74° in tent at 3.30 p.m.
Yes, some sneaking, jealous person put in the papers an unfounded story
about my being stoned. The members of the French Mission did get into a row,
and they had a lot of people arrested and flogged, and a poor woman came crying
to me to aid her in begging for release of her husband. As to our party, all the
Moors, high and low, wild and tame, vie with each other in showing us attention,
civility, and hospitality. ‘Ingliz, you are just; you are kind; you are generous; we look
upon you as one of us.’ These are the expressions of every one, including the
Sultan.
My friend Diosdado[60] has gained golden opinions everywhere. ‘Kindness and
charity; honour bright.’ He has won the good-will of the Sultan and his Court, and
reversed the bad policy that existed. He and I were like brothers.
I think I told you I was behind the curtain to arrange the question between
France and Morocco. . . . the chief object of my Mission is accomplished.
The basis of the Commercial Convention is arranged, but nothing will be done
till the state of the country and prospects improve.
One hundred and two claims settled.

Señor Diosdado, mentioned in the foregoing letter, was the

Spanish Envoy, who was at Fas on a Mission to the Sultan at the
same time as Sir John. One of the members of this Spanish Mission
was the Father Superior of the Franciscan Brotherhood in Morocco,
a very able man, much beloved and esteemed by persons of all
nationalities and creeds throughout Morocco, and whom Sir John
had always regarded as a good and trusted friend.
 When, after Sir John’s death, his elder daughter returned to
Tangier in the spring of 1894, the Father Superior called on her, and
expressed heartfelt sympathy with her and her family in their
bereavement, and his regret for the loss to himself of an old and
valued friend. He added:—

Your father always showed me great kindness and friendship, and never shall I
forget how much I owe him for a few words of encouraging recommendation at a
critical moment.
It was during my visit to the Court of Morocco, where I had gone with the
Spanish Envoy. Your father had preceded us there, also on a Mission to the
Sultan, and one evening, immediately after our arrival, I called on His Excellency
with the Spanish Minister and some of his suite. Sir John happened to be in
conclave with one of the Sultan’s most influential Ministers, but their conversation
was suspended during our visit, and we were severally introduced to this dignitary.
When presenting me, your father said, ‘Let me introduce my old friend Padre
Lerchundi, and recommend him to you as a good man, a friend to the Moors and
kindly to all. Believe me, you may put entire trust and confidence in him.’
This recommendation, added the Superior, assisted me more than any
representations could have done from other quarters.

Europeans, who were ignorant of the nature of the people with

whom Sir John had to deal, were prone to attribute the lack of
progress throughout the country to his apathy. An English Company
had been formed to carry out railway, telegraphic and other works in
Morocco, and Sir John was asked to use his influence to secure
concessions. He promised his aid and did his utmost; but, writing in
1883, after an interview with one of the projectors, he says:—

During thirty-seven years, having unceasingly worked to obtain all this for
British subjects without success, I could hold out little hope that the Company
would succeed so long as the present form of Government existed in Morocco and
the Sultan’s venal Ministers remained in power.
I said, however, that he should carry out his intention of visiting the Court, and
then he would be able to judge from all he heard and saw whether there was any
hope of success; for people in England seem to be foolishly impressed with the
idea that I uphold this Government in its rejection of all improvements and reforms.
 The state of this country becomes daily more hopeless, and I do not see even a
glimmering of hope for the future. Unjust claims are daily pressed on them, and
protected Jews concoct false documents and extract thousands of dollars from the
Mohammedans. I have warned the chief Jews that, if the Sultan dies and there is a
revolution, the Mohammedans will not forget their wrongs, and there will be a
general massacre.
I am sick of Morocco and its affairs, and am thinking seriously of taking off the
galling collar; for I pull and pull, and the vehicle only backs. . . .
One of the Company formed for regenerating Morocco has started for the
Court. I am glad; for the Company will learn, I expect, that it is not Drummond Hay
who stops the way.

In 1883 the forward policy of France was already producing

serious results in Morocco, which seemed in the near future likely to
assume still more formidable proportions. Writing from London on
October 4, Sir John expresses his fears of the effect of French
machinations:—

The action of France appears to be that of paralysing all government and

authority of the Sultan by covert proceedings, and, when anarchy takes place,
then, perhaps, la Grande Nation hopes to be asked by the civilised world to step in
and protest.

In the following spring matters looked very serious, and, after his
return from a week’s pigsticking at the ‘Lakes,’ he writes to his sister
on the subject, after a short allusion to his favourite sport:—

Ravensrock, May 20, 1884.

We enjoyed our camp life during the first week of May, though for two days it
blew an easterly hurricane, which spoilt our sport. Nevertheless fourteen of the
enemy were slain. J.’s fine horse badly wounded; also the horse of the Basha’s
son.
You will perhaps have seen in the papers contradictory reports about the state
of relations between France and Morocco,—telegrams asserting that relations are
broken off and flag hauled down; then telegrams declaring that the most friendly
relations exist. The fact is Ordega has been blustering here and threatens to break
off relations, to march an army across the frontier, to send a fleet and encourage
the Sheríf[61] to raise the standard of rebellion and march upon the capital, and
that a French force would cross the frontier and support him. All this to obtain the
dismissal of Jebar, the Khalífa of Wazan, who was unfriendly to the Sheríf, the
protégé of France. Some one of Jebar’s dependants had called the Sheríf’s son an
infidel for accepting French protection; upon which the latter seized the Moor and
had him flogged and poured boiling water on him. Naturally, a few days afterwards
he died. Then Ordega sent a Secretary of Legation to inquire about this, and the
Sheríf’s son brought his witnesses to prove that Jebar had poisoned his own
dependant[62]!
The Sultan refused to dismiss Jebar without inquiry. Then followed menace
upon menace, and finally Ordega left for Paris. The French flag was hauled down,
and all letters from Moorish authorities were returned by the Chargé d’Affaires.
The impression of course left on the mind of the Sultan and of every one (except
myself) was that war was imminent. I telegraphed to my Government, and so did
my other colleagues. Lord G. telegraphed back that Ferry assured him the
Governments of Morocco and France were on the most friendly relations, and that
they had no desire to create disturbance. In the meantime the Sultan appealed to
the Austrian, British, German, Italian, and Spanish Governments against the
proceedings of Ordega in affording protection to the Sheríf, his sons, and
thousands of dependants, and complained that he was fomenting insurrection by
sending emissaries to all parts of the Empire to call upon the population to rebel
against His Sherifian Majesty’s authority. No reply has as yet been given to this
appeal, and the conspiracy continues, backed by the French. I think it probable,
however, that Ordega has misled his Government and declared that he has not
done what he has done; for he took care not to write his threats, and when a letter
was addressed to him containing a repetition of all he had menaced, he would not
receive it.
On his arrival in Paris an article appeared in the Gaulois of May 8, signed by
the Editor, making calumnious charges regarding the corrupt practices of all the
Representatives (myself included) except the German. Then on the 10th, in
consequence of the demand of a Spanish diplomat resident at Paris, the Editor
retracted as regards Diosdado. I telegraphed to Lord G. that I required retraction
and an apology to be inserted in the Gaulois: for the Editor openly declares that he
has reported the language held by Ordega to Ferry on May 7, and subsequently
communicated to him.
The Italian and Portuguese Ministers both wrote to their Governments to
demand satisfaction.
S. takes it up personally, and threatens to call out Ordega if an apology is not
offered.
 Our days of Quixotism are passed; but my fingers tingle to box the fellow’s
ears. I do not know how he can return here amidst the nest of hornets he has
roused. . . .
Unfortunately, just at this crisis, i.e. on the 14th, appeared an article in the
Times which declares that England has no commanding interest in the political
condition of Morocco, leading the reader to believe that the action of France is
beneficial to British interests.
This leader may have been inspired by those who desire to prepare and mould
the minds of the British public for the prosecution of their undoubted future, if not
immediate, design of becoming mistress of the Straits and of the Mediterranean
from Spartel to Tripoli, and perhaps hereafter to Egypt.
Should France annex or establish a protectorate over Morocco, the port of
Tangier might be made a safe and well-fortified harbour for torpedo vessels and
the like craft, and other harbours could be formed likewise to the eastward
between Tangier and Ceuta. France and Spain would probably be allied in case of
war, and our shipping would only pass by running the gauntlet. Gibraltar must fall
or come to be of little value as a harbour of refuge. Nelson said, ‘Tangier must
either remain in the hands of a neutral Power like Morocco, or England must hold
it.’ Of what avail is it that we took our stand about the passage of the Dardanelles,
and defended at such cost our free passage through the Suez Canal, if this end of
the way to the East and to India can be stopped when it pleases France? Yet the
Times says, ‘we have no political interest in Morocco.’
This article is based on a letter dated May 12, addressed to the Editor of the
Times by the Secretary of the Anti-Slavery Society, wherein he dwells, and with
strong grounds, upon the state of misgovernment, slavery, and other abuses in this
country; but Mr. Allen is not always correct, especially when he speaks of my being
‘all-powerful,’ and yet intimates that I do not use my influence to good purpose.
The leader in the Times says that (in the case of the man murdered at Wazan)
‘the action of France has on the whole tended to promote the cause of civilisation,
good government, and freedom in a country which has long been a stranger to
these blessings.’
Now I know not one single act of the French Government or its Representative
in this country which has been beneficial to the cause of civilisation or introduced
any reform or improvement in Morocco, and I defy any Frenchman to state them.
The compulsory payment by the Moorish Government of numerous claims of
French citizens and protected subjects has certainly been obtained, but that can
hardly be regarded as beneficial to civilisation, unless the protection afforded to
the Sheríf can be considered beneficent. If the proceedings of the Sheríf are not
put a stop to, and civil war ensues, a state of anarchy will be produced, and
probably a massacre of the unfortunate Jews in the interior.