Information Security Applications Brent

Byunghoon Kang
Visit to download the full and correct content document:
https://textbookfull.com/product/information-security-applications-brent-byunghoon-ka
ng/
More products digital (pdf, epub, mobi) instant
download maybe you interests ...

Introduction to Geographic Information Systems 9th

Edition Kang-Tsung Chang

https://textbookfull.com/product/introduction-to-geographic-
information-systems-9th-edition-kang-tsung-chang/

Robust Control: Theory and Applications Kang-Zhi Liu

https://textbookfull.com/product/robust-control-theory-and-
applications-kang-zhi-liu/

Handbook of Multimedia Information Security Techniques

and Applications Amit Kumar Singh

https://textbookfull.com/product/handbook-of-multimedia-
information-security-techniques-and-applications-amit-kumar-
singh/

Light Lordy 1st Edition Brent Tyman Tyman Brent

https://textbookfull.com/product/light-lordy-1st-edition-brent-
tyman-tyman-brent/
Information security fundamentals Peltier

https://textbookfull.com/product/information-security-
fundamentals-peltier/

Introduction to Machine Learning with Applications in

Information Security 1st Edition Mark Stamp

https://textbookfull.com/product/introduction-to-machine-
learning-with-applications-in-information-security-1st-edition-
mark-stamp/

Electric Circuits James S. Kang

https://textbookfull.com/product/electric-circuits-james-s-kang/

Principles of Information Security 6th Edition Whitman

https://textbookfull.com/product/principles-of-information-
security-6th-edition-whitman/

Principles of information security Fifth Edition

Mattord

https://textbookfull.com/product/principles-of-information-
security-fifth-edition-mattord/
 Brent ByungHoon Kang
Taesoo Kim (Eds.)
LNCS 10763

Information
Security Applications
18th International Conference, WISA 2017
Jeju Island, Korea, August 24–26, 2017
Revised Selected Papers

123
Lecture Notes in Computer Science 10763
Commenced Publication in 1973
Founding and Former Series Editors:
Gerhard Goos, Juris Hartmanis, and Jan van Leeuwen

Editorial Board
David Hutchison
Lancaster University, Lancaster, UK
Takeo Kanade
Carnegie Mellon University, Pittsburgh, PA, USA
Josef Kittler
University of Surrey, Guildford, UK
Jon M. Kleinberg
Cornell University, Ithaca, NY, USA
Friedemann Mattern
ETH Zurich, Zurich, Switzerland
John C. Mitchell
Stanford University, Stanford, CA, USA
Moni Naor
Weizmann Institute of Science, Rehovot, Israel
C. Pandu Rangan
Indian Institute of Technology Madras, Chennai, India
Bernhard Steffen
TU Dortmund University, Dortmund, Germany
Demetri Terzopoulos
University of California, Los Angeles, CA, USA
Doug Tygar
University of California, Berkeley, CA, USA
Gerhard Weikum
Max Planck Institute for Informatics, Saarbrücken, Germany
More information about this series at http://www.springer.com/series/7410
Brent ByungHoon Kang Taesoo Kim (Eds.)
•

Information
Security Applications
18th International Conference, WISA 2017
Jeju Island, Korea, August 24–26, 2017
Revised Selected Papers

123
Editors
Brent ByungHoon Kang Taesoo Kim
KAIST Georgia Institute of Technology
Daejeon Atlanta, GA
Korea (Republic of) USA

ISSN 0302-9743 ISSN 1611-3349 (electronic)

Lecture Notes in Computer Science
ISBN 978-3-319-93562-1 ISBN 978-3-319-93563-8 (eBook)
https://doi.org/10.1007/978-3-319-93563-8

Library of Congress Control Number: 2018947332

LNCS Sublibrary: SL4 – Security and Cryptology

© Springer International Publishing AG, part of Springer Nature 2018

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the
material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now
known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this book are
believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors
give a warranty, express or implied, with respect to the material contained herein or for any errors or
omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in
published maps and institutional affiliations.

Printed on acid-free paper

This Springer imprint is published by the registered company Springer International Publishing AG
part of Springer Nature
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
 Preface

The 18th World International Conference on Information Security and Application

(WISA 2017) was held at Jeju Island, Korea, during August 24–26, 2017. The con-
ference was supported by three research institutes: KISA (Korea Internet and Security
Agency), ETRI (Electronics and Telecommunications Research Institute), and NSR
(National Security Research Institute). Especially this year at WISA 2017, the
US ONRG (Office of Naval Research Global) and ITC-PAC sponsored and also
organized a Tri-Service cyber security panel to share each service's cyber security
research interests and directions, provide engagement opportunities, and increase
potential collaborations with international cyber communities.
The program chairs, Brent Byunghoon Kang, KAIST, and Taesoo Kim, Georgia
Tech, prepared a valuable program along with the Program Committee members listed
here, many of whom have served in top security conferences such as IEEE S&P,
ACM CCS, Usenix Security, and NDSS. We received 59 submissions, covering all
areas of information security, and finally selected 12 outstanding full papers and an
additional 15 short papers out of 53 papers that were carefully reviewed by the Program
Committee. The excellent arrangements for the conference venue were led by the
WISA 2017 general chair, Prof. Donghoon Lee, and organizing chair, JungTaek Seo.
We were specially honored to have the two keynote talks by Professor Virgil Gligor,
Carnegie Mellon University (Former Cylab Director) on “Establishing and Maintaining
Root of Trust on Commodity Computer Systems,” and Dr. J. Sukarno Mertoguno,
Office of Naval Research on “Autonomic Computing and Hybrid Formal-Statistical
Reasoning (for Cyber Interaction and Attack).” Both talks were geared toward the roles
of security in the fourth Industrial Revolution focusing on core platform systems and
automated reasoning and the security protection required for millions of cyber
interactions.
Many people contributed to the success of WISA 2017. We would like to express
our deepest appreciation to each of the WISA Program Committee and Organizing
Committee members as well as the paper contributors. Thanks to their invaluable
support and sincere dedication, WISA 2017 was a success. We believe the following
selected papers from WISA 2017 will contribute to new directions in cyber security,
handling the core security issues of the fourth Industrial Revolution.

August 2017 Brent ByungHoon Kang

Taesoo Kim
 Organization

The 18th World International Conference on Information Security Applications

August 24–26, 2017, Lotte City Hotel, Jeju Island, Korea
http://www.wisa.or.kr

General Chair
Donghoon Lee Korea University, Korea

Orgaizing Committee Chair

JungTaek Seo Soonchunhyang University, Korea

Program Committee Co-chairs

Brent ByungHoon Kang KAIST, Korea
Taesoo Kim Georgia Tech, USA

Program Committee
Amir Houmansadr University of Massachusetts Amherst, USA
Bogdan Warinschi University of Bristol, UK
Byoungyoung Lee Purdue University, USA
Cristina Nita-Rotaru Northeastern University, USA
David Lie University of Toronto, Canada
Dooho Choi ETRI, South Korea
Eul Gyu Im Hanyang University, South Korea
Gang Tan Pennsylvania State University, USA
Harry Wechsler George Mason University, USA
Howon Kim Pusan National University, South Korea
Huy Kang Kim Korea University, South Korea
Jason Hong Carnegie Mellon University, USA
Ji Sun Shin Sejong University, South Korea
John Junghwan Rhee NEC Laboratories America, USA
Jong Kim POSTECH (Pohang University of Science
and Technology), South Korea
Kevin Butler University of Florida, USA
Long Lu Stony Brook University, USA
Marcus Peinado Microsoft Research
Min Suk Kang National University of Singapore, Singapore
Sang Kil Cha KAIST, South Korea
VIII Organization

Seong-je Cho Dankook University, South Korea

Seungwon Shin KAIST, South Korea
Sooel Son Google
SooHyung Kim ETRI, South Korea
Ulrich Rührmair University of Bochum, Germany
Virgil D. Gligor Carnegie Mellon University, USA
Yinqian Zhang The Ohio State University, USA
 Contents

Attack and Defense I

Lightweight Fault Attack Resistance in Software Using Intra-instruction

Redundancy, Revisited. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Hwajeong Seo, Taehwan Park, Janghyun Ji, and Howon Kim

Exposing Digital Forgeries by Detecting a Contextual Violation Using

Deep Neural Networks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Jong-Uk Hou, Han-Ul Jang, Jin-Seok Park, and Heung-Kyu Lee

Robust 3D Mesh Watermarking Scheme for an Anti-Collusion

Fingerprint Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Jong-Uk Hou, In-Jae Yu, Hyun-Ji Song, and Heung-Kyu Lee

Theory in Security

The Search Successive Minima Problem Is Equivalent

to Its Optimization Version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
Haoyu Li and Yanbin Pan

An Improved Algorithm to Solve the Systems of Univariate

Modular Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
Jingguo Bi, Mingqiang Wang, and Wei Wei

O2 TR: Offline Off-the-Record (OTR) Messaging . . . . . . . . . . . . . . . . . . . . . 61

Mahdi Daghmehchi Firoozjaei, Sang Min Lee, and Hyoungshick Kim

ARM/NEON Co-design of Multiplication/Squaring . . . . . . . . . . . . . . . . . . . 72

Hwajeong Seo, Taehwan Park, Janghyun Ji, Zhi Hu, and Howon Kim

Web Security and Emerging Technologies

WheelLogger: Driver Tracing Using Smart Watch. . . . . . . . . . . . . . . . . . . . 87

Joon Young Park, Jong Pil Yun, and Dong Hoon Lee

Evolution of Spamming Attacks on Facebook. . . . . . . . . . . . . . . . . . . . . . . 101

Minsu Lee, Hyungu Lee, and Ji Sun Shin
X Contents

Systems Security and Authentication

Model Parameter Estimation and Inference on Encrypted Domain:

Application to Noise Reduction in Encrypted Images. . . . . . . . . . . . . . . . . . 115
Saetbyeol Lee and Jiwon Yoon

Breaking Text CAPTCHA by Repeated Information . . . . . . . . . . . . . . . . . . 127

Jae Hyeon Woo, Moosung Park, and Kyungho Lee

Automatic Mitigation of Kernel Rootkits in Cloud Environments . . . . . . . . . 137

Jonathan Grimm, Irfan Ahmed, Vassil Roussev, Manish Bhatt,
and ManPyo Hong

Glitch Recall: A Hardware Trojan Exploiting Natural Glitches

in Logic Circuits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
Jungwoo Joh, Yezee Seo, Hoon-Kyu Kim, and Taekyoung Kwon

Design and Implementation of Android Container Monitoring

Server and Agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
Kwon-Jin Yoon, Jaehyeon Yoon, and Souhwan Jung

Improved EM Side-Channel Authentication Using Profile-Based

XOR Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
Momoka Kasuya and Kazuo Sakiyama

Holistic Tracking of Products on the Blockchain Using

NFC and Verified Users. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
Vanesco A. J. Boehm, Jong Kim, and James Won-Ki Hong

Attack and Defense II and Network Security

Abusing TCP Retransmission for DoS Attack Inside Virtual Network . . . . . . 199
Son Duc Nguyen, Mamoru Mimura, and Hidema Tanaka

Improving Detection of Wi-Fi Impersonation by Fully Unsupervised

Deep Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
Muhamad Erza Aminanto and Kwangjo Kim

Cyber Influence Attack: Changes in Cyber Threats Seen in the Russian

Hacking Incident. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
Mookyu Park, Moosung Park, and Kyungho Lee

A Protection Technique for Screen Image-Based Authentication Protocols

Utilizing the SetCursorPos Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
Insu Oh, Kyungroul Lee, and Kangbin Yim
 Contents XI

Crypto Protocols

A General Two-Server Cryptosystem Supporting Complex Queries . . . . . . . . 249

Sha Ma and Yunhao Ling

Efficient Software Implementation of Modular Multiplication in Prime

Fields on TI’s DSP TMS320C6678 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
Eito Miyamoto, Takeshi Sugawara, and Kazuo Sakiyama

Key Managements of Underwater Acoustic Communication Environments . . . 274

Hyunki Kim, Jaehoon Lee, and Okyeon Yi

Parallel Implementations of SIMON and SPECK, Revisited . . . . . . . . . . . . . 283

Taehwan Park, Hwajeong Seo, Garam Lee, Md. Al-Amin Khandaker,
Yasuyuki Nogami, and Howon Kim

Attact Detections and Legal Aspects

Detecting Online Game Chargeback Fraud Based on Transaction Sequence

Modeling Using Recurrent Neural Network . . . . . . . . . . . . . . . . . . . . . . . . 297
Namsup Lee, Hyunsoo Yoon, and Daeseon Choi

The Digits Hidden in the Virtual World: Approximate Estimation Applying

Capture and Recapture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310
Da Mi Hwang and In Seok Kim

Legal Consideration on the Use of Artificial Intelligence Technology

and Self-regulation in Financial Sector: Focused on Robo-Advisors. . . . . . . . 323
Keun Young Lee, Hun Yeong Kwon, and Jong In Lim

Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337

Attack and Defense I
 Lightweight Fault Attack Resistance in
Software Using Intra-instruction
Redundancy, Revisited

Hwajeong Seo1 , Taehwan Park2 , Janghyun Ji2 , and Howon Kim2(B)

1
IT Department, Hansung University, 116 Samseong Yoro-16gil,
Seongbuk-gu, Seoul 136-792, Republic of Korea
hwajeong84@gmail.com
2
School of Computer Science and Engineering, Pusan National University, San-30,
Jangjeon-Dong, Geumjeong-Gu, Busan 609-735, Republic of Korea
{pth5804,jjh0819,howonkim}@pusan.ac.kr

Abstract. Fast implementations of block cipher is fundamental build-

ing block to achieve the high-speed and secure communication between
IT platforms. Even though the communication is securely encrypted, the
system can be exploited by malicious users if the attackers inject fault
signal to the system and extract the user’s secret information. For this
reason, we need to ensure the high performance encryption together with
secure countermeasures against side channel attacks. In this paper, we
present a novel countermeasure against fault attack on Single Instruction
Multiple Data (SIMD) architecture (e.g., ARM–NEON, INTEL–SSE,
INTEL–AVX2). The methods achieved the fault attack resistance with
intra-instruction redundancy feature in SIMD instruction set. Finally,
we applied the new fault attack countermeasures on the block cipher
LEA and achieved the intra-instruction redundancy and high perfor-
mance over modern ARM-NEON architectures.

Keywords: Fault attack countermeasure

Intra-instruction redundancy · Block cipher · SIMD · LEA
ARM–NEON

1 Introduction
In WISA’13, Lightweight Encryption Algorithm (LEA) was announced by the
Attached Institute of ETRI [6]. The LEA algorithm selected the Addition-
Rotation-exclusive-or (ARX) architecture and shows high speed in software and
This work was supported by the Energy Efficiency & Resources Core Technology
Program of the Korea Institute of Energy Technology Evaluation and Planning
(KETEP), granted financial resource from the Ministry of Trade, Industry & Energy,
Republic of Korea. (No. 20152000000170). Hwajeong Seo was supported by the ICT
R&D program of MSIP/IITP. [B0717-16-0097, Development of V2X Service Inte-
grated Security Technology for Autonomous Driving Vehicle].
c Springer International Publishing AG, part of Springer Nature 2018
B. B. Kang and T. Kim (Eds.): WISA 2017, LNCS 10763, pp. 3–15, 2018.
https://doi.org/10.1007/978-3-319-93563-8_1
4 H. Seo et al.

hardware implementations and security against timing attacks (due to the nature
of constant timing). In ICISC’13, LEA implementations on high-end ARM-
NEON and General-Purpose computing on Graphics Processing Units (GPGPU)
platforms are introduced [19]. The works present efficient techniques for Sin-
gle Instruction Multiple Data (SIMD) and Single Instruction Multiple Thread
(SIMT) based parallel computations. In [14], LEA block cipher is also evaluated
in web languages such as JavaScript. The results show that the LEA implemen-
tation can achieve the high performance over web-browsers as well. In FDSE’14,
Van Nguyen et al. implemented the secure Near Field Communication (NFC)
protocols by using the LEA block cipher [21]. In WISA’15, LEA implementation
is also evaluated on low-end 8-bit AVR processor [15]. The author presented
both speed and size optimized techniques on 8-bit low-end processors and com-
pared the performance with representative ARX block ciphers including SPECK
and SIMON. The work shows that LEA is the most optimal block cipher for
low-end embedded applications. In WISA’16, Seo et al. improved the parallel
implementation on ARM-NEON processors [20]. The work shows that the fine
combination of ARM and NEON instruction sets can further achieve the perfor-
mance enhancements in parallelism. As listed above, many LEA related works
proved that LEA is the most promising block cipher for both high-end comput-
ers and low-end microprocessors. However, majority of works mainly considered
the high-speed implementations on the platforms, which is vulnerable to side
channel attacks. In this paper, we propose new secure implementation of LEA
on SIMD architecture against fault injection attacks. Several new techniques are
employed and the implementations are successfully protected from fault injec-
tion attacks. Furthermore, the proposed techniques are not limited to LEA block
cipher implementations. The proposed generic methods can be easily applied to
the other ARX block ciphers such as SIMON and SPECK with simple modifi-
cations.
The remainder of this paper is organized as follows. In Sect. 2, we recap the
fault model, fault attack countermeasures, LEA block cipher, ARM–NEON plat-
form and previous implementations of LEA on ARM–NEON platform. In Sect. 3,
we present the secure fault attack countermeasure and secure implementations
of LEA on ARM-NEON platform. In Sect. 4, we evaluate the performance of
proposed methods in terms of clock cycle and security. Finally, we concludes the
paper in Sect. 5.

2 Related Works

2.1 Fault Attack Model

In this paper, we used the identical fault models used in previous works to
evaluate our secure implementations [12]. The fault model can perform the fault
injection on a cryptosystem and manipulate the instruction opcodes (i.e. instruc-
tion faults) or data (i.e. computation faults). First, the computation faults cause
errors in the data that is processed by a program. The adversary can inject the
 Intra-instruction Redundancy Based Fault Attack Countermeasure 5

faults and the target data can be manipulated. The detailed descriptions of four
computational fault models are as follows.

– Random Word: The adversary can target a specific word in a program and
change its value into a random value unknown to the adversary.
– Random Byte: The adversary can target a specific word in a program and
change a single byte of it into a random value unknown to the adversary.
– Random Bit: The adversary can target a specific word in a program and
change a single bit of it into a random value unknown to the adversary.
– Chosen Bit Pair: The adversary can target a chosen bit pair of a spe-
cific word in a program, and change it into a random value unknown to the
adversary.

Second, the instruction faults can change the program flow (the opcode of
an instruction) by fault injection. The common instruction fault model is the
instruction skip fault model. This model replaces the opcode of original instruc-
tion with a no-operation (nop) instruction. With this fault attack, the adversary
can skip the execution of specific instructions in the program and change the
program flow.

2.2 Previous Fault Attack Countermeasures

We can prevent the fault attack by using detection based countermeasures in soft-
ware. The techniques are based on time redundancy of encryption (namely dupli-
cate encryption). This technique figures out the fault attempts by comparing the
consistency of the redundant executions or computations. To achieve the higher
robustness than the duplication approach, some previous works perform the
instruction triplication [1]. However, the instruction duplication and triplication
cause 3.4 and 10.6 times performance overheads, respectively. Furthermore, the
sophisticated fault injection setup can easily break the instruction duplication or
triplication through consistent fault injection attacks. Another line of detection
based fault attack countermeasures is information redundancy, which evaluates
the additional check variables or parity bits. This approach detects the fault
attack when the parity bits output wrong results. The techniques are generic
and we can apply to general algorithms with simple modifications. However,
the approach also causes 3.5–4.7 times performance overheads and cannot cap-
ture the instruction set level faults. In SAC’16, the intra-instruction redundancy
based fault attack countermeasure is suggested [12]. The method implemented
the redundant bit-slicing and provides the ability to detect both instruction
faults and computation faults. However, the bit-slicing implementation is only
efficient over certain computers with a number of general purpose registers and
the modern computer architectures mainly support powerful SIMD instruction
set. Under this modern computer environments, the bit-slicing method cannot be
the best choice. Furthermore, previous works failed to describe how to generate
the random sequences for random shuffling and perform random shuffling on the
word. In this paper, we propose the efficient intra-instruction redundancy over
6 H. Seo et al.

SIMD instruction set. This new method simultaneously generates the random
sequences when the random number is needed. Finally, we applied to the LEA
encryption to achieve high security against fault attacks.

Table 1. Instruction set summary for ARM–NEON

Mnemonics Operands Description Cycles

VADD Qd,Qn,Qm Vector Addition 1
VORR Qd,Qn,Qm Vector OR 1
VEOR Qd,Qn,Qm Vector Exclusive-or 1
VSWP Qd,Qn Vector Re-ordering 1
VSHL Qd,Qm,#imm Vector Left Shift 1
VSRI Qd,Qm,#imm Vector Right Shift with Insert 2

2.3 Target Block Cipher: LEA

In 2013, new block cipher LEA was announced by the Attached Institute of ETRI
[6]. This new algorithm has simple Addition-Rotation-eXclusive-or (ARX) and
non-S-box architecture for high performance on both software and hardware
environments. The LEA has 128-bit block size and the word size is 32-bit wise.
Three different security levels including 128-bit, 192-bit and 256-bit are avail-
able. The number of rounds is 24, 28 and 32 for 128-, 192- and 256-bit keys,
respectively. The algorithm consists of key schedule, encryption and decryption
steps.

2.4 Target SIMD Platform: ARM–NEON

NEON engine is a 128-bit SIMD architecture for the ARM Cortex-A series. The
difference between traditional ARM processors and new ARM-NEON proces-
sors is NEON based SIMD architecture. The NEON engine offers 128-bit wise
registers and instruction sets. Each register is considered as a vector of elements
of the same data type and this data type can be signed/unsigned 8-bit, 16-bit,
32-bit, or 64-bit. The detailed instructions for ARX operations are described in
Table 1. A number of general arithmetic operations (e.g., Addition, logical or,
logical exclusive-or) are defined. The SIMD feature provides precise operation in
various vector sizes, performing multiple data in single instruction. With SIMD
features, the NEON engine can accelerate data processing by at least 3X that
provided by ARMv5.
The SIMD implementation can boost previous implementations by using
SIMD instruction sets. In CHES’12, NEON-based cryptography implementa-
tions including Salsa20, Poly1305, Curve25519, and Ed25519 were presented [3].
To improve the performance, the authors provided novel bit-rotation, integra-
tion of multiplication and reduction operations in NEON instructions. In CT–
RSA’13, ARM–NEON implementation of Grøstl shows that 40 % performance
 Intra-instruction Redundancy Based Fault Attack Countermeasure 7

enhancements than the previous fastest ARM implementation [5]. In HPEC

2013, finite field multiplication is optimized by using a multiplicand reduction
method. Particularly, they used ARM-NEON platform and improved the NIST
curves [4]. In CHES 2014, the Curve41417 implementation adopts 2-level Karat-
suba multiplication in the redundant representation [2]. In ICISC 2014, Seo et
al. introduced a novel 2-way Cascade Operand Scanning (COS) multiplication
for RSA implementation, which avoids the pipeline stall problems [16,17]. In
ICISC 2015, Seo et al. introduced an efficient modular multiplication and squar-
ing operations for NIST P-521 curve, which combines Karatsuba algorithm and
modular reduction efficiently [18]. Recently, Ring-LWE implementation is also
accelerated by taking advantages of NEON instructions [9].

2.5 Previous Implementations of LEA on ARM–NEON

In [6], LEA implementation takes advantages of 32-bit word wise instruction

sets and multiple load/store operations on ARM processor. The implementa-
tion achieved higher performance than AES implementations [10]. In [19], the
first LEA implementation on NEON engine is suggested, which accelerates the
performance of LEA encryption in SIMD instructions. For performance enhance-
ments, the interdependency between each instruction set is removed and multi-
ple operations are performed in parallel way. In [20], several new techniques to
improve the LEA block cipher in parallel way further were studied. The author
suggested the efficient combination of ARM and NEON instruction sets, which
fully utilizes both instruction sets. Another line of study is side channel attack.
In [8], Kim and Yoon performed the first experimental result of power analysis
attacks on LEA implementation on FPGA boards. They perform the correlation
power analysis (CPA) based on a linear relationship between measured power
consumption and an intermediate value. The attack showed that the straightfor-
ward implementation of LEA is very vulnerable to CPA. In [7], Jap and Breier
proposed a Differential Fault Analysis attack on LEA. By injecting random bit
faults in the last round and penultimate round, they were able to recover the
secret key by using 258 faulty encryptions in average. The works showed that
LEA implementation should consider the countermeasures against side channel
attacks. However, previous works mainly concerned on high-speed implementa-
tion rather than secure implementation. By considering the importance of secure
LEA implementation, we need to further study on secure implementations of
LEA. In this paper, we propose the several new techniques to ensure the high
security of LEA block cipher in SIMD architecture. The proposed countermea-
sures are not limited to the LEA block cipher but they can be applied to the
other block ciphers including SPECK and SIMON with simple modifications.

3 Proposed Method

In this paper, we propose new secure design of LEA implementation against fault
injection attack. The target platform (ARM–NEON) supports 128-bit vectorized
8 H. Seo et al.

instruction sets. We implement the LEA operations in 32 × 4 vector type, which

performs one 32-bit wise operation for four 32-bit data sets. To ensure security
against fault injection attack, we construct the intra-instruction redundancy
in SIMD architecture. In the 32 × 4 vector type, we put four different word
types including random number generator (RN G), redundant random number
generator (RRN G), plaintext (W ), and redundant plaintext (RW ). First, the
random number is generated through LEA encryption with counter mode of
operation [13], performing the main LEA encryption. In every encryption, the
random number is used to randomly shuffle the intermediate results in the NEON
register. Second, the duplicated random number (RRN G) is placed in second
word slot. After the encryption, the original random number and duplicated
random number are compared to check whether the target platform is attacked
by adversaries or not. Third, the original plaintext (W ) is stored in one word
slot. Lastly, the duplicated plaintext (RW ) is placed in last word slot. After
the encryption, the original plaintext and duplicated plaintext are compared to
check whether the target platform is attacked by adversaries or not. With above
data packet setting, we established the secure LEA implementation as follows.

Parameter Loading → Message Duplication →

Message Shuffling #1 → Round Function #1 → · · · →
Message Shuffling #24 → Round Function #24 → Last Message Shuffling

First, we loaded all parameters including random number generator seed and
plaintext from the memory to the NEON register. Among them, the loaded
plaintext is duplicated and stored into redundant plaintext. In the 32-bit ARM
register, we load 32-bit random number. Afterward, the message is randomly
shuffled with the random number. The process also shuffles the round key, which
ensures the correct round keys for encryption. When both message and round
key are properly shuffled, the round function is performed. For optimized ARX
operations for LEA encryption, we follow the implementation techniques covered
in [20]. The combination of message shuffle and round function steps perform 24
times to complete the 128-bit LEA encryption. Lastly, the shuffled results are
reordered in last message shuffle step.
The detailed message shuffle step is described in Fig. 1. First, we generate
the shuffled message by using swap instruction. Afterward, both original and
shuffled messages are masked with random number bit (rb). Depending on the
random bit (rb), we generate two different mask words (0 − rb, 232 − 1 ⊕ (0 − rb)).
When the random bit is set to 1, it generates mask words (232 −1, 0). Otherwise,
it generates opposite mask words (0, 232 − 1). After the mask operation, only
one message format maintains the value. Afterward, we perform OR operation to
choose and output one message format.
The detailed message shuffling codes are described in Algorithm 1. In Step 1,
the random number (r4) is rotated to right by 1. This always keeps the random
number bit fresh during 24 rounds. In Step 2–3, we extract the least significant
bit and subtract the zero with the bit to generate the value (0 or 232 − 1). In
Step 4, the output of previous step is exclusive-ored with 232 − 1. In Step 5, the
 Intra-instruction Redundancy Based Fault Attack Countermeasure 9

(1) swap (2) mask

RNG RRNG W RW
(3) or
(4) output
W RW RNG RRNG (2) mask

Fig. 1. Intra-instruction redundancy with random number generation and mask tech-
nique, where RN G: random number generator, RRN G: redundant random number
generator, W : plaintext in word, RW redundant plaintext in word

Algorithm 1. Message shuffling in ARM–NEON, where r4: random number,

r5: current state, r8: zero, r9: 232 − 1, q0–q3: plaintext
1: ror r4, r4, #1 14: vdup.32 q12, r6
2: lsl r6, r4, #31 15: vdup.32 q13, r7
3: sub r6, r8, r6, lsr #31
4: eor r7, r9, r6 16: vand q0, q0, q12
5: eor r5, r5, r6 17: vand q1, q1, q12
18: vand q2, q2, q12
6: vmov q4, q0 19: vand q3, q3, q12
7: vmov q5, q1 20: vand q4, q4, q13
8: vmov q6, q2 21: vand q5, q5, q13
9: vmov q7, q3 22: vand q6, q6, q13
23: vand q7, q7, q13
10: vswp d0, d1
11: vswp d2, d3 24: vorr q0, q0, q4
12: vswp d4, d5 25: vorr q1, q1, q5
13: vswp d6, d7 26: vorr q2, q2, q6
27: vorr q3, q3, q7

current random bit is exclusive-ored with current state register (r5). The current
state register is used to track the current state of shuffling in the message and
perform the last message shuffling. In Step 6–9, the plaintext is duplicated from
the NEON registers (q0–q3) to other registers (q4–q7). In Step 10–13, the lower
and higher values in the register are exchanged. In Step 14–15, the mask bits
(r6, r7) are duplicated to the NEON registers (q12, q13). The NEON registers
can retain two states [(0, 2128 − 1) or (2128 − 1, 0)]. In Step 16–23, the plaintext
and shuffled plaintext are masked with mask registers. Lastly both plaintext
registers are combined together in Step 24–27 to generate the output.
After 24 times of message shuffling and round function, the results are shuf-
fled again to be aligned. In Step 1, the opposite state of current state (r5)
is stored into register (r7). In Step 3–6, the plaintext is duplicated from the
NEON registers (q0–q3) to the other registers (q4–q7). In Step 7–10, the lower
and higher values in the register are exchanged. In Step 11–12, the mask bits
(r6, r7) are duplicated to the NEON registers (q12, q13). In Step 13–20, the
plaintext and swapped plaintext are masked with mask registers. Lastly both
plaintext variables are added together in Step 21–24.
10 H. Seo et al.

Algorithm 2. Last message shuffling in ARM–NEON, where r5: current state,

r9: 232 − 1, q0–q3: plaintext
1: eor r7, r9, r5
2: mov r6, r5 13: vand q0, q0, q12
14: vand q1, q1, q12
3: vmov q4, q0 15: vand q2, q2, q12
4: vmov q5, q1 16: vand q3, q3, q12
5: vmov q6, q2 17: vand q4, q4, q13
6: vmov q7, q3 18: vand q5, q5, q13
19: vand q6, q6, q13
7: vswp d0, d1 20: vand q7, q7, q13
8: vswp d2, d3
9: vswp d4, d5 21: vorr q0, q0, q4
10: vswp d6, d7 22: vorr q1, q1, q5
23: vorr q2, q2, q6
11: vdup.32 q12, r6 24: vorr q3, q3, q7
12: vdup.32 q13, r7

W#1 RW#1 W#2 RW#2

W#3 RW#3 W#4 RW#4
Fig. 2. Intra-instruction redundancy without random number generation, where W :
plaintext in word, RW redundant plaintext in word

For one encryption, we need 24 random bits for 24 times of message shuffling.
In our implementation, we generate 128-bit random number with 128-bit LEA
implementation. One time of random number generation can cover 4 times of
LEA encryption. To accelerate the performance, we perform 2-way encryption
techniques. First, we generate the random number with message packet described
in Fig. 1. This mode only performs one encryption per computation. Afterward,
we perform the encryption only mode as described in Fig. 2. This mode performs
four encryptions per computation. The message packet consists of plaintext and
redundant plaintext.
The detailed descriptions of 2-way encryption are described in Algorithm 3.
In Step 1, we calculate the number of messages. When the number of message is
equal or larger than 4 128-bit messages, we perform one encryption with random
number generator and four encryption without random number generator in Step
3–6. If the message size is not long enough, we only perform the encryption with
random number generator as described in Step 7–9. Finally, in Step 10, we output
the ciphertext.
 Intra-instruction Redundancy Based Fault Attack Countermeasure 11

4 Evaluation
For performance evaluation, we selected the ARM Cortex-A9 board, which is
32-bit processor with L1 cache and L2 cache. The board is widely used in mini
computers such as PandaBoard/Odroid development platforms. Particularly, our
target platform supports quad-core operated at 1.4 GHz and NEON engine. The
program codes are written in assembly language and compiled with NDK android
library. The performance is measured in system time function.

Algorithm 3. 2-way encryption, where EN C1 : 1 encryption with random

number generation, EN C2 : 4 encryptions only
Require: Plaintext P , Length of plaintext L (in bytes)
Ensure: Ciphertext C
1: N ← L/16, T ← 0
2: while N do
3: if N ≥ 5 then
4: C[T ] ← EN C1 (P [T ])
5: C[T + 1 : T + 4] ← EN C2 (P [T + 1 : T + 4])
6: N ← N − 5, T ← T + 5
7: else
8: C[T ] ← EN C1 (P [T ])
9: N ← N − 1, T ← T + 1
10: return C

The comparison results are drawn in Table 2. In [6], the LEA implementation
on ARM instruction sets achieved the 20.06 cycle/byte. This implementation
does not take advantages of NEON instruction sets. In [20], the performance
is improved by taking advantages of barrel shifter instruction. The technique
optimizes the rotation overheads and enhances the performance by 13.8% for
ARM platform. In terms of NEON instruction, the parallel LEA implementa-
tion achieved the 10.06 cycle/byte in [19]. The works used several optimization
techniques such as efficient ARX instructions in NEON and pipelined imple-
mentations. In [20], the author further improved the performance of LEA imple-
mentation. First, they suggested the compact round key access techniques. The
method reduces the required registers for round keys and increases the number
of plaintext for encryption in each round. Second, ordinary format is efficiently
converted into SIMD format with transpose and swap instructions. With these
techniques, NEON implementation is improved by 15.5%. Third, the ARM and
NEON instruction sets are finely integrated together, which hides the latency
for ARM instruction into NEON instruction. The results achieved the 20.4%
performance enhancements than [19].
However, previous implementations only achieved the highest performance.
For practical usages, the implementation should be secure against side channel
attacks. In this paper, we presented new fault injection attack countermeasures
12 H. Seo et al.

Table 2. Comparison of LEA implementations on ARM–NEON architectures in terms

of execution timing (c/b: cycle/byte) and security against fault attack

Method Speed(c/b) Instruction Fault Attack

Hong et al. [7] 20.1 ARM Insecure
Seo et al. [20] 17.3 ARM Insecure
Seo et al. [19] 10.1 NEON Insecure
Seo et al. [20] 8.5 NEON Insecure
Seo et al. [20] 8.0 ARM–NEON Insecure
Proposed Method ver 1 99.0 NEON Secure
Proposed Method ver 2 50.4 NEON Secure

in NEON instruction set. The countermeasure adds some overheads to the imple-
mentation and we achieved 99.0 cycles/bytes for secure LEA implementations on
ARM–NEON platform. For better performance, we used 2-way encryption tech-
nique, which improved the performance by 49.1 % and requires 50.4 clock/byte.
In terms of security model, we tested several different fault attack scenario
studied in previous works as follows [12].

– Random Word: The adversary has no control on the number of faulty bits.
The adversary can only create random faults in the target word (32-bit).
– Random Byte: The adversary can tune the fault injection to randomly
affect a single byte of the 32-bit data.
– Random Bit: The fault injection can be tuned to affect single bit of the
target word.
– Chosen Bit Pair: The adversary can inject faults into two chosen, adjacent
bits of the target word.

In the unprotected LEA implementation, any computation or instruction

fault injection attacks are easily exploited by the adversaries because this unpro-
tected model doesn’t have any countermeasures to prevent the fault injection
attacks. In the previous secure implementation by [12], they suggested the fault
attack countermeasure with bit-slice technique. However, the bitslicing imple-
mentation has many limitations in practice. First, the ARX based block cipher
(LEA, SIMON, and SPECK) cannot be implemented with bitslicing technique
due to non-linear addition operation generating carry bits. Second, the bitslic-
ing is efficient when the word size is small enough. In [12], they used very old
32-bit SPARC processor called LEON3. However, for modern 64-bit processors,
bitslicing is not efficient because the 64 encryptions are required to perform bit-
slicing method and it requires a number of general purpose registers (at least
128 registers for 128-bit block size). Considering that 64-bit INTEL processors
only support 16 64-bit general purpose registers, the bitslicing technique reduces
the performance significantly. Instead of bitslicing, we can accelerate the perfor-
mance significantly with powerful SIMD instruction set. Third, they suggested
the random shuffling technique but they failed to describe how to make it work.
 Intra-instruction Redundancy Based Fault Attack Countermeasure 13

In the word, bit-wise shuffling imposes huge overheads without the dedicated
bit-wise shuffling instruction but the target platform does not support it. On
the other hand, we fully describe the efficient random shuffling technique using
SIMD instruction set. The proposed technique is generic and can be used for
any modern processors.
In terms of security, the bitslicing is vulnerable to fault injection attacks
on chosen bit pair. If the attacker affects chosen bit pair such as plaintext and
its redundant plaintext, the users cannot figure out the fault injection attacks.
However, our technique is based on word wise computations so two chosen bits
cannot manipulate the two different words and same bit at once.
Moreover, we support the computations from one encryption. This feature
provides high scalability to utilize the encryption mode. For efficient random
number generation and encryption, we also support 2-way encryption mode. For
this reason, our approach is more secure and efficient than previous works on
the modern computers. The detailed comparison results are given in Table 31 .

Table 3. Security comparison of proposed method, where RNG: random number gen-
erator, RW: random word, RB: random byte, Rb: random bit, CbP: chosen bit pair,
IS: instruction skip

Method Instruction RNG RW RB Rb CbP IS

Seo et al. [20] SIMD
Patrick et al. [12] Normal
Proposed Method SIMD

5 Conclusion

In this paper, we presented new secure fault attack countermeasures for LEA
block cipher algorithm on representative SIMD architecture, namely ARM-
NEON platform. We achieved the intra-instruction redundancy by using 128-
bit wise NEON registers. This new technique successfully prevent several fault
attack models that is feasible in previous works. Furthermore, we perform the
2-way encryption for high-speed encryption and random number generation.
Finally, we achieved the 50.4 cycle/byte for secure LEA encryption.
The proposed methods improved the security of LEA block cipher on ARM-
NEON platforms. For this reason, there are many future works remained. First,
we can directly apply the fault attack countermeasures to the other ARX block
ciphers such as SPECK and SIMON. Recent works by [11,20] do not consider
the any secure measures proposed in this paper so we can enhance the security
by applying the proposed method. Second, we only explore the ARM-NEON
1
For reproduction of results, the source codes will be public domain in follow-
ing address. (https://github.com/solowal/WISA2017 SCA). The source code is
encrypted with the password (wisa2017).
14 H. Seo et al.

platform in this paper. However, INTEL processors also provide SIMD instruc-
tions such as AVX and SSE. The functionality of SIMD instructions are very
similar to NEON instructions so we can directly apply the proposed techniques
to other SIMD instruction sets without difficulty.

References
1. Barenghi, A., Breveglieri, L., Koren, I., Pelosi, G., Regazzoni, F.: Countermeasures
against fault attacks on software implemented AES: effectiveness and cost. In:
Proceedings of the 5th Workshop on Embedded Systems Security, p. 7. ACM (2010)
2. Bernstein, D.J., Chuengsatiansup, C., Lange, T., Schwabe, P.: Kummer strikes
back: new DH speed records. In: Sarkar, P., Iwata, T. (eds.) ASIACRYPT 2014.
LNCS, vol. 8873, pp. 317–337. Springer, Heidelberg (2014). https://doi.org/10.
1007/978-3-662-45611-8 17
3. Bernstein, D.J., Schwabe, P.: NEON crypto. In: Prouff, E., Schaumont, P. (eds.)
CHES 2012. LNCS, vol. 7428, pp. 320–339. Springer, Heidelberg (2012). https://
doi.org/10.1007/978-3-642-33027-8 19
4. Faz-Hernández, A., Longa, P., Sánchez, A.H.: Efficient and secure algorithms for
GLV-based scalar multiplication and their implementation on GLV-GLS curves.
In: Benaloh, J. (ed.) CT-RSA 2014. LNCS, vol. 8366, pp. 1–27. Springer, Cham
(2014). https://doi.org/10.1007/978-3-319-04852-9 1
5. Holzer-Graf, S., Krinninger, T., Pernull, M., Schläffer, M., Schwabe, P., Seywald,
D., Wieser, W.: Efficient vector implementations of AES-based designs: a case
study and new implemenations for Grøstl. In: Dawson, E. (ed.) CT-RSA 2013.
LNCS, vol. 7779, pp. 145–161. Springer, Heidelberg (2013). https://doi.org/10.
1007/978-3-642-36095-4 10
6. Hong, D., Lee, J.-K., Kim, D.-C., Kwon, D., Ryu, K.H., Lee, D.-G.: LEA: a 128-
bit block cipher for fast encryption on common processors. In: Kim, Y., Lee, H.,
Perrig, A. (eds.) WISA 2013. LNCS, vol. 8267, pp. 3–27. Springer, Cham (2014).
https://doi.org/10.1007/978-3-319-05149-9 1
7. Jap, D., Breier, J.: Differential fault attack on LEA. In: Khalil, I., Neuhold, E.,
Tjoa, A.M., Da Xu, L., You, I. (eds.) CONFENIS/ICT-EurAsia -2015. LNCS,
vol. 9357, pp. 265–274. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-
24315-3 27
8. Kim, Y., Yoon, H.: First experimental result of power analysis attacks on a FPGA
implementation of LEA. IACR Cryptology ePrint Archive 2014:999 (2014)
9. Liu, Z., Azarderakhsh, R., Kim, H., Seo, H.: Efficient software implementation of
Ring-LWE encryption on IoT processors. IEEE Trans. Comput. (2017)
10. Osvik, D.A., Bos, J.W., Stefan, D., Canright, D.: Fast software AES encryption.
In: Hong, S., Iwata, T. (eds.) FSE 2010. LNCS, vol. 6147, pp. 75–93. Springer,
Heidelberg (2010). https://doi.org/10.1007/978-3-642-13858-4 5
11. Park, T., Seo, H., Kim, H.: Parallel implementations of SIMON and SPECK. In:
2016 International Conference on Platform Technology and Service (PlatCon), pp.
1–6. IEEE (2016)
12. Patrick, C., Yuce, B., Ghalaty, N.F., Schaumont, P.: Lightweight fault attack resis-
tance in software using intra-instruction redundancy. In: Avanzi, R., Heys, H. (eds.)
SAC 2016. LNCS, vol. 10532, pp. 231–244. Springer, Cham (2017). https://doi.org/
10.1007/978-3-319-69453-5 13
 Intra-instruction Redundancy Based Fault Attack Countermeasure 15

13. Seo, H., Choi, J., Kim, H., Park, T., Kim, H.: Pseudo random number generator
and hash function for embedded microprocessors. In: 2014 IEEE World Forum on
Internet of Things (WF-IoT), pp. 37–40. IEEE (2014)
14. Seo, H., Kim, H.: Low-power encryption algorithm block cipher in JavaScript. J.
Inf. Commun. Converg. Eng. 12(4), 252–256 (2014)
15. Seo, H., Liu, Z., Choi, J., Park, T., Kim, H.: Compact implementations of LEA
block cipher for low-end microprocessors. In: Kim, H., Choi, D. (eds.) WISA 2015.
LNCS, vol. 9503, pp. 28–40. Springer, Cham (2016). https://doi.org/10.1007/978-
3-319-31875-2 3
16. Seo, H., Liu, Z., Großschädl, J., Choi, J., Kim, H.: Montgomery modular multi-
plication on ARM-NEON revisited. In: Lee, J., Kim, J. (eds.) ICISC 2014. LNCS,
vol. 8949, pp. 328–342. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-
15943-0 20
17. Seo, H., Liu, Z., Großschädl, J., Kim, H.: Efficient arithmetic on ARM-NEON
and its application for high-speed RSA implementation. IACR Cryptology ePrint
Archive 2015:465 (2015)
18. Seo, H., Liu, Z., Nogami, Y., Park, T., Choi, J., Zhou, L., Kim, H.: Faster ECC over
F2521 −1 (feat. NEON). In: Kwon, S., Yun, A. (eds.) ICISC 2015. LNCS, vol. 9558,
pp. 169–181. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-30840-
1 11
19. Seo, H., Liu, Z., Park, T., Kim, H., Lee, Y., Choi, J., Kim, H.: Parallel implemen-
tations of LEA. In: Lee, H.-S., Han, D.-G. (eds.) ICISC 2013. LNCS, vol. 8565, pp.
256–274. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-12160-4 16
20. Seo, H., et al.: Parallel implementations of LEA, revisited. In: Choi, D., Guilley, S.
(eds.) WISA 2016. LNCS, vol. 10144, pp. 318–330. Springer, Cham (2017). https://
doi.org/10.1007/978-3-319-56549-1 27
21. Van Nguyen, H., Seo, H., Kim, H.: Prospective cryptography in NFC with the
lightweight block encryption algorithm LEA. In: Dang, T.K., Wagner, R., Neuhold,
E., Takizawa, M., Küng, J., Thoai, N. (eds.) FDSE 2014. LNCS, vol. 8860, pp. 191–
203. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-12778-1 15
 Exposing Digital Forgeries by Detecting
a Contextual Violation Using Deep
Neural Networks

Jong-Uk Hou, Han-Ul Jang, Jin-Seok Park, and Heung-Kyu Lee(B)

School of Computing, KAIST, Daejeon, Republic of Korea

heunglee@kaist.ac.kr

Abstract. Previous digital image forensics focused on the low-level fea-

tures that include traces of the image modifying history. In this paper,
we present a framework to detect the manipulation of images through
a contextual violation. First, we proposed a context learning convolu-
tional neural networks (CL-CNN) that detects the contextual violation
in the image. In combination with a well-known object detector such as
R-CNN, the proposed method can evaluate the contextual scores accord-
ing to the combination of objects in the image. Through experiments,
we showed that our method effectively detects the contextual violation
in the target image.

1 Introduction
In the age of high performance digital camera and the Internet, digital images
have become one of the most popular information sources. Unlike text, images
remain an effective and natural communication medium for humans, for their
visual aspect makes it easy to understand the content. Traditionally, there has
been confidence in the integrity of visual data, such that a picture in a newspaper
is commonly accepted as a certification of the news. Unfortunately, digital images
are easily manipulated, especially since the advent of high-quality image-editing
tools such as Adobe Photoshop and Paintshop Pro. Therefore, digital-image
forensics, a practice aimed at identifying forgeries in digital images, has become
an important field of research.
To cope with image forgeries, a number of forensic schemes were proposed
recently. Most of the previous digital image forensics focused on the low level
features that include traces of the image modifying history. They are based on
detecting local inconsistencies such as resampling artifacts [14], color filter array
interpolation artifacts [2], JPEG compression [3], and so on. The pixel photo
response non-uniformity (PRNU) is also widely used for detecting digital image
forgeries [1,6,7]. There are also some methods for detecting identical regions
caused by copy-move forgery [8,16].
Most of the local inconsistency based methods are vulnerable to the general
image processing such as JPEG and GIF compression, white balancing, and noise
addition. On the other hand, high-level features such as lighting condition [10],
c Springer International Publishing AG, part of Springer Nature 2018
B. B. Kang and T. Kim (Eds.): WISA 2017, LNCS 10763, pp. 16–24, 2018.
https://doi.org/10.1007/978-3-319-93563-8_2
 Exposing Digital Forgeries by Detecting a Contextual Violation 17

Fig. 1. Examples of image forgery detection based on contextual violation. (left) The
boat was manipulated on the road, and (right) the lamb was found in the office. The
manipulated parts (areas in the red box) can be detected because these objects caused
contextual violations. (Color figure online)

shading and shadows [11] provide fairly robust clues for the forgery detection
system against the general image processing. In this paper, we present a foren-
sic scheme to detect the manipulation of images through contextual violation
based on high-level objects in target image (See Fig. 1). Relationship informa-
tion between objects in an image is used as a robust feature that is not affected
by general image processing.
Our research proposes a model that can learn the context by learning the
image label combination by spatial coordinate. First, we proposed a context
learning convolutional neural networks (CL-CNN) that detects the contextual
violation in the image. CL-CNN was trained using an annotated large image
database (COCO 2014), and learned spatial context that was not provided by
existing graph-based context models. In combination with a well-known object
detector such as R-CNN, the proposed method can evaluate the contextual scores
according to the combination of objects in the image. As a result, our method
effectively detects the contextual violation in the target image.

2 Context-Learning CNN
In this research, we proposed Context-Learning convolutional neural networks
(CL-CNN) to learn co-occurrence and spatial relationship between object cat-
egories. The proposed CL-CNN provides an explicit contextual model through
deep learning with large image databases. CL-CNN is learned to return a high-
value for the natural (or learned) combination of the object categories, while
it returns low value for strange combinations or spatial contexts of the image
labels.

2.1 Input Data Structure

The process of generating input data is as follows. We used an annotated image
database for object location and category information. Because size of the image
18 J.-U. Hou et al.

Fig. 2. Input data structure and encoding process

is too large to be used, the size is reduced to N ×N , where N < height and width
of the input image, to record the position information of the object. The channel
size of the input data structure is the total number of categories to distinguish
each category. The label defined part is padded with 1 value and the remaining
blank area is filled with 0 values.
In this research, we reduced the size of the image to 8 × 8 and chose the
category with 80. The final input data size is 8 × 8 × 80 and the generation
process is shown in Fig. 2.

2.2 CL-CNN Structure

The structure of CL-CNN is as follows (See Fig. 3). It receives input data of
8 × 8 × 80, passes through two convolutional layers, and passes through three
fully connected layer. Then the fully connected layer finally outputs 2 × 1 vector.
The first value of the output is a score that evaluates how natural the label is in
combination with the category and spatial context, and the second value is the
score that evaluates how the category combination and spatial context of the
label are awkward. The loss function used Euclidian loss L which is defined by
1
L= (yi − ai )2 , (1)
2 i

where y is the output of the CL-CNN, and a is the label of the data sample.

2.3 Dataset Generation

In order to learn the proposed network, it is necessary to acquire a large amount

of datasets. We need to have both a collection of natural labels and a collection
of unnatural labels. Moreover, we also need data that shows both the location
and type of the object. A dataset that meets these criteria is Microsoft COCO:
Common Objects in Context [13]. Microsoft COCO 2014 provides 82,783 train-
ing image sets and label information, and 40,504 validation images and label
information. This can aid in learning detailed object models capable of precise
2D localization and contextual information.
Before we use the dataset, we excluded single-category images since they are
useless for learning contextual information. Thus, we used 65,268 multi-category
images for learning of the CL-CNN. The categories are divided into 80 categories.
 Exposing Digital Forgeries by Detecting a Contextual Violation 19

Fig. 3. Overall structure of the context-learning convolutional neural networks

A positive set was constructed using label information of t multi-category images,

as the way described in Sect. 2.1.
We cannot use negative sets based on the existing databases because they
need to learn combinations of unnatural labels that do not actually exist. For this
reason, we generated the negative set in two ways. Negative set 1 was created by
changing the size and position of the object while maintaining the category com-
bination. Negative set 2 was created by selecting the combination of the less cor-
related categories. Figure 4 shows the histogram of the co-occurrences between
object categories. Using the probability P (c1 , c2 ) from the co-occurrence his-
togram, combinations of classes c1 , c2 , which has which has a low co-occurrence
probability P (c1 , c2 ), was selected to generate a negative dataset. Next, the neg-
ative set 2 was modified by changing the size and position of the object while
maintaining the category combination.

2.4 Network Training

With a simple learning approach, object combination and location shuffled

dataset are trained at the same time. Next, we tested ‘combination and location
shuffling’ and ‘location shuffling’ and obtained the accuracy of 0.97 and 0.53,
respectively. When learning ‘combination and location shuffling’ at the same
time, ‘combination change’ was strongly learned. As a result, learning of the
spatial context was ignored by the over-fitted CL-CNN. Therefore, we need to
improve the learning method so that the ‘location shuffling’ was learned enough.
Therefore, we trained CL-CNN by learning ‘location shuffling’ of object first,
and then fine-tuning the ‘combination and location shuffling’ part sequentially.
We set the learning rate to 0.001 for ‘location shuffling’ and set the learning
rate to 0.00001 for ‘combination and location shuffling’ learning. We also tested
‘combination and location shuffling’ and ‘location shuffling’ and obtained an
accuracy of 0.93 and 0.81, respectively. The test accuracy for ‘location shuffling’
Another random document with
no related content on Scribd:
valiant, where now is thy boasted courage?’ ‘Pardon me,’ he then
said, ‘I humbly beg, and I shall most willingly deliver up the arms; for
I have done much amiss.’ I accordingly took the arms from him, and
went off, much pleased with the adventure.
All these things being now effected, on the days after I had first
seen His Grace the Duke of Perth, we began our march for Preston;
and on the road, passing by a house of a person, who had in the
year 1715 been the chief cause of my Father’s misfortune, I turned
my horse in order to have hanged the man and certainly would have
done it, had not maturer thoughts intervened, and stopt my
revengeful rage. Continuing our journey, we arrived at Preston about
2 o’clock in the afternoon, having with us 39 English Recruits, whom
I presented to His Grace the Duke of Perth, giving him an account of
what had passed. At first he seemed to be somewhat concerned;
but, recollecting himself, he afterwards welcomed me back, and said,
he doubted not soon to see a great joining, and desired me to come
to him the next day—for the Army halted two days at Preston. So,
taking leave, I went to see some of my friends in the Town; and,
amongst the rest, to acquaint my Father with what I had done. He
approved much of it, and gave me very salutary advice, telling me
always to have the love and fear of God before me, and never to
deviate from my duty, but to act to the best in my power the part of a
brave soldier, and never to deject but comfort all those I found in
misery. He then said to me, that as the infirmities of old age no
longer permitted him to espouse so good a cause, in which he had
once been actively engaged, he would continually invoke the
Almighty for our success and preservation. So kissing me, he gave
me his kind benediction: and with the viaticum in my pocket I took
leave of the tenderest of Fathers and best of Parents.
It being now the day on which the army marched from Preston, I
waited, according to appointment, on the Duke of Perth, who told
me, if I pleased, he would give me a Captain’s Commission in his
Regiment; or, as one Mr. Gorogan[390] was to have a Colonel’s
Commission over the English, that I should have a Company under
him, and command the men I brought with me to Preston. The latter
offer I accepted; and on being recommended to the Colonel by the
Duke, I was made Captain; and Captain Larrey,[391] now living at St.
Omers, was the other Captain, with 3 or 4 more. So we with the
army began our march for Wiggan, where we were joined by a few
more Loyalists. Having lain all night at Wiggan, we marched the next
day for Manchester. The ringing of the Bells, and the great rejoicings
and salutations with which we were welcomed, gave us mighty
expectations. But too true is the saying: Parturiunt Montes, nascetur
ridiculus mus.
Word was immediately brought to the Prince, that a number of
men were at his service; and to please and content the Town, it was
thought necessary, that what men were raised at Manchester—vizt.,
the English Regiment—should be called the Manchester Regiment,
and all inferior Officers displaced as not being sufficiently
Manchesterfied. However, regarding how matters went, I observed a
little Man, by name Morgan,[392] deputed by the Prince with orders to
inspect and commission new Candidates, come into the room
appointed for that purpose, and after salutations made, take his
place at the head of the Table, with the Blackguard Dog[393] at his
elbow, whether coming there by orders or impudence, I know not.
Mr. Morgan began to tell the reason of his being sent, saying, that
His Royal Highness was highly charmed at the report he had heard
of the great number of Manchester men who were to join his
standard; and assured them all of his particular protection, and of his
willingness to grant them every favour in his power. This Declaration
gave great joy to all present.
A dispute then arose concerning the making of a new Colonel:
[394] but taking a dislike at some of their proceedings, I gave up all
pretentions to anything amongst them, and joined Lord Elcho’s
Guards;[395] so the place was vacant. The rest however were called
upon to be Regimental Captains, and so on, according as the
aforesaid pursuivant of Mr. Morgan notified: for, on demand who was
to be the first Captain, all, conscious of each other’s merit, were
silent; till he, with a face of assurance, named such a man, for he
had great interest; after him, another, for he could raise a great
number of men; and after him another, for he had great merit and
power; till all the Captain’s Commissions were disposed of according
to his direction; and then, looking about him, he said of the rest, it
was hang choice which was pitched upon first.
The Manchester Officers, being now formed, agreed to petition
the Prince to stay another day there; which he agreed to in the
expectation of raising a great number of men. I was as credulous as
they: but was much surprised to see the next day those men whom I
had brought from behind Preston, and on the road thence to that
place, enrolled for the most part in the Manchester Regiment, and
thus Manchesterised, if it may be said so: and much troubled I was
to see many of those men who had followed me, had been paid out
of my own pocket and been under my care till our arrival in
Manchester, disposed of, nay taken away from me in that manner.
Quos Ego—sed motos praestat componere fluctus.
However, being willing to be as useful as possible to the cause, I
acquiesced in whatever they thought proper. The Prince, tho’ again
requested to stay, being weary of delay to no purpose, ordered the
following day the army to make a short march, and gave leave to the
Manchester Regiment to stay a day longer to get more recruits, on
promise to march up to the army the day after: which we did; but our
stay was not productive of much benefit.
The Comand of the Army, which till then had been the Duke of
Perth’s, was at this time given to Lord George Murray.[396] The real
cause of this change I cannot pretend to divine: all I can say, is that
the received opinion amongst us was that Lord George being looked
upon as a man, whose name would bear a greater sway in England,
especially amongst loyal Protestants, and help to efface the
prejudice and notion of Popery and arbitrary Power, which some,
though vainly imagined were rushing in like a torrent upon them; it
was just proper to place the chief command in him: and the Duke of
Perth, for the good of his King and Country, most readily resigned,
shewing himself willing to promote the cause in any station, and
giving a notable example of a brave warrior, willing to command and
willing to obey. Whether there was any other secret reason, I must
leave it for time to unfold.
 The Army being now arrived at Congleton in Cheshire, nothing
particular happened there, except that a patrol took one Captain
Wier, a famous spy,[397] with seven dragoons, who were feasting at
a house some distance off. This Wier was by birth a Scotchman, and
had been employed in many villanies, and having served the Court
not only as a spy upon us, but amongst other foreign Powers, had
been promoted for his diligence in this business. He was conducted
back with us to Carlisle—how unfortunate for us, that he was not put
to death, considering what he has since done!—but his life was
saved through the innate clemency of the Prince, though he merited
the worst of punishments.
On the first of December we departed from Macclesfield, in order
to march to Leek; where we staid all night, and marched the next day
for Derby. All that morning it was rumored amongst us, that we
should have an engagement as the enemy’s army was said to be
lying about five or six miles from us at Newcastle and Stone in
Staffordshire. So we marched in the best order we could to receive
them: and about eleven o’Clock, having espied a party at some
distance on the mountains, we drew up in order of battle, and stood
so for some time, and would have fought them: but perceiving it was
a false alarm, we continued on our route to Derby, where we arrived
somewhat late and fatigued. But two days’ repose sufficiently
refreshed us.
Derby is a large and handsome town. The heads of it were much
terrified at our entrance, many of them having made large
subscriptions to Government; and therefore had quitted their houses
with the utmost precipitation. It fell to my lot to be quartered in one of
them, vizt., one Mr. Chambers. Coming in with my billet, I asked if I
could lodge there. The Steward immediately replied that I could—
adding ‘And any thing we can do for you, shall be done: only pity us
in our situation, which is most deplorable.’ At this wondering much
what he meant, I told him to be of good courage—that neither I nor
any of us were come to hurt him or any one. Having thus abated the
horrid notion they had of us, which was only capable of being
conceived too hard for expression, being so very strange; he
conducted me to the Housekeeper, who was also in tears. She was
somewhat seized with horror at the sight, though my countenance
was none of the roughest: but soon collected herself and made the
same answer with the utmost feminine tenderness, putting
themselves and the whole house under my mercy. I truly was much
surprised, for anything of this kind was quite new to me: however,
after pulling off my riding-coat and boots she conducted me into a
fine room; where, at entering, I perceived a number of jewels and
watches lying confusedly up and down, and many things else in the
utmost confusion. I demanded, to whom they belonged, and what
was the reason of their being so carelessly laid up. The housekeeper
then began to tell me the whole affair—‘Sir,’ said she, ‘Mr.
Chambers, the master of this house, no enemy to you, has retired
with his lady and family into the country,’ ‘Why so?’ said I. ‘Not
conscious,’ replied she, ‘of any thing particular against you, but out
of fear of what the Highlanders might do against him.’ She then
begged, that I would have compassion on them, and be their
protector; which, after some short discourse, I promised, telling
them, that what was consistent with reason, and a countryman, they
should always find in me. I then ordered all the things that lay so
confusedly thrown about, to be locked up, assuring them, that
nothing should be touched or broke open, unless with authority. So
for two days I ruled master there, and, I hope they will generously
acknowledge, much to their content and satisfaction in that situation
of affairs—having preserved the Young Lady’s jewels from the hand
of rapine, and hindered the house from being damaged.
A rumor was here spread amongst us, that Cumberland Will and
Ligonier[398] intended to give us battle; which I believe would have
happened, if we had marched a day or two more towards London.
Every one prepared himself to act in the best manner the valiant
Soldier. But the Prince’s Council judged it more proper to retire back
into Scotland without risking a battle, and there to await the arrival of
foreign Succors. How far they acted amiss or well in this, I know not:
but a great alteration was afterwards seen amongst us. The brave
Prince at that out of a generous ardour and Love to his country,
wished he had been twenty feet under ground! but, notwithstanding
all this, a march back was agreed upon, after we had halted two
days at Derby. Here I cannot pass by an accident that happened
somewhat ominous—though I am none of the most credulous—but
thence we may date our first misfortune. Great numbers of People
and Ladies (who had come from afar to see the Prince), crowding
into his room, overturned a table, which in falling overturned and
broke the Royal Standard—soon after our return was agreed upon—
so I leave the reader to judge and make his reflexions on this. It
would seem certain at least that Providence miraculously concurs,
while such and such things are carried on. Thus, when Moses held
up his hands, Joshua prevailed; but when through weariness he in
the least relaxed the Israelites had the worst of it. So perhaps it was,
that our enterprise was not vigorously enough pursued: and
remarkable it certainly was, that the Royal Standard should be
broken immediately after our return was resolved upon.
The third day being come, our march was proclaimed; and we
began our return,[399] wondering what it could forebode. About this
time we heard of Lord John Drummond’s Regiment having landed in
Scotland, and that more troops were daily expected from France.
Some few afterwards came, but the whole, including the said
Regiment, did not exceed Five Hundred effective men—too small an
assistance in the then state of our affairs![400] Soon after their arrival,
Lord Lewis Gordon, being joined by the foresaid regiment, defeated
Lord Loudon at Inverury, so completely dispersing his army, that it
was rendered ineffective during the remainder of the campaign.[401]
This happened very luckily for us; for if Lord Loudon had not met
with a check, he would probably have been able to collect a strong
army to cut off our retreat, or at least give us a very warm reception
on our return to Scotland. This news therefore gave us great
comfort; and we courageously continued our march to join the above
mentioned troops (in number one thousand men) by the same road
we travelled before. The English Army, being informed of our retreat,
immediately pursued us; but we found the saying to be very true ‘A
good pair of heels is worth two pair of hands.’ Diverting it was, to
hear those bells, which before had rung for us, turn to ring for them;
we sometimes going out at one end of a town, whilst they were
coming in at the other: and no less odd it was, to see the Magistrates
who had canted when the Prince was amongst them, immediately
after taking an opposite side, and cursing and detesting those whom
they had just before saluted with a Judas-kiss.
The Enemy, finding that they could not come up with us,
resolved, though very ungenerously to their own party, to endeavour
to raise the inhabitants of the towns through which we had to pass
against us, by spreading false reports, that the Prince’s Army had
been entirely defeated, and that nothing remained for them but to
exert themselves like good subjects, and to suffer none of the
fugitives to escape. This report was believed in many of the towns:
great rejoicings were made, and every man thought himself capable
of knocking out a Rebel’s brains with a club or a staff. Then you
might see heroic valour displayed among cocks that never crowed
but among hens upon their own dunghill. But the malicious
expectations of our enemies were disappointed; and what they had
contrived for our ruin redounded much to our profit. Notwithstanding
the mildness of the Prince during the march of his army through
England, and though he had suffered no one to be oppressed, we
heard betwixt Derby and Manchester, that the latter town had made
great rejoicing, and had raised some men to stop us: but we soon
made these mighty heroes tremble, and the town pay for their rash
determination for entering regularly and triumphantly, we shewed
them we were not the people they took us for, and convinced them,
that our situation was not so bad as had been falsely reported.
Every one therefore, vexed to the very heart at being so
deceived, began to lay upon the mob the fault of what had
happened, and the Mob on the Heads of the Place, so that discord
and confusion arose among them. The Piper, however was well paid
for their dancing; for it cost them five thousand pounds Sterling—
scarcely a sufficient atonement for their malice.[402] Here I cannot
pass by mentioning a barbarous deed perpetrated before we came
to Manchester which shewed the innate cruelty of our enemies on
the one hand, and our clemency on the other. A young English lad,
who had joined the Prince, being somewhat before the army, had
through weariness laid himself down to rest under a hedge, and
fallen fast asleep. He was soon perceived by a woman and her boy:
and this cruel fiend immediately determined to murder him as he lay
sleeping like a lamb, conscious of no harm; she accordingly with the
assistance of her son cut the poor young man’s throat. The army
coming up soon after, we espied the mangled body in that shocking
condition; and on searching the next house adjacent, we found a
young boy in bed much besmeared with blood, and trembling, who
confessed the fact, and said that his mother was the chief author of
it. They were both taken into custody, and a report of the whole
made to the Prince: but he was against their being put to death, so
that by a wonderful clemency they escaped the just reward of their
crime.
We halted two days at Manchester, and on the third marched for
Wiggan. When going out of the town, a gun was fired at the Prince
by a villain, who, mistaking him, shot at a Mr. Sullivan, and luckily
missed him. Search was made for him, but in vain—and no great
matter for any thing he would have suffered from us; for many
exercised their malice merely on account of the known goodness
and clemency of the Prince, which however they would not have
dared to do, if he had permitted a little more severity in punishing
them. The Army irritated by the frequent instances of the enemy’s
malice began to behave with less forbearance. And now few there
were, who would go on foot, if they could ride; and mighty taking,
stealing, and pressing of horses there was amongst us, for none of
us was ever sure of keeping his own. Diverting it was to see the
Highlanders now mounted without either breeches, saddle, or
anything else but the bare back of the horse to ride on; and for their
bridle, only a straw rope. In this manner we marched out of England,
many a good horse being brought in to give us a lift.
During our march to Wiggan, and thence to Preston, nothing
particular happened; only the enemy continued to pursue us, yet we
made no more haste on that account. I met upon the road my old
patron the Duke of Perth; who asking me how I did, and how I liked
the service. I told him ‘Very well!’ He then inquired ‘How I could bear
the thought of going into Scotland’—and I immediately replied That I
had ever been curious to see that kingdom, and was proud of
benefiting the cause, or occasion that was offered. His Grace was
pleased to promise, that he would recommend me to the Colonel,
who, he doubted not, would be a father to me; as it accordingly fell
out. Marching on to Preston, we halted there two days; and on the
third in the morning, immediately after we had quitted the town, the
enemy took possession of it. On our arrival at Lancaster in the
evening, I was recommended to Colonel Elphinstone, afterwards
Lord Balmerino,[403] who commanded the second troop of the
Prince’s Life Guards. Having equipped myself for that purpose when
in Preston, I rode on somewhat before the Army, to be in readiness
at Lancaster: but on the road, nine miles distant from that place,
alighting from my horse to refresh myself at a Public house, and
leaving my horse two or three minutes at the door; he was
immediately taken away by some of the soldiers, and I entirely lost
him. Vexed I was to be served so; and having nine miles to walk in
my boots sorely harrassed me. However, fortifying myself with
patience, and premeditating revenge, I at last, though somewhat late
got up to Lancaster: and now being in great anxiety for a horse,
being the next day to have the honour of riding in the Guards, I
resolved to wait an hour or two till my servant Dick, a Yorkshire-Man,
came up, and to make use of his horse, till such time as I could get
another. But his arrival brought me little satisfaction; for, after he had
refreshed himself, and I had told him the misfortune that had
happened to me, he went out, and wrote me a few lines to the
following purport:—
‘Dear Master,—I am truly sorry for your misfortune
but I do not much like the Army’s behaviour; neither can I
think of going into Scotland, and you know a Yorkshire-
man coming home without a horse is laughed at.
Therefore not doubting but you can provide yourself better
than with this, I beg leave to be your Humble Servant
Dick, promising to rejoin you, when I see you in these
parts again.’
He left these lines with the landlord of the house to give me;
which he did about an hour after.
 So, being served a true Yorkshire trick, I lost both man and horse.
I bethought myself of applying to one Mr. Grant,[404] Colonel of the
Artillery, thinking there might be a spare horse. When I had informed
him of what had happened, he promised that he would endeavour to
provide me with a horse the next morning, till he could buy or get
another; which kindness of his was very seasonable. But going down
the Castle-Hill, where the Artillery lay, I espied two Highlanders
stealing a horse, and breaking down a pair of barn-doors. When they
saw me advancing towards them, one of them went, and the other
stood guard. Now I thought this a good opportunity to provide myself:
So I went boldly in, and inquired whether Captain McDonald’s Horse
were not there; the man answered he knew not; at which taking
courage, and going up to the horse, I demanded whose that was. He
made me the same answer. ‘Friend,’ said I, ‘if you do not, I do,’ and I
immediately took the horse, well pleased, that I had got a Titt for Tatt
(as the Lancashire saying is). So returning to my lodgings with my
prize, I repaired to wait upon Colonel Elphinstone, who received me
very graciously, having been spoken to before by the Duke of Perth.
He welcomed me among them, assured me of all that was kind and
civil in his power, and then invited me to sup with him. After thanking
him, I accepted his kind invitation; and when I took my leave of him
he renewed his promises of kindness, and desired me to be in
readiness to mount next morning at his quarters.
At this time the Yorkshire Hunters[405] endeavoured to shew
themselves against us, but little to their honour. This was a regiment
composed of the Yorkshire Gentlemen, many of whom mounted
themselves and their servants on the best light horses they had, and
gave themselves the fore-said name. As we were marching into
Lancaster, these Hunters attacked some of our Light Horse in the
rear: but finding they had caught Tartars, they quickly retreated; and
being pursued, some of them were taken prisoners—so that
Yorkshire seemed to be put upon Yorkshire.
About this time the Prince’s Council resolved to despatch some
one with orders to Lord Lewis Gordon and Lord John Drummond,
and the rest of the Prince’s adherents who were in arms, to meet us
at Carlisle; and for this purpose the Duke of Perth set out attended
by about One-hundred Light-Horse. When they arrived at Kendal,
the inhabitants of that place, seeing them come in great haste,
judged they were flying from the battle and endeavouring to make
their escape, assembled in a tumultuous manner, and, after insulting
them, at last fired upon them out of the windows—and at going out of
the town, a ball was fired at the Duke of Perth in his Chaise, who,
looking out courageously, noticed the place it came from. Though the
ball happily missed the Duke it shot Captain Cameron’s horse. Being
now out of the town, we resolved boldly to re-enter it, and quell that
insulting mob. So, marching back, they took the man who fired the
fore-said ball, and killed two or three: yet this villain escaped with
only a few blows from the gentleman whose horse he had killed;
though certainly he would have been put to death, had not the Duke
interceded for him. The town of Kendal being thus appeased, we
continued our route: but hearing that the Militia of Pe[n]r[i]th and
other places, thinking our Army had been cut off, had risen, and
were, in conjunction with the Hessians who had landed thereabouts,
[406] intercepting the road they thought they were to pass; the Duke
resolved to pass by another in order to miss them. But his guide,
instead of shewing the right way, conducted them upon the enemy;
so that ascending the summit of the hill, they perceived their danger,
being within gun-shot of a great body of both horse and foot, which
unexpected sight struck a great panic amongst the Duke’s party. The
Duke was for fighting his way through them: but Colonel Bagott,[407]
Colonel of the Hussars, was of another opinion, and he being
seconded by some others, a retreat was agreed upon. The enemy
immediately dispatched some of their Light-horse in pursuit of them:
but the Duke’s party retiring in good order, kept their pursuers,
notwithstanding all their firing, at a good distance; and after being
pursued five or six miles, returned upon them with such vigour, that
they took three or four prisoners, rescued the Duke’s baggage,
which had just before been seized, and made them retire faster than
they had come. Finding it however in vain to pursue his enterprise,
the Duke rejoined the Army.
We having staid all night at Lancaster, I went, according to
appointment, to wait upon my Colonel, who, being mounted at the
head of his troops, placed me in his corps. We were ordered that day
to march in the rear of the army; and, as we left the town, the enemy
immediately took possession of it, and followed us some little way
out, so that we did not directly know their intention. But, finding they
soon returned, we marched on (nothing particular happening) to
Kendal where we halted all night and made the town pay for its past
behaviour. We then continued our retreat in exceeding bad weather
and roads to Carlisle: but by the severity of the season we were
obliged to make a shorter march than we intended, and halt all night
at a village called Shap, where we suffered very much on account of
the bad accommodation it afforded. However staying there all night
we assembled together early next morning. Some few of us got that
night to Carlisle. But I cannot pass by an affair that happened at
Clifton which was as follows:—The enemy’s advanced Guard,
commanded by General Honeywood, having got somewhat before
us, planted himself in ambush near the road we were to march, and
from behind the hedges expected, as the army approached to cut
them off; but there the biter was bit: for though we came unknowingly
upon them, we had the good fortune to receive their attack in good
order, killed and drowned many of them, and put the rest to flight:
General Honeywood himself narrowly escaped, having lost his arm,
and severely wounded. This brave action was chiefly owing to the
courage of the McPhersons, commanded by Clued McPherson,[408]
their Chief, who behaved most gallantly on this occasion; and most
pleasing it was, to see the champions come into Carlisle, loaded with
the spoils of their enemies.[409]
The whole army being now in Carlisle, our thoughts lay for some
time suspended in order to act for the best; for we supposed that the
enemy would come and give us battle. I cannot say that we
somewhat waited them, but finding they did not, it was judged proper
after two days’ stay and deliberation, to continue our march farther
North, and to leave at Carlisle the few English who were with the
army and about three hundred Scots. But of this proceeding I can
say nothing farther, leaving to the world to judge as they think proper.
The English then were about two hundred men; for many had quitted
and returned home being unwilling to go to Scotland. Now some
there are, who censure the Prince for leaving them at Carlisle,
thinking it was out of disregard and a desire of being rid of them: but
if they will pry a little more narrowly into the affair they will find, that it
was the desire of Mr. Townley Colonel of the English who not only
petitioned the Prince in his own name, but in the name of all the
officers of the Regiment, to be left there, though the latter never
assented to or desired it, many of them wishing to undergo the same
fate as their Royal Master. However on Mr. Townley’s coming back
and telling them that it was the Prince’s pleasure that they should
remain at Carlisle, they all, taking it as coming from the Prince, most
willingly acquiesced, shewing true English bravery in any situation to
obey:—and now, seeing themselves deprived of the satisfaction of
following him they so greatly loved, they resolved to accompany him
still in their hearts—all that was in their power. They shewed a true
generosity in disposing of all that was not of immediate use to them,
viz., their horses and riding-equipage, to all who stood in need; and
though I seemed somewhat picqued at them in the beginning of this
Narrative, I must ingenuously own, that they were a set of brave
men; and though it is often objected to them, that they were not of an
extraordinary rank, yet they behaved so as to make those of a nobler
birth blush; for, from the time they had the honour of joining the
Prince’s standard, they never sought pay either for themselves or
their men, honourably maintaining and supporting the Regiment
themselves:—Unhappy Gentlemen! They merited a much better fate
than what was awaiting them![410]
Every thing being now in readiness, we began our march, in
order (alas! as it happened) to bid adieu to Old England for ever! On
the 22nd of December 1745[411] (being the Prince’s birthday) about
four o’Clock in the afternoon, we crossed the river Esk, which
separates the two kingdoms. The deepness and rapidity of the river,
joined to the obscurity of the night, made it most terrible: but the
good Prince, here, in particular, animated the men; and how noble
was it to see these Champions, who had refused him nothing now
marching breast-deep, one supporting another, till wonderfully we all
passed safe. The Duke of Perth here signalized himself much by his
goodness; for, crossing the river several times on horseback, he took
behind him several of the common Soldiers, whose strength was not
sufficient to bear up against the current. Nor was the Prince wanting
in giving a notable specimen of his generosity and condescension to
his subjects, in imitation of the Great Alexander, who, in his
expedition into Persia, suffered a poor Soldier, much fatigued, to
repose himself in the king’s chair, which till then it had been death for
anyone but himself to sit in. So the Prince I think, imitates, or fully
equals, this great hero in point of affability to his men; for, taking on
horseback a common soldier behind him, he carried him over the
water, giving us all a great example of goodness and courage to
follow him. But at this river I narrowly escaped drowning; for in
crossing it, and being near the middle of the stream I perceived two
women (tho’ never an army was known with so few) rolling down it
and in imminent danger of perishing if I did not guide my horse in
order to stop them: and I had like to have paid dearly for it, for
coming against me, they laid hold of me in such a manner, that I was
not able to dismount; and being so beset by the two women, and my
horse, who was none of the biggest, going down the stream with me,
I gave myself up for lost. I did not, however, lose courage and
conduct in managing my horse as well as I could; and perceiving a
man mounted upon a very high horse wading where I was
swimming, I called to him for assistance. He seeing me in that
piteous case, came immediately, and rescued me from the imminent
danger I was in of perishing, and freed me also with a great deal of
difficulty from my two companions: but how I got quit of them or out
of the water, I remained an entire stranger; being come to the other
side, I was immediately seized with a great panic and trembling from
the fright of the danger I had so narrowly escaped, and in this state I
continued for three or four hours notwithstanding all the efforts I
could make against it. That night however, being, as I said, on the
other side, somewhat before the rest, I saw them come almost half
round together with my deliverer, to whom I did not fail to pay my
most hearty thanks, as the poor women did to me.
But here let us stop and take a short View of the Army’s
behaviour whilst in England. It may be said, and is allowed even by
the judgment of our enemies, that never such an army could be
expected to behave as they did, giving the greatest marks of
generosity to our enemies, paying for what they had, and revenging
or oppressing few or none; and shewing to the World, that a noble
design rather than either malice, rapine or plundering, was at the
bottom of our proceedings. But as I may perhaps be suspected of
partiality towards them, I shall refer the Reader to what was
acknowledged by one of our greatest enemies, viz., one Parson
Bissett in Aberdeen,[412] who was one of the heads of the
Presbytery, and much esteemed by his own party for his morals and
great talents as a preacher. This Gentleman, mounting the pulpit,
soon after we had left Aberdeen, and while Cumberland’s Army was
there, began his Sermon with a comparison betwixt us and those
then in town, saying: ‘When the Prince, as they call him, was here,
[413] I made it my business night and day to inspect their conduct,
and observe their most minute actions; and instead of finding
subjects of complaint, I found much to the contrary, and drew honey
whence I thought it was not to be extracted. I heard what they said
and did; and I heard of no robbery, of no ill towards the people: but
since these men are come amongst us, what is the secret that is not
revealed? What is holy, that is not polluted? What’s forbid, that’s not
transgressed? and in fine, where is God or Man obeyed? Here,
cursing and blaspheming; there Drunkenness, Whoredom and
Debauchery are carried on in full career, and reign with unlimited
sway.’ So, going on in this strain, he concluded by telling his
auditors, that it was a shame to be out-done by us; that, in
punishment for their sins, nothing less than the curse of God could
fall upon them, and make instruments of us to punish them.
But now I shall return again to the river Esk. Having wonderfully
got safe over, we marched all that night through excessively bad
roads, all of us being very wet and cold, without any refreshment,
except what we had before we left Carlisle, till ten o’Clock the next
day, when we arrived at Dumfries; and extraordinary it was to see
the Army, notwithstanding all their fatigue, come in as merry and gay
as if they had only marched that morning. At our entrance into the
Town, we saw the great rejoicing that had been made for our defeat,
the candles being still in the windows, and the bonfires
unextinguished. And now it was, that being in Scotland, my Colonel
began to shew me great civility upon account of my being an
Englishman, and so did afterwards several of the gentlemen of that
country; for seeing me amongst them, they thought they could
scarce do enough for me, especially my Colonel, who, for fear of my
being any time badly quartered, ordered that I should always be
lodged chez lui. So that from that time I had ordinarily his quarters,
and sometimes a part of his bed, when there happened to be no
other, so that I had every reason to be satisfied with my lot, having
such resource of comfort in all my difficulties.
After we had halted two days at Dumfries, and made them pay
for their past behaviour,[414] we marched on for Hamilton; and here
the Prince, attended by a few of his gentlemen, went to take the
diversion of shooting in the Park; in which he behaved to the
admiration and surprise of all present, killing or hitting every thing he
shot at, so that, without flattery, he was looked upon to be the best
marksman in the army. After we had staid some time at Hamilton, we
continued our march to one of the prittiest (but most whiggish) towns
in all Scotland, viz., Glasgow. Here we arrived on the 25th of
December, 1745, much to their confusion, and halted six or seven
days.[415] That Town had given, when the Prince marched for
England, five thousand pounds for its good behaviour, and paid us
now as much over again for breaking the same, rebelling against us,
and raising the Militia in our absence. So we taught them more wit,
how to break their words another time. The Army having been here
provided with cloathing and other necessaries, of which they were
very much in want, the Prince resolved to make a general inspection
and review of them. Accordingly orders were issued one morning for
that purpose, for us all to repair to a place at a little distance from the
Town. So we marched out with drums beating, colours flying, bag-
pipes playing, and all the marks of a triumphant army to the
appointed ground, attended by multitudes of people, who had come
from all parts to see us, and especially the ladies, who before were
much against us, were now, charmed at the sight of the Prince,
become most loyal;[416] and many afterwards, when they could not
testify it to us by their good offices, did it in imitation in their hearts. I
am somewhat at a loss to give a description of the Prince at this
Review. No object could be more charming, no personage more
taking, no deportment more agreeable, than his at that time was, for,
being well mounted and princely attired,[417] having, too, great
endowments both of body and mind, he appeared to bear a sway
above any comparison with the heroes of the last ages, and the
majesty and grandeur he seemed to display most noble and divine.
The Army being now drawn up in all form, and every one putting
himself out for the best, the Prince rode through the ranks, greatly
encouraging and delighting all who saw him. After the Review we
returned again to Glasgow; and about this time the unhappy news
reached us, that Carlisle was invested by Cumberland, who, having
got up cannon from Whitehaven, was in a fair way of taking it.
It being now judged proper to continue our march for Stirling, we
quitted Glasgow in a handsome manner, and soon sat down before
that town, which we took after two or three days’ cannonading, and
then began the siege of the Castle, having just received a few pieces
of Cannon from France, which would have been sufficient, as many
are pleased to say, if our Engineer, viz., one Mirabell,[418] a
Frenchman by birth, had been good for any thing: but erecting our
batteries in an improper place against the Castle, we spent three
weeks’ labour in vain; during which time news arrived from Carlisle,
by Mr. Brown,[419] who was left Lieutenant Colonel by the Prince,
and one Mr. Maxwell,[420] who made their escape over the wall while
the Articles of Capitulation were signing. These gentlemen
acquainted us, that, after Cumberland had lain six or seven weeks
before the town, and heralds had been frequently sent in to
summons it to surrender, Mr. Hamilton, Governor of the Castle had
at last resolved to obey them but whether with a true fear or promise
of his life, is disputed. Certain however it is, that he employed that
villainous Wier, whom I have mentioned before (being left a prisoner
at Carlisle) with secret Messages to and from the enemy; and
instead of hanging him, invited him daily to his own table. How far
this conduct was good, I leave the world to judge.
Mr. Townley, Governor of the Town, being informed that the
resolution to capitulate was taken, endeavoured, seconded by his
whole corps, officers and soldiers, to oppose him: but finding no
means effective to hinder the place from being given up, he was
obliged to send, by Hamilton’s direction, articles of Capitulation to
Cumberland, who returned for answer That the Town and Castle
should be surrendered at discretion, and that the Officers and
Soldiers should be at his Father’s Mercy, with whom he promised to
intercede for their safety. How sincerely he behaved in this, is
sufficiently known:[421] and when I reflect upon this, I think I may say
I have good fortune, in leaving that unhappy town and regiment, that
I was not on some gallows or other made a partaker also of his
clemency! This news was at first hard to be believed amongst us: but
it appearing but too true, the Prince was exceedingly troubled, and
lamented much the loss of his subjects, especially the English, who
were to be made sacrifices of; and also did generally the whole
army, many of them wishing they had been there, nay even to be
made victims of to Cumberland’s fury!
We had now scarce got well into the siege of the Castle, before
news came, on the other hand, that General Hawley was advancing
towards us with about eleven or twelve thousand men. The Prince’s
Army at this time amounted to about eight thousand effective men,
having been joined by Lord John Drummond’s Regiment, and Lord
Lewis Gordon’s, of whom I have spoken before, and some few
others. So, seeing we must inevitably fight we endeavoured to
prepare ourselves in the best manner for that purpose. The enemy,
we heard, were now come to Falkirk, which was only eight or ten
miles distant from us. We waited two days in expectation of their
coming on to attack us: but finding that they continued at Falkirk, we
on the third morning, leaving a sufficient force for the siege of
Stirling, boldly marched out in quest of them. And here it was the
soldiers shewed the greatest alacrity; the foot marching with such
celerity as kept the horse on a full trot, so that by two o’Clock in the
afternoon we came up with them, notwithstanding we almost
marched round them, in order to have the wind somewhat favourable
for us. And now the day, from being an exceeding fine one, became
on a sudden obscure; the sun which till then shone upon us, was
now as it were eclipsed, and all the elements in confusion, so that
the heavens seemed to fulminate their anger down upon us, by the
impetuous storm of hail, wind and rain, that fell just at the time of the
engagement. The enemy at this time scarcely knew any thing of our
march towards them till it had been almost too late; as they lay in an
entire security and defiance of us, thinking it not worth their while to
take the necessary precaution of having spies out, as other prudent
Generals would have done, notwithstanding the contempt they might
have had for us. We now roused them out of their lethargy, being just
upon them at their going to dinner. Cursing their bad fortune, they
immediately mounted and turned out of their camp, somewhat in
confusion to meet us, which they did at a short distance from it,
though not to our disadvantage.
Their cavalry was in front of their left wing, unsupported with foot;
and their infantry in the right, unsupported by horse: but the day
being so excessively bad, hindered their cannon from coming up, so
that we were upon an equal footing with them in that respect, we
bringing none with us. We were about four hundred light Horse
ordered to face the enemy’s dragoons, being fronted and supported
by a strong line of McDonells; and our foot, with the Prince, against
their foot. Here I must acknowledge, that when I saw this moving
cloud of horse, regularly disciplined, in full trott upon us down the
summit, I doubted not but that they would have ridden over us
without opposition (I mean the front line) and bear us down without
difficulty in their impetuous progress: but I soon found myself
mistaken; for immediately upon our bearing upon them in order to
meet them, there blew such a storm of wind and hail, which was
before on our side, and now turned miraculously, as we turned, on
our backs; and notwithstanding that almost disabled us to bear up
against them, it so harrassed the enemy, that cursing and
blaspheming was made the dying-speech of many of them. And now
kind Heaven seemed to declare for us.
The brave front-line of McDonells suffered the enemy to come
within ten or twelve paces of them before firing. Nobly altogether
presented, and sent their benediction upon them, so that in the third
part of a minute that rapid and impetuous torrent, which seemed in
rolling to lay all waste before it, was now checked and stemmed in
such a manner, that it was made to retake its course faster than it
had proceeded. Upon seeing this, we immediately seconded our
work with a hearty huzza, victory now declaring for us. But on the
Prince’s wing it remained somewhat more obstinate: yet soon after
they returned the same, having happily finished their affair with
scarce the loss of forty men, and we had only one wounded. We now
pursued them sharply for about sixty paces, and fetched down a
good many of them: but unfortunately being ordered to halt, when a
little further pursued would have finished our affair, we let an
opportunity slip out of our hands, which never afterwards presented
itself again. This piece of conduct belonged chiefly to Lord George
Murray,[422] who would not permit the army to pursue any farther. So
being ordered to halt, we modestly bid our enemies retire, and as it
were tell them to come better provided another time. Here I could
say something more: but I cannot at present; therefore I’ll say
nothing, leaving it to Midwife Time to say it for me, and contenting
myself with halting with the army.
And now after four or five hours’ halt, we heard, that we might go,
if we pleased, and take possession of the town. So the army was
ordered immediately to enter the town, and about eighteen of us
Guards, who were present, to go about three miles off, and take fifty-
four of the enemy prisoners. When we arrived at the house, where
they had got together they shut the door against us. We therefore
surrounded the house, and summoned them immediately to
surrender:—which they refusing at first to do, we fired into the house
—and they immediately gave themselves up to our mercy. So, after
taking them and fifty-four stand of arms, we conducted them
Prisoners into Falkirk that night. And here it was, that I happened to
perform an action which gave me great comfort afterward in my
distress.—A fine young boy, who was somewhat out of order, being
found in the house after all was almost done, was espied by some of
our party, who bringing him out asked who would take care of him. I
told them to give him to me, which they immediately did. The young
boy, being now in my possession, says to me: ‘I hope Your Honour
will not kill me.’ Upon which, being a little surprised at what he said, I
asked him, ‘Have you not well merited it?’ ‘If I may be permitted to