Professional Documents
Culture Documents
threats aimed at the critical systems and processes communi- Monitoring Protection Control Optimization
k,(((
,(((,QWHUQDWLRQDO&RQIHUHQFHRQ&RPPXQLFDWLRQV&RQWURODQG&RPSXWLQJ7HFKQRORJLHVIRU6PDUW*ULGV6PDUW*ULG&RPP
vehicle integration, while new variants and/or attacks have National Laboratory has conducted a comprehensive evalua-
emerged targeting a zero-day vulnerability. The trained model tion of several classic machine learning approaches in binary
can perform poorly in the new environment against the new classification tasks [17], which established a benchmark for
threats [7], while in many real-world scenarios it is expensive, different machine learning approaches. Subsequent studies
if not prohibitive, to recollect new datasets and re-train a based on neural networks and ensemble learning have reached
deployed model. over 95% overall accuracy [18], [19]. However, these existing
In machine learning research, transfer learning has been approaches took the default assumption that training and
widely adopted in various image/video applications [8], [9] testing datasets are from the same distribution and the same
as a promising solution to transfer a learned model into new attacks have appeared in both datasets. While traditional
domains or tasks. By learning a new feature space where machine learning can suffer degrading accuracy when the
inner-class similarity and inter-class distance are preserved, assumption fails to hold, the proposed framework can al-
TL methods allow machine learning-based classifiers to retain low trained classifiers to retain robust performance against
their performance on incoming data with new distributions unknown threats.
and/or classes. However, such capacity has not been extended
B. Transfer Learning for Knowledge Adaption
or validated in complex cyber-physical systems like the smart
grid, where the common practice is still to manually select Transfer learning (TL), also known as inductive learning or
and test different feature subsets. domain adaptation, aims at applying knowledge or patterns
Motivated by these gaps, this paper proposes a domain- learned in a certain domain (or task) to a different but related
adversarial framework leveraging based on latest deep ad- domain (or task) [20]. The “transfer” is similar to the process
versarial learning technique [10] to extract novel features where a graduate tries to apply what he/she learns in class
and transfer different machine learning classifiers for new to a practical problem, and the key is to find a mapping
operation scenarios and attack threats. A realistic smart grid between the problems so that the learned knowledge can be
security dataset, collected over hardware-in-the-loop simula- adapted to the practical situation and resolve the problem.
tors [11] with multiple attack and normal scenarios as well Over the last two years, efforts have started to introduce
as cyber and physical system information, has been chosen transfer learning for anomaly detection in Internet [21] and
to evaluate the performance of the proposed framework. The cloud [22] applications. Juan et al. proposed a feature-
dataset allows the study to leverage deep packet inspection based transfer learning framework that was able to boost
(DPI) of the payloads (measurements), system logs, and classifier performance on the well-known NSL-KDD dataset
snort alerts, instead of packet headers or traffic statistics, of TCP traffic [23], demonstrating the potential merit for
to utilize the cyber-physical information against unknown cyber-security analysis. Meanwhile, inspired by the highly-
threats. Similar DPI method has also been adopted in [12], successful generative adversarial networks (GANs) [24],
[13] to extract voltages and currents for detecting false data Ganin et al. proposed the domain-adversarial training based
injection and fault localization. Ablation analysis has been on neural networks (DANN) [10] as an unsupervised domain
performed to provide insights on the choice and influence adaption technique for image applications. Seen the potential
of critical hyper-parameters. The results have shown that the and the gap of TL methods leveraging cyber-physical data
domain-adversarial TL framework can significantly improve for smart grid communication and security, the proposed
classification performance against unknown threats of differ- framework introduced the domain-adversarial TL as the core
ent types and locations. Discussions on hyper-parameters are feature extractor for the detection of unknown threats.
also provided for practical uses. III. D OMAIN -A DVERSARIAL T RANSFER L EARNING
The rest of this paper is organized as follows. Section II F RAMEWORK AGAINST U NKNOWN T HREATS
reviews the related work. Section III presents the domain- The paper proposes a framework that leverages domain-
adversarial transfer learning framework for intrusion detec- adversarial transfer learning, one of the latest domain adap-
tion. Section IV presents the experimental setup, the simula- tion techniques, to extend existing knowledge against un-
tion results of unknown threat types and unseen threat loca- known threats in the smart grid. The overall architecture,
tions, and the discussions on the choice of hyper-parameters. illustrated in Fig. 2, consists of four steps: (1) collect labeled
Section V draws the conclusions and future works. source data and unlabeled target data; (2) train the model
over both domains to obtain a new feature space in a domain-
II. R ELATED W ORKS adversarial manner; (3) use the new feature representations of
A. Machine Learning for Smart Grid Security the labeled data in source domain to train classifiers; (4) map
the new data to the new feature space and predict their class
Following the increasing availability of data and computing labels. The detailed problem formulation and methodology
power, machine learning (ML) has been extensively investi- are as follows.
gated as an intrusion detection and classification technique
in communication/network security [7] as well as smart A. Problem Formulation
grid security [14], [15]. Among them, adversarial machine In transfer learning, a domain D consists of two compo-
learning has been introduced in applications like synthesizing nents: a feature space X and a marginal probability distri-
normal data for load forecasting [16]. In 2014, Oak Ridge bution P(X). Given a specific domain D= {X ,P(X)}, a task
,(((,QWHUQDWLRQDO&RQIHUHQFHRQ&RPPXQLFDWLRQV&RQWURODQG&RPSXWLQJ7HFKQRORJLHVIRU6PDUW*ULGV6PDUW*ULG&RPP
Label Predictor
Feature Extractor
Label prediction loss
Source Domain Tuning
Labeled Training Data
(known threats) Class
Label Projected
Model Training
Labeled
Source Data
Domain Classifier
Feature Domain
Target Domain
Label Selected
Unlabeled Training Data Tuning
Classifier
(unknown
Domian Adaption Loss
threats/variants
Step 2: Transfer Learning Model (DANN) Training Step 3: Classifier Traning
Target Domain
Unlabeled Testing Data Trained Predict Model Testing
(unknown Classifier Label
threats/variants
Fig. 2. The proposed domain-adversarial transfer learning framework for smart grid intrusion detection.
consists of two components: a label space Y and an objective will be back-propagated to adjust the weights of the feature
predictive function f (·) to be learned from the training data. extractor.
We model the attack detection of smart grid as a bi- Similarly, the label predictor aims to learn a function Gy
nary classification problem, that is, to classify the events from the extracted features to the class labels:
as an attack or a natural event. The source domain
DS and target domain DT have the same feature space Gy (x) = sof tmax(VGf (x) + c) (2)
X but different combination of attack types and nor- where sof tmax(a) = [ |a|
e ai |a|
] , (V, c) ∈ RL×D × RL
mal events. The marginal probability distribution P(X) is j=1 eaj i=1
thus different. We denote the labeled source domain as is a matrix-vector pair of the weights V and the bias c of the
DS = {(xS1 , yS1 ), ..., (xSnS , ySnS )} and the unlabeled label predictor.
target-domain as DT = {(xT1 , yT1 ), ..., (xSnT , ySnT )}. The Given a sample-label pair (x, y), the loss function of the
goal is to learn a predictive function f (·) from the training binary classification problem is given as:
data that predicts labels in the target domain. Ly (x, y) = −y log[Gy (x)] − (1 − y) log[1 − Gy (x)] (3)
The following problems will be considered in this paper:
• One threat appears only in DS and another new threat For the source domain, the domain-adversarial training can
appears only in DT ; thus be formulated as the following optimization problem:
• The same type of threats appears in both DS and DT 1
but each domain contains attacks at different locations. min Ly (x, y) + λ · R(W, b) (4)
W,b,V,c nS
x∈DS
B. The Original Domain-Adversarial Training Design where R(W, b) is a regularizer weighted by a hyper-
The DANN architecture consists of three neural networks: parameter λ. The regularizer determines the penalty for
a feature extractor, a domain classifier, and a label predictor. the feature extractor, which provides the mapping from the
Note that each network can be implemented as a single or source/ target domains to a new feature space where the
multi-layer perceptron, where the simplest structure was used domain adaption loss, or the dissimilarity of samples mapped
in this paper, as by Ganin et al. in [10]. from the source and target domains, are minimized.
The feature extractor learns a function Gf : X → RD that The domain classifier learns a logistic regression GD →
maps an m-dimensional input into a D-dimensional feature: [0, 1] that predicts whether an input x is from DS or DT :
where dx = 0 if x ∈ DS and dx = 1 if x ∈ DT .
Subsequently, the regularizer R(W, b) can be defined as:
1 1
R(W, b) = − Ld (x, dx )− Ld (x, dx ) (7)
nS nT
x∈DS x∈DT
TABLE III
C OMPARISON OF ORIGINAL AND DOMAIN - ADVERSARIAL MACHINE LEARNING CLASSIFIERS AGAINST UNKNOWN THREAT VARIANTS
Cases Methods AdB kNN SVM RF CART ANN
Original 72.0% 77.6% 57.9% 51.4% 53.2% 83.0%
1 Domain-Adversarial 86.8% 86.0% 82.9% 88.2% 76.3% 84.5%
Improvement +14.8% +8.5% +25.0% +36.8% +23.1% +1.5%
Original 77.3% 82.7% 71.0% 84.5% 76.7% 86.5%
2 Domain-Adversarial 94.2% 90.4% 85.5% 95.2% 79.2% 87.8%
Improvement +16.9% +7.8% +14.5% +10.7% +2.5% +1.3%
Original 73.0% 75.1% 64.8% 76.0% 61.3% 81.7%
3 Domain-Adversarial 83.6% 82.2% 80.9% 84.5% 75.7% 83.6%
Improvement +10.6% +7.1% +16.1% +8.5% +14.4% +1.9%
Original 71.2% 80.5% 66.7% 83.0% 69.5% 85.9%
4 Domain-Adversarial 89.3% 88.2% 85.3% 90.0% 79.6% 87.6%
Improvement +18.1% +7.7% +18.6% +7.0% +10.1% +1.6%
with increasing customer participation via smart meters, the • λ: from 0.01 to 2.00 at the 1st, 2nd, and 5th decile (0.01,
adaptability provided by the TL framework can allow the 0.02, 0.05, 0.1, ...).
learned models to handle known threats occurred on new The results are shown in Fig. 5, with two x-axes above
devices and locations in the network, improving the resiliency and below the figure. The accuracy increases as Nf increases
,(((,QWHUQDWLRQDO&RQIHUHQFHRQ&RPPXQLFDWLRQV&RQWURODQG&RPSXWLQJ7HFKQRORJLHVIRU6PDUW*ULGV6PDUW*ULG&RPP