Endpoint Defense As Code (Edac) Configurable

Dakota State University
Beadle Scholar
Masters Theses & Doctoral Dissertations
2-2023
ENDPOINT DEFENSE AS CODE (EDAC): CONFIGURABLE

CONTEXTUAL ANALYSIS OF PROCESS BEHAVIORS FROM
KERNEL/USER EVENT TRACING
Trevor M. Lewis
Follow this and additional works at: https://scholar.dsu.edu/theses

ENDPOINT DEFENSE AS CODE (EDAC):
CONFIGURABLE CONTEXTUAL ANALYSIS OF
PROCESS BEHAVIORS FROM KERNEL/USER
EVENT TRACING
A dissertation submitted to Dakota State University in partial fulfillment of the

requirements for the degree of
Doctor of Philosophy
in
Cyber Operations
February 7, 2023
By
Trevor M. Lewis
Dissertation Committee:
Bhaskar Rimal, Ph.D.
Cody Welu, Ph.D.
Yong Wang, Ph.D.
Beacom College of Computer and Cyber Sciences
DISSERTATION APPROVAL FORM
This dissertation is approved as a credible and independent investigation by a candidate for the Doctor of
Philosophy degree and is acceptable for meeting the dissertation requirements for this degree. Acceptance of
this dissertation does not imply that the conclusions reached by the candidate are necessarily the conclusions
of the major department or university.
Student Name:
Dissertation Title: _______________________________________
_________________________________________________________________________________
Dissertation Chair/Co-Chair: Date: _____

Name:
Dissertation Chair/Co-Chair: Date: _____

Name:
Committee member: ______ Date: _____

Name:

Name:

Name:

Name:
Original to Office of Graduate Studies and Research

Acid-free copies with written reports to library
ACKNOWLEDGMENTS
The first person I would like to thank is my wife Emily. The amount of work that
has gone into this degree program and dissertation would not have been possible had
it not been for her continued support and encouragement. She has been with me since
the beginning and through all of my academic endeavors and I could not have asked
for a better person to be with throughout this process. Second, I would like to thank
Matt Guinn and Ben Medlin of the Georgia Tech Research Institute (GTRI) for their
continued support of my vision for the future of cyber security and endpoint defense and
all of the research that has been conducted in support of that vision. Third, I would like to
thank the Georgia Institute of Technology (GT) and the Georgia Tech Research Institute
(GTRI) for providing funding to support this research. Fourth, I would like to thank
my committee members, Josh Stroschein, D.Sc. (former), Bhaskar Rimal, Ph.D., Cody
Welu, Ph.D., and Yong Wang, Ph.D. for their support in my vision and their guidance
throughout this dissertation research. Fifth, I would like to thank my family and my
wife’s family for their support and encouragement throughout my various degree programs
over the past 15 years. Finally, I would like to specifically thank my mother, Johnnie
Lewis, Author/Illustrator, for providing her professional editing expertise in reviewing
this dissertation.
iii
ABSTRACT
The current industry standard to detect cyber threat activity on endpoints (workstations,
servers, etc.) centers around the use of endpoint defense software. The software products
marketed are Endpoint Protection Platforms (EPP), Endpoint Detection and Response
(EDR), and eXtended Detection and Response (XDR) solutions. These solutions are
typically deployed onto endpoints across enterprises and monitor various aspects of each
operating system for malicious activity. Current generations of these three solutions
have similar underlying software architectures, user workflows, and detection capabilities.
These solutions also have a number of issues that inadvertently allow advanced cyber
threat actors to succeed in their operations, such as, lack of resilience to intentional eva-
sions against critical software components, lack of resilience against user configuration
errors, low detection rates of atomic techniques, low configurability for process-level be-
haviors, and semantically inappropriate alert messages. As proven in prior research and
research that the author is conducting concurrently alongside this research, these issues
can be capitalized on by knowledgeable and observant attackers to enable their technique
chains to succeed undetected.
Through years of professional experience deploying, testing, and evaluating various
commercial endpoint solutions in various system architectures (commercial enterprise sys-
tems, government systems, disconnected/air-gapped systems, etc.), the author has learned
that many commercial endpoint defense technologies are designed to make decisions for
the operators on what activity is benign and what activity is malicious, without giving
operators the ability to change this decision making. Vendors of these solutions add to
this by illustrating a measure of trust in the solution’s efficacy by releasing their detection
statistics of known Indicators of Compromise (IOCs). These IOCs may or may not be
used by attackers in the future as new attack techniques are developed. This creates a
iv
detection gap between known techniques that can be detected, and actual techniques that
are being executed. In addition to this, the author has observed in organizations across
many industries a level of indiscriminate trust in commercial endpoint solutions. Many
organizations fully trust endpoint solutions to be the sole defense mechanism on an end-
point without fully testing the solution for resiliency or detection gaps. All of these facts
and circumstances create gaps, inconsistencies, and avenues for highly observant cyber
attackers to maneuver in and out of systems undetected.
This document illustrates all of the research that has been completed as part of this
dissertation to solve the identified issues with current-generation endpoint defense so-
lutions. The overarching approach to solving the identified problems was to use the
Design Science Research (DSR) methodology to develop a software artifact that is suffi-
ciently di↵erent and more impactful than existing solutions, and test the designed artifact
against real-world attack technique stimulus to prove its validity and usefulness within
real-world system architectures. The developed artifact gives operators the flexibility to
define attack technique behaviors of interest through a custom developed configuration
syntax and utilizes Event Tracing for Windows (ETW) telemetry emanating from the
Windows operating system in a unique way to detect the defined attack behaviors. Val-
idation experiments on the developed artifact proved that the artifact, along with the
user-defined configuration file, successfully detected 36/48 of the chosen atomic attack
technique stimuli. The results represent a significantly broad coverage of detection that
current-generation endpoint solutions fail to accomplish, thereby illustrating the need to
incorporate the developed artifact into real-world environments to combat cyber-attack
activity.
v
DECLARATION
I hereby certify that this dissertation constitutes my own product, that where the language
of others is set forth, quotation marks so indicate, and that appropriate credit is given
where I have used the language, ideas, expressions or writings of another. I declare that
the dissertation describes original work that has not previously been presented for the
award of any other degree of any institution.
Signed,
T˚rè›vô˘rffl M. Lè›w˘i¯s
Trevor M. Lewis
vi
TABLE OF CONTENTS
Dissertation Approval Form ii
Acknowledgments iii
Abstract iv
Declaration vi
Table of Contents vii
List of Tables xi
List of Figures xiii
Chapter 1:
Introduction 1
1.1 Background of the Problem . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.1.1 Endpoint Defense Software Architecture . . . . . . . . . . . . . . . 5
1.1.2 User-Mode Hook Dependency Problem . . . . . . . . . . . . . . . . 7
1.1.3 Endpoint Solution Resiliency Problem . . . . . . . . . . . . . . . . 8
1.1.4 Technique/Signature Detection Problem . . . . . . . . . . . . . . . 9
1.1.5 Alert Semantics/Endpoint Configurability Problem . . . . . . . . . 11
1.1.6 Indiscriminate Trust Problem . . . . . . . . . . . . . . . . . . . . . 13
1.2 Special Considerations for Air-Gapped, Disconnected, & DDIL Systems . . 13
vii
1.3 Objectives & Impact of Research . . . . . . . . . . . . . . . . . . . . . . . 15
Chapter 2:
Literature Review/Related Work 17
2.1 Attack Technique Detection Research . . . . . . . . . . . . . . . . . . . . . 18

2.1.1 Technique Detection Using Sysmon . . . . . . . . . . . . . . . . . . 18
2.2 Process Behavior Monitoring Research . . . . . . . . . . . . . . . . . . . . 19
2.3 Process Behavior Classification Research . . . . . . . . . . . . . . . . . . . 22
2.4 Endpoint Defense Solution Research . . . . . . . . . . . . . . . . . . . . . 25
2.4.1 EDR User-Mode Hook Research . . . . . . . . . . . . . . . . . . . . 26
2.4.2 Endpoint Solution Resiliency Research . . . . . . . . . . . . . . . . 28
2.4.3 Endpoint Solution Research . . . . . . . . . . . . . . . . . . . . . . 29
Chapter 3:
Research Methodology & Proposed Solution 31
3.1 Overall Research Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.1.1 Chosen Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.1.2 Hypothesis and Research Questions . . . . . . . . . . . . . . . . . . 32
3.1.3 Background/Supporting Research Phase . . . . . . . . . . . . . . . 33
3.1.4 Artifact Development Phase . . . . . . . . . . . . . . . . . . . . . . 34
3.1.5 Artifact Validation Phase . . . . . . . . . . . . . . . . . . . . . . . 34
3.2 Background & Supporting Research Phase . . . . . . . . . . . . . . . . . . 35
3.2.1 Event Tracing for Windows (ETW) Introduction . . . . . . . . . . 35
3.2.2 ETW Event Data Completeness/Attack Technique Coverage Problem 38
3.2.3 Automated ETW Event Generation . . . . . . . . . . . . . . . . . . 39
3.2.4 Automated Attack Technique Generation/Execution . . . . . . . . 41
3.2.5 Iterative Analysis of ETW Event Data Against Attack Techniques . 54
3.3 Artifact Design & Development Phase . . . . . . . . . . . . . . . . . . . . 100
viii
3.3.1 Artifact Research Goals/Requirements . . . . . . . . . . . . . . . . 100
3.3.2 Artifact Development Goals/Requirements & Initial Design . . . . . 101
3.3.3 Final Artifact Design, Class Structure, & User Workflows . . . . . . 118
3.3.4 Artifact Performance Analysis . . . . . . . . . . . . . . . . . . . . . 149
Chapter 4:
Validation & Results 159
4.1 Validation Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

4.2 Validation Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
4.3 Validation Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
4.3.1 Representative System Configurations . . . . . . . . . . . . . . . . . 163
4.3.2 Validation Experiments . . . . . . . . . . . . . . . . . . . . . . . . . 168
4.4 Validation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
4.4.1 Initial Access Category Results . . . . . . . . . . . . . . . . . . . . 171
4.4.2 Execution Category Results . . . . . . . . . . . . . . . . . . . . . . 176
4.4.3 Persistence Category Results . . . . . . . . . . . . . . . . . . . . . . 182
4.4.4 Credential Access Category Results . . . . . . . . . . . . . . . . . . 188
4.4.5 Lateral Movement Category Results . . . . . . . . . . . . . . . . . . 193
4.4.6 C2 Category Results . . . . . . . . . . . . . . . . . . . . . . . . . . 198
4.4.7 Final Experiment Results Discussion . . . . . . . . . . . . . . . . . 202
Chapter 5:
Conclusions 205
5.1 Research Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205

5.2 Answering the Research Questions . . . . . . . . . . . . . . . . . . . . . . . 207
5.3 Conclusions on the Hypothesis . . . . . . . . . . . . . . . . . . . . . . . . . 209
5.4 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
5.5 Limitations & Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . 213
ix
5.5.1 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
5.5.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
References 217
x
LIST OF TABLES
Table 3.1 Attack Techniques Chosen for Validation in Automation Script 45

Table 3.2 Attack Technique Automation Details . . . . . . . . . . . . . . 46
Table 3.3 Experiment 1 - Software Exploit Atomic Techniques . . . . . 57
Table 3.4 Experiment 2 - Macro-Enabled Word Document Atomic Tech-
niques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
Table 3.5 Experiment 3 - PowerShell Command Atomic Techniques . . 66
Table 3.6 Experiment 4 - Simulated Malware Executable Atomic Tech-
niques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
Table 3.7 Experiment 5 - Persistent Windows Service Atomic Tech-
niques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
Table 3.8 Experiment 6 - Persistent Registry Run Key Atomic Tech-
niques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
Table 3.9 Experiment 8 - Credential Access (LSASS) Atomic Techniques 82
Table 3.10 Microsoft-Windows-Kernel-Audit-Api-Calls ETW Provider
- Audited Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
Table 3.11 Experiment 10 - SMB Lateral Movement Atomic Techniques 90
Table 3.12 Experiment 11 - Command and Control (C2) HTTP Atomic
Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
Table 3.13 ETW Event Coverage - All Validation Experiments/Tech-
niques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
Table 3.14 Experiment 10 - Artifact Performance Analysis - UDC Parse 151
xi
Table 3.15 Experiment 10 - Artifact Performance Analysis - Class Gen-
eration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
Table 3.16 Experiment 10 - Artifact Performance Analysis - ETW Con-
sumption/Behavior Check . . . . . . . . . . . . . . . . . . . . . . . . 154
Table 4.1 Windows 10 Standard Endpoint - System Parameters . . . . 165

Table 4.2 Windows 10 Edge Compute (DDIL) Endpoint - System Pa-
rameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
Table 4.3 Windows 10 Air-Gapped Endpoint - No Internet Connectivity168
Table 4.4 Experiment 1 - Software Exploit Experiment Results . . . . . 173
Table 4.5 Experiment 2 - Macro Word Document Experiment Results 175
Table 4.6 Experiment 3 - PowerShell Command Experiment Results . 179
Table 4.7 Experiment 4 - Simulated Malware Binary Experiment Results182
Table 4.8 Experiment 5 - Service Persistence Experiment Results . . . 184
Table 4.9 Experiment 6 - Registry Persistence Experiment Results . . 187
Table 4.10 Experiment 8 - Credential Dump (LSASS) Experiment Results190
Table 4.11 Experiment 9 - Credential Dump (SAM) Experiment Results192
Table 4.12 Experiment 10 - Lateral Movement (SMB) Experiment Re-
sults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
Table 4.13 Experiment 11 - C2 HTTP/HTTPS Experiment Results . . 201
Table 4.14 Techniques Not Detected During Experiments . . . . . . . . . 203
xii
LIST OF FIGURES
Figure 1.1 Modern EDR Hooking/Detection Architecture . . . . . . . . . . . . 7

Figure 1.2 FireEye Endpoint Alert on PowerShell Mimikatz Variant . . . . . . 11
Figure 2.1 SwiftOnSecurity Sysmon Configuration File [20] . . . . . . . . . . . 21

Figure 2.2 Atomic Attack Technique Telemetry Mapping Steps 1-3 [14] . . . . . 24
Figure 2.3 Atomic Attack Technique Telemetry Mapping Steps 2-3 [14] . . . . . 25
Figure 2.4 Debugger Illustrating an EDR User-Mode hook . . . . . . . . . . . . 26
Figure 2.5 Identified Endpoint Solution Problem Categories and Failure Paths . 29
Figure 3.1 Windows 10 21H1 ETW Providers . . . . . . . . . . . . . . . . . . . 36

Figure 3.2 Windows Event Tracing Model [24] . . . . . . . . . . . . . . . . . . 37
Figure 3.3 SilkETW Capturing Events with Evidence of Mimikatz Behaviors [26] 38
Figure 3.4 Automated ETW Event Generation Script Execution Flow . . . . . 41
Figure 3.5 MITRE ATT&CK Framework Matrix . . . . . . . . . . . . . . . . . 43
Figure 3.6 Automated Attack Technique Execution Flow . . . . . . . . . . . . . 48
Figure 3.7 Automated Attack Technique Execution Script - Technique File/-
Folder Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
Figure 3.8 Automated Attack Technique Execution Script - Dynamic Payload
Generation Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
Figure 3.9 Automated Attack Technique Execution Script - Setup UI . . . . . . 52
Figure 3.10 Automated Attack Technique Execution Script - Setup UI . . . . . . 52
Figure 3.11 Automated Attack Technique Execution Script - Initial Menu . . . . 53
xiii
Figure 3.12 Automated Attack Technique Execution Script - C2 Technique Ex-
ecution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
Figure 3.13 Automated Attack Technique Artifacts . . . . . . . . . . . . . . . . 55
Figure 3.14 Identifying Malicious Connections to Vulnserver in ETW Event Data 58
Figure 3.15 Executable Embedded in Word Document . . . . . . . . . . . . . . . 60
Figure 3.16 Identifying Macro-Enabled Word Document Launch in ETW Event
Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
Figure 3.17 Identifying Word Document Writing EXE to Disk in ETW Event Data 63
Figure 3.18 Identifying EXE Being Launched by Word in ETW Event Data . . . 64
Figure 3.19 Base64 Encoded PowerShell Command to Execute Meterpreter TCP
C2 Channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
Figure 3.20 Microsoft-Windows-PowerShell ETW Provider EventNames for Ex-
periment #3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
Figure 3.21 ETW Event Showing First Level Decoding of PowerShell Command
to Execute Meterpreter TCP C2 Channel . . . . . . . . . . . . . . . . . . 69
Figure 3.22 ETW Event Showing PowerShell Code to Load the Ability to Call
Windows API Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
Figure 3.23 ETW Event Showing PowerShell Code to Allocate Memory, Copy
Payload into Memory, and Create a Thread to Execute Copied Payload . 70
Figure 3.24 Source Code of Simulated Malware Executable . . . . . . . . . . . . 72
Figure 3.25 ETW Event Showing Malware File was Downloaded by Microsoft
Edge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
Figure 3.26 ETW Event Showing Malware File was Launched by User in Win-
dows File Explorer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
Figure 3.27 ETW Event Showing a Windows Service was Installed to Execute
Malware Persistently . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
Figure 3.28 Metasploit Registry Persistence Module Execution . . . . . . . . . . 78
xiv
Figure 3.29 Results of Metasploit Registry Persistence Module Execution . . . . 79
Figure 3.30 ETW Event Showing an Attacker Process Accessing Registry Run
Key . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
Figure 3.31 Reverse Engineering of Kernel Function Audit Event Properties [46] 84
Figure 3.32 ETW Event Showing Evidence of PowerShell Reading Memory from
LSASS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
Figure 3.33 ETW Event Count for Kernel Registry Provider . . . . . . . . . . . 87
Figure 3.34 ETW Event Showing Password Hashes Retrieved from the Registry 88
Figure 3.35 Metasploit Psexec Module Executing Lateral Movement Over SMB . 89
Figure 3.36 ETW Event Showing Successful Administrative Authentication to
SMB Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
Figure 3.37 ETW Event Showing Request to Open/Start Powershell.exe Over
SMB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
Figure 3.38 ETW Event Showing Request to Open/Start Interaction with the
Windows Service Control Manager . . . . . . . . . . . . . . . . . . . . . 93
Figure 3.39 ETW Event Showing DNS Request of C2 Server Before HTTP(S)
Connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
Figure 3.40 ETW Event Showing Full URL/URI in C2 Channel Connection . . . 97
Figure 3.41 EDAC Initial Artifact Design . . . . . . . . . . . . . . . . . . . . . . 102
Figure 3.42 Ansible Inventory Definition with Variable Assignment (.ini file) [53] 104
Figure 3.43 Ansible Inventory Definition with Variable Assignment (.yaml file)
[53] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
Figure 3.44 Initial EDAC User-Defined Context File Syntax . . . . . . . . . . . 106
Figure 3.45 User-Defined Context File - Overall Structure . . . . . . . . . . . . . 107
Figure 3.46 User-Defined Context File - Telemetry Definition Section . . . . . . 109
Figure 3.47 User-Defined Context File - Telemetry Definition Allowed Elements 110
xv
Figure 3.48 User-Defined Context File - Telemetry Definition User Providers vs.
Kernel Providers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
Figure 3.49 User-Defined Context File - Construct Definition Allowed Elements . 112
Figure 3.50 User-Defined Context File - Construct Definition Section . . . . . . 113
Figure 3.51 User-Defined Context File - Behavior Definition Allowed Elements . 114
Figure 3.52 User-Defined Context File - Behavior Definition Example . . . . . . 116
Figure 3.53 User-Defined Context File Configuration Line Development . . . . . 118
Figure 3.54 EDAC Main Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
Figure 3.55 User-Defined Context File Parser Code - Section Parsing . . . . . . 120
Figure 3.56 User-Defined Context File Parser Code - Allowed Syntax Elements . 122
Figure 3.57 User-Defined Context File Parser Code - Syntax Tree Iteration . . . 122
Figure 3.58 User-Defined Context File Parser Code - Collect Required Telemetry
Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
Figure 3.59 User-Defined Context File Parser Code - Construct Parsing . . . . . 125
Figure 3.60 User-Defined Context File Parser Code - Behavior Parsing . . . . . . 126
Figure 3.61 User-Defined Context File - Construct Reference . . . . . . . . . . . 127
Figure 3.62 User-Defined Context File Parser Code - Construct Value Fetch . . . 128
Figure 3.63 EDAC User Interface - Main Menu . . . . . . . . . . . . . . . . . . . 128
Figure 3.64 EDAC User Interface - Import UDC Menu . . . . . . . . . . . . . . 129
Figure 3.65 EDAC User Interface - Specify File Path to UDC File . . . . . . . . 129
Figure 3.66 EDAC User Interface - UDC File Parse Results . . . . . . . . . . . . 130
Figure 3.67 EDAC User Interface - UDC File Parse Results . . . . . . . . . . . . 131
Figure 3.68 Arya .NET Crypter Using .NET Reflection to Load C# Code at
Run-Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
Figure 3.69 ETW Monitoring Class Template . . . . . . . . . . . . . . . . . . . 135
Figure 3.70 EDAC User Interface - ETW Class Generation Process . . . . . . . 136
Figure 3.71 ETW Class Generation Process . . . . . . . . . . . . . . . . . . . . . 137
xvi
Figure 3.72 ETW Class Template - Behavior IF Criteria Template (Named Field)138
Figure 3.73 ETW Class Template - Behavior IF Criteria Template (Any Field) . 138
Figure 3.74 ETW Class Template - Behavior IF Block Template . . . . . . . . . 139
Figure 3.75 ETW Class Template - Behavior IF Block Template . . . . . . . . . 140
Figure 3.76 ETW Class Template - Behavior Check Function Template . . . . . 141
Figure 3.77 ETW Class Generation Process - Add to Dictionary for Compilation 142
Figure 3.78 ETW Class Template - Go() Function . . . . . . . . . . . . . . . . . 143
Figure 3.79 ETW Class Generation Process - Compilation and Execution . . . . 143
Figure 3.80 EDAC User Interface - ETW Classes Ready to Compile . . . . . . . 144
Figure 3.81 EDAC User Interface - ETW Classes Compiled and Executed Suc-
cessfully . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
Figure 3.82 EDAC ETW Provider Subscription Code (User) . . . . . . . . . . . 145
Figure 3.83 EDAC ETW Provider Subscription Code (Kernel) . . . . . . . . . . 146
Figure 3.84 EDAC ETW Event Processing Pipeline . . . . . . . . . . . . . . . . 148
Figure 3.85 EDAC ETW Health Check Function . . . . . . . . . . . . . . . . . . 157
Figure 3.86 Example Overwrite of Bytes at EtwEventWrite . . . . . . . . . . . . 157
Figure 3.87 EDAC User Interface - Tamper Warning for EtwEventWrite Overwrite158
Figure 4.1 EDAC Validation Architecture . . . . . . . . . . . . . . . . . . . . . 162

Figure 4.2 Windows Version Distribution 2021-2022 by StatCounter [67] . . . . 163
Figure 4.3 Attack Technique Path Across the MITRE ATT&CK Framework . . 170
Figure 4.4 Experiment 1 - User-Defined Context File Telemetry Section . . . . 172
Figure 4.5 Experiment 1 - User-Defined Context File Construct Section . . . . 172
Figure 4.6 Experiment 1 - EDAC Alert Console . . . . . . . . . . . . . . . . . . 173
Figure 4.8 Experiment 2 - User-Defined Context File Construct Section . . . . 175
xvii
Figure 4.11 Experiment 3 - User-Defined Context File Behaviors Section . . . . 178
Figure 4.13 Experiment 4 - User-Defined Context File . . . . . . . . . . . . . . . 181
Figure 4.16 Experiment 5 - User-Defined Context File Construct/Behavior Section184
Figure 4.19 Experiment 6 - User-Defined Context File Behavior Section . . . . . 187
Figure 4.22 Experiment 8 - User-Defined Context File Behavior Section . . . . . 190
Figure 4.24 Experiment 9 - User-Defined Context File . . . . . . . . . . . . . . . 192
Figure 4.27 Experiment 10 - User-Defined Context File Constructs Section . . . 195
Figure 4.28 Experiment 10 - User-Defined Context File Behavior Section . . . . 196
Figure 4.29 Experiment 10 - User-Defined Context File Behavior Section . . . . 197
Figure 4.30 Experiment 10 - EDAC Alert Console . . . . . . . . . . . . . . . . . 198
Figure 4.31 Experiment 11/12 - User-Defined Context File Telemetry Section . . 199
Figure 4.32 Experiment 11/12 - User-Defined Context File Constructs Section . 200
Figure 4.33 Experiment 11/12 - User-Defined Context File Behavior Section . . 200
Figure 4.34 Experiment 11/12 - EDAC Alert Console . . . . . . . . . . . . . . . 201
xviii
Chapter 1
Introduction
The cyber-attack landscape is constantly changing with new threat groups forming, new
attack vectors being discovered, and new vulnerabilities being discovered putting com-
puter systems around the world at risk of being attacked. One of the fundamental ca-
pabilities to combat threats in this ever changing landscape is ”endpoint protection”
software, where ”endpoint” refers to any workstation or server operating system. Other
names for software in this class can be ”endpoint defense” software or simply ”endpoint”
software. Software products in this class are designed to monitor the operating system
they are installed on and alert users when an action is taken that is consistent with
cyber-attacker actions. The endpoint protection software industry has evolved over the
past two decades from standard ”anti-virus” software, to Endpoint Protection Platforms
(EPP), to Endpoint Detection and Response (EDR) platforms, to eXtended Detection
and Response (XDR) platforms. As with any computer software, these solutions have
evolved over time to incorporate new features, new detection capabilities, new monitoring
areas on systems, and new analytic capabilities to detect the most sophisticated attack
techniques. As noted by Columbus [1], a former contributor to Forbes, 70% of breaches
occur at the endpoint. This statistic illustrates the importance to having e↵ective and
capable defensive solutions installed on the endpoint and corroborates why the endpoint
industry has such continued focus on developing new technologies. It is estimated by The
Radicati Group [2] that the endpoint market will be a $20 billion industry by 2025.
1
Organizations around the world typically deploy one of these classes of endpoint so-
lutions within their environments to detect cyber attacker techniques on endpoints. The
types of attack techniques that endpoint solution vendors choose to detect is dependent
on the vendor’s specific detection methodology, underlying software architecture, and
signature development skills of product engineers. However, the overarching goal is to
detect as many techniques as is feasible based on a standardized body of known attack
techniques that the industry references. The standardized body of techniques that many
endpoint solution vendors choose as a baseline is the MITRE ATT&CK Framework [3],
developed by the MITRE Corporation. As defined by Crowdstrike [4], an industry-leading
endpoint solution vendor, the MITRE ATT&CK framework is a ”curated knowledge base
that tracks cyber adversary tactics and techniques used by threat actors across the entire
attack life-cycle.” While the endpoint industry is growing at a substantial rate in market
value and feature sets, endpoint solutions have a history of detection issues that have
enabled attackers to execute their techniques without detection.
The research presented in this document encompasses all of the research conducted
to identify specific problems with current-generation endpoint defense solutions, the in-
fluence and input that went into identifying a possible solution to some of the problems,
the proposed solution to fix specific problems, the results of validating that the proposed
solution solved the identified problems, as well as the impacts that the proposed solution
will have on the cybersecurity industry. Overall, the research presented in this document
encompasses many years of experience researching and identifying problems in current-
generation endpoint defense solutions culminating in the creation of new software that
detects cyber attack techniques on endpoints, allows operators to configure how the soft-
ware detects techniques, and allows operators to configure the detection scheme on each
endpoint in the enterprise at the process-level. The intent of this research was to develop
a solution that augments existing endpoint defense solutions for many di↵erent types of
system architectures that enables cyber defenders to detect more granular/atomic behav-
2
iors on an endpoint than traditional endpoint systems. By using a combination of the
developed solution and existing commercial endpoint defense solutions, cyber defenders
have a greater chance of detecting more advanced/stealthy cyber threat techniques on
their systems.
1.1 Background of the Problem
The earliest form of endpoint protection was referred to as ”Anti-Virus” software. This
class of software focused on detecting malicious files/executables by identifying when a
known malicious file was written to the disk of an endpoint. Identification was possible
through the use of cryptographic hashes of known malicious files. When a file was written
to the disk of an endpoint, a hash check was performed against a known malicious file
signature database. While this worked in concept, the fundamental design was flawed
because cyber attackers could simply change the contents of the malicious file to achieve no
detection. To combat this simple evasion technique, Anti-Virus software vendors adapted
and built in ”heuristic” detection to detect malicious files. As defined by Kaspersky
[5], heuristic analysis is the process of disassembling a malicious file and examining the
contents of the file to assess the intent of the file before it executes. This level of analysis
worked well for combatting known malicious files and families of malicious files, however,
it still relied on attackers to be static in their development practices leading more stealthy
attackers to develop techniques to evade heuristic analysis. Additionally, heuristic analysis
was focused on analyzing malicious files as opposed to any activities occurring within
the process memory space of processes on a system. Dating back to the early 2010s,
many attacker groups started using PowerShell and Visual Basic scripts for their attack
techniques on Windows targets. Moran et al. of FireEye indicate [6] in a threat intelligence
report from 2014 that APT 3, a Chinese based cyber threat group, used PowerShell
commands to execute many techniques in their campaigns.
3
At that particular time period in the endpoint protection software industry, many com-
mercial endpoint solutions could not detect PowerShell based techniques as many of these
techniques stayed within the process memory space of the attacker-controlled process and
did not write files to disk. This, among other reasons discussed in the next section, caused
a shift in the industry to develop a new architecture of endpoint protection software that
represents the current-generation of solutions developed today. These solutions are more
commonly named, Endpoint Protection Platforms (EPP), Endpoint Detection and Re-
sponse (EDR), and eXtended Detection and Response (XDR). A research paper by Arfeen
et al. identified [7] that the di↵erence between EDR (original) and XDR (newest) solu-
tions is that XDR solutions typically include the ability to ingest data from devices other
than endpoint solutions to give defensive teams visibility and tracking of more devices
and systems than just endpoints. Despite their unique naming conventions and evolving
feature sets many of the more modern endpoint solutions (EPP, EDR, & XDR) share a
similar underlying architecture that includes the use of ”user-mode hooks”, among other
features, that intercept Windows API function calls within each process on a system to
identify malicious attack techniques. Current-generation endpoint solution architectures
have been under inspection by security researchers for many years and numerous bypasses
have been identified in these current-generation solutions that introduce doubt in their
e↵ectiveness for today’s modern threat techniques.
The author’s professional experience encompasses penetration testing, red teaming,
cyber threat emulation, Security Operations Center (SOC) design, threat hunting, and
Defensive Cyber Operations (DCO) consulting for government entities, Fortune 500 enter-
prises, and small/medium sized businesses in many di↵erent industries. This experience
includes a significant breadth of experience with di↵erent types and sizes of system archi-
tectures and has enabled a unique perspective on the ”threat detection at the endpoint”
problem. In the context of current-generation commercial endpoint defense solutions, the
author has had to reverse engineer endpoint solution architectures to identify detection
4
gaps, attack many di↵erent types of systems and develop bypasses for modern endpoint
solutions, and deploy endpoint solutions within systems with Denied, Disrupted, Inter-
mittent, and Latent (DDIL) connectivity. Throughout this experience, the author has
identified numerous gaps in attack technique detection and operational capability that
directly enable advanced attackers to succeed in their attack campaigns.
1.1.1 Endpoint Defense Software Architecture
Before problems within modern endpoint defense solutions can be illustrated, a review
of current-generation endpoint solution architecture and the historical impetus is needed.
Many modern endpoint defense solutions, such as EDR, EPP, and XDR share a sim-
ilar internal software architecture for Windows endpoints that includes a combination
of Windows Services, Windows Drivers, user-mode ”hooks” on Windows API functions,
and user-mode Dynamic-Link Libraries (DLLs) that are loaded into user-mode processes.
This collection of components stems from a historical progression of improvements made
to the Microsoft Windows Kernel over the past decade. As noted by Stein [8], Microsoft
implemented Kernel Patch Protection (KPP) in 2005 which prevents third parties from
making modifications to the Windows Kernel (”patching”) for custom functionality. Also
illustrated by an author with the handle of ”0x00dtm” [9], Anti-Virus software manufac-
turers would perform these kinds of Kernel modification techniques to obtain the ability
to detect and prevent malicious activity on a system. Modifying the Windows Kernel to
obtain the lowest possible level of telemetry for making decisions on malicious or benign
activity is a logical thought process and, in theory, should be the most resilient to attacks
on the technology itself. Once Microsoft introduced KPP, however, third-parties could
not longer patch the Kernel for security purposes.
Endpoint solution vendors identified the challenge of needing to obtain the same capa-
bility of being able to detect and prevent malicious activity on a system without having
access to kernel-level operations such as API calls. The solution was to monitor each
5
user-mode process and the Windows API calls made from these processes. As defined
by Microsoft, user-mode processes can make API calls to the Windows operating system
through a number of user-mode DLLs that can be loaded into each process, such as ker-
nel32.dll. Calls made to APIs within these user-mode DLLs are eventually forwarded to
lower-level ”NT API” APIs within ntdll.dll, which is also loaded into each process, before
being passed o↵ to the kernel to perform the action called. There are a variety of methods
to ”intercept” this API call chain for defensive purposes. One such method to accomplish
this is using software from a project created by Microsoft in 2002 called ”Detours” [10].
This open-source project allows developers to ”redirect” calls to specific API functions
to other locations in memory for inspection before returning back to the original branch
point in the API call chain.
Fundamentally, this is the primary method modern endpoint defense solution vendors
use to detect/block malicious actions by cyber attackers and the terminology used by the
security community is ”user-mode hooking.” Additionally, there are thousands of API
functions available in Windows, however, only a small number of them are used by cyber
attackers. Each endpoint solution vendor has identified which functions are important
for their detection methodology and go through the process of ”hooking” them when
each process is launched. In the endpoint software context, a hook is simply a stub of
assembly code that is inserted before a specific Windows API function of interest that
redirects any calls to that function to a detection DLL for analysis. Figure 1.1 depicts
this architecture used by endpoint solution vendors under the scenario of an endpoint
downloading a malicious executable. In the depicted scenario, the malware executable
does not get blocked by the EDR solution file write monitor because it is nascent. Once
the malware executable is launched on the endpoint system, the endpoint software will
map a DLL containing all of the detection logic into the process and and go through the
process of hooking Windows API functions. Once the malicious process begins calling
Windows API functions to execute malicious logic, the detection DLL inspects each call
6
and determines whether to allow or block the call based on predefined signature criteria.
While this architecture works, there are significant resiliency issues with using user-mode
hooking in attacker-controlled processes. For example, if the attacker-controlled process
has the proper privileges, the attacker can remove user-mode hooks on any function and
in some cases, block the loading of the detection DLL into the process.
Figure 1.1: Modern EDR Hooking/Detection Architecture
1.1.2 User-Mode Hook Dependency Problem
Over the past year and half, the author has been conducting research at the Georgia Tech
Research Institute (GTRI) in collaboration with Bhaskar Rimal of Dakota State Uni-
versity (DSU) on endpoint solution resilience. The two primary research projects being
executed are a study of the e↵ects of removing user-mode hooks from attacker-controlled
processes and a study of various endpoint resilience problems across open-source observa-
7
tions. These research projects are active and will be published after this writing, however,
the majority of the data collection and experimentation work has already been completed.
This work has served as influence and input for the research presented in this document.
The intent of the user-mode hook research is to identify which attack techniques across
various MITRE ATT&CK framework categories (software exploit, command and control,
credential theft, etc.) of a model attack chain would still be detected after user-mode
hooks were removed.
The author and Rimal have identified from experimentation with the open-source
EDR solution, OpenEDR, that only 1/8 atomic attack techniques were detected under
baseline experiments where no user-mode hooks are removed. Under evasion experiments
where user-mode hooks are removed from attacker-controlled processes, OpenEDR did not
detect any techniques executed in the experiment attack chain. These results indicate that
either the OpenEDR agent solely relies on user-mode hooking to detect malicious activity,
or the technique that was executed is not a technique the solution typically detects even
with user-mode hooks installed. In either conclusion, the EDR solution did not detect
primary attack techniques or supporting techniques that are needed to execute the primary
techniques. The author’s professional experience in reverse engineering commercial EDR
solutions can also corroborate that these problems also exist in various industry-leading
commercial EDR solutions.
1.1.3 Endpoint Solution Resiliency Problem
The second research project that the author and Rimal are executing as of this writing
is focused on studying resiliency issues in current-generation commercial endpoint de-
fense solutions based on observations that can be collected in the open-source domain.
Focusing on collecting observations in the open-source domain as opposed to using pro-
fessional experience or experimentation is required because endpoint solution vendors
explicitly prohibit the publishing of any performance metrics, vulnerabilities, or weak-
8
nesses in products. The intent of this project is to collect observations of commercial
endpoint solutions exhibiting failures in specific predetermined failure categories that the
author and Rimal identified before conducting the study. These failure categories are
fault-tolerance of underlying software components, fault-tolerance of the software config-
uration, variability in detection signatures, and alert semantics. The hypothesis for this
separate research project is that collected observations will illustrate a pattern of failures
in current-generation endpoint defense design that methodical and observant attackers
can capitalize on to operate undetected on target systems.
The preliminary results of this research project illustrate that many industry-leading
commercial endpoint defense solutions have had critical problems in one or more of the
identified problem categories that have allowed attack techniques executed by researchers
to go undetected without notifying the user interface that a failure in one of the prob-
lem categories has occurred. The results also indicate that the techniques executed by
researchers against the commercial solutions under test are techniques in a broad range
of the most impactful MITRE ATT&CK framework categories. The current dataset in-
cludes attack techniques and observed failures in the Initial Access, Execution, Privilege
Escalation, Defense Evasion, and Credential Access categories. These initial observation-
s/findings served as influence and input to the research presented in this document because
they illustrate more examples of issues that current-generation endpoint solutions have
with detecting known attack techniques. Section 2.4.2 illustrates in depth the collected
data for this endpoint solution resiliency project.
1.1.4 Technique/Signature Detection Problem
Across both of the research projects being conducted by the author and Rimal, a larger
problem with current-generation endpoint defense solutions has been identified regard-
ing attack technique detection coverage across the body of possible techniques that are
available to attackers. The results of the user-mode hook research and OpenEDR not
9
detecting common techniques during baseline experiments coupled with the author’s pro-
fessional experience with commercial EDR solutions introduces reasonable doubt that
current-generation endpoint defense solutions are designed to detect only specific/known
techniques. While this seems like an obvious fact that endpoint vendors cannot detect
what is not known, the problem lies in the specificity of detection criteria. If endpoint
solution vendors configure their product to only detect the string ”Invoke-Mimikatz” for
the PowerShell variant of Mimikatz credential scraping tool [11], and an attacker changes
the name of the script/parameters to run, the standard alert will not generate. Figure
1.2 illustrates FireEye Endpoint alerting on this behavior and the specific criteria used to
generate the alert. Instead of trying to detect Mimikatz using this single atomic technique
out of the entire body of techniques that an overall technique executes, endpoint solution
vendors should produce many alerts for all of the supporting techniques that Mimikatz
executes on a system to increase the information a defender receives to reach their own
conclusions. The research presented in this document uses Mimikatz as one of the valida-
tion experiments and exhaustive research on the atomic techniques executed by this tool
are detailed in Section 3.4.5.
10
Figure 1.2: FireEye Endpoint Alert on PowerShell Mimikatz Variant
1.1.5 Alert Semantics/Endpoint Configurability Problem
Continuing with the previous example of Mimikatz alerts in FireEye Endpoint, another
problem that is prevalent across current-generation endpoint defense solutions is seman-
tically inappropriate alert messages for detected techniques. In the previous example, the
alert message for detecting a PowerShell variant of Mimikatz that is running the command
”DumpCreds” simply says ”Suspicious Mimikatz Activity.” There are two separate pieces
of information missing from this alert that are critical to defenders being able to make a
decision. First, the fact that a PowerShell variant of Mimikatz is running on the endpoint
introduces an entirely di↵erent set of response questions and actions that may need to
be taken. The alert does not easily indicate that this activity refers to the PowerShell
variant of Mimikatz. There is a high likelihood that that this is a PowerShell variant
11
of Mimikatz being executed because the name ”Invoke-Mimikatz” is the command that
is executed after the Invoke-Mimikatz.ps1 PowerShell script is loaded on an endpoint.
The Invoke-Mimikatz.ps1 PowerShell script was created by the PowerShellMafia and is
publicly available on Github [12].
The second critical piece of information missing from the alert is the fact that the
script ran the function ”DumpCreds” upon launching it. This function takes the action
of dumping user credentials out of memory on the target endpoint. Calling the script
”Invoke-Mimikatz” indicates to a defender that A) the PowerShell variant of Mimikatz
is resident on the endpoint somehow and B) the script has been loaded/executed by
PowerShell. The ”DumpCreds” function call is an entirely separate action that indicates
a more severe action has been taken. None of these details are displayed to the user
alert in a meaningful way to enable a more informed defensive response. In addition to
this, many endpoint defense solutions do not allow users to change the alert strings or
the detection criteria that are native or built into the solution. The vendor decides for
the entire user-base what techniques warrant alerting and what information should be
displayed to the user. The author can confirm from professional experience working in,
designing, and consulting for Security Operations Centers (SOCs) in organizations of all
sizes that these additional facts and circumstances described are the pieces of information
that defenders need conveyed from commercial security solutions. Defenders need explicit
information on what activity occurred with as much context as possible. Otherwise, alerts
could be left unattended without a response due to semantically incorrect, inaccurate, or
inappropriate alert strings. While these examples focus specifically on FireEye Endpoint,
this problem occurs in many other commercial endpoint solutions and is detailed in the
resiliency research in Section 2.4.2.
12
1.1.6 Indiscriminate Trust Problem
The final problem that the author has observed with current-generation endpoint defense
solutions during consulting engagements for commercial and government organizations of
all sizes is an indiscriminate trust problem with the performance and reliability of these
solutions. Some of the organizations that the author has consulted for treat endpoint
defense solutions as a simple security control that must be installed on endpoints without
regard to whether the best endpoint defense solution is chosen for attack technique detec-
tion coverage or not. Some organizations purchase and install industry-leading endpoint
defense solutions, but trust the vendor’s stated efficacy claims for technique detection
coverage without exhaustively testing the veracity of these claims in their operational
environment. This is a problem for organizations because without fully testing efficacy,
performance, or resiliency in their own operational environment, all of the identified issues
described in the previous sections then become possible circumstances for attackers to cap-
italize on without the organization’s knowledge that the problems exist. These real-world
observations coupled with the existing research being conducted illustrate that current-
generation endpoint defense solution design needs to be changed to solve the identified
issues, or another solution is needed to augment existing solutions.
1.2 Special Considerations for Air-Gapped, Discon-
nected, & DDIL Systems
Much of the endpoint defense solution industry is moving to a cloud-based connectivity

model where endpoint agents communicate directly with the endpoint solution vendor’s
public cloud infrastructure for all functions of the solution (alerting, policy enforcement,
etc.). The author has prior experience collaborating with industry-leading EDR vendors
and identified that the reason behind this move is to allow the vendor to perform large-
13
scale data analytics on telemetry emanating from their customer’s endpoints to improve
their detection methodologies. Although the exact enhancements being made to detection
methodologies are proprietary and not known to customers, vendors have illustrated that
by aggregating data across industry verticals, the solution’s efficacy on advanced attacks
will improve over time. This is an optimistic goal with little oversight or validation of the
claims. Additionally, connectivity to cloud systems also becomes problematic for certain
networks and systems that are in use in commercial and government organizations called
”air-gapped” systems/networks.
The primary characteristic of these systems is that they are fully disconnected from
other networks and systems due to their critical operational or sensitive nature. If a system
within an air-gapped network has a traditional cloud-based endpoint defense solution
installed, the endpoint agent will not be able to send telemetry or send alerts to the
cloud system, thereby reducing the solution’s e↵ectiveness. Endpoint solution vendors
try to solve this problem by implementing government approved versions of their cloud
systems for use on air-gapped networks. This can work for some systems, however, there
are classes of systems in operation in government and commercial organizations that
may experience Denied, Disrupted, Intermittent, Latent (DDIL) network connections
throughout their operations. These types of systems frequently experience connection
instability, low throughput, or entirely severed connections. Systems that experience
this type of network connection would not be able to take advantage of any endpoint
solution that receives/analyzes telemetry or generates alerts in a central location. The
only type of endpoint solution that is viable for detecting attack techniques in these types
of environments is one that is managed and monitored locally on the endpoint itself.
As commercial solutions continue to vector away from local management/monitoring,
these critical air-gapped systems will continue to be unprotected from advanced attack
techniques used by well-positioned cyber threats.
14
1.3 Objectives & Impact of Research
The previous sections illustrate in significant depth the state of problems for current-
generation endpoint defense solutions. The most appropriate question that was asked
before beginning the research presented in this document was ”how do you fix the iden-
tified problems?” To answer this, an introspective look at endpoint characteristics across
many architectures in di↵erent industries is needed. The primary observation is that
each system in any architecture is di↵erent. There are di↵erent users for each system,
di↵erent levels of trust for each system, di↵erent levels of trust for each process on each
system, and di↵erent levels of trust for each behavior executed by each process on each
system. This level of granularity is what is needed to accurately define the operational
boundaries of systems, and thus, define the expected behavior that systems and processes
should exhibit. If a software solution were configured or programmed to understand these
distinctions on systems, the solution could then alert on very granular actions taken on a
system. The research presented in this document focused on developing a solution with
these characteristics. The overarching objectives of the research were to:
1. Create a software artifact that analyzes process/system events in real-time against

a user-defined configuration file.
2. Create a user-defined configuration file along with a useful syntax that allows users
to configure behavioral boundaries for each process on the system.
3. Test the developed software artifact against real-world attack simulations and sys-
tem configurations to determine the efficacy and usefulness of such a solution.
Given the rapid increase in complexity in newly discovered cyber attack techniques
over the past decade, granular definition and inspection of system behavior is a promising
area to solve the problem of positively identifying cyber attack activity going forward. It
should be noted that this research does not intend to replace traditional endpoint defense
15
solutions as these systems are still useful for detecting known malicious techniques that
are based on known threat intelligence. The hope for the software artifact developed in
this research is that it can be used in concert with traditional commercial technologies
to give system operators the necessary augmentation to know, more granularly, what is
happening on each system in an enterprise.
16
Chapter 2
Literature Review/Related Work
The research presented in this document focuses on developing a software agent that
detects cyber threat techniques using solely Event Tracing for Windows (ETW) as the
telemetry source, however, there is a significant amount of related research executed by the
author and executed by independent researchers that have served as influence and input to
drive the direction of all of the components of this research. Over the past few years, the
author has conducted a number of internal Research & Development (R&D) projects at
the Georgia Tech Research Institute (GTRI) as Principal Investigator (PI) in the subjects
of Endpoint Detection and Response (EDR) agent resiliency, process behavior monitoring
using Sysmon [13], and process behavior classification [14]. In addition to this, numerous
existing research projects were identified in the open-source and academic domains that
corroborated the author’s prior research as well as the direction of the research in this
document. The following sections detail these research projects that focused on similar
aspects of attack technique detection and process monitoring that shaped and informed
the research presented in this document.
17
2.1 Attack Technique Detection Research
2.1.1 Technique Detection Using Sysmon
Prior to conducting the research presented in this document, there were a number of
questions that needed to be answered regarding native/free telemetry sources available
for Windows, their performance impact to systems, their ease of use in real environments,
their ability to be used programmatically using custom developed tools, and their useful-
ness with respecting to detecting attack techniques. During the literature review, a small
number of academic research projects were found to be helpful for this research that focus
on using telemetry sources such as Sysmon and ETW to detect various types of attack
techniques. One of the projects identified first by Matsuda et al. [15] focuses on using
Sysmon to monitor DLL loads for attack identification. Sysmon is a free tool developed
by Microsoft [13] that logs specific types of behaviors for each process that occur within
the Windows operating system. Each behavior identified in one of these categories is then
logged to the Windows Event Log for operator analysis.
In the project by Matsuda et al., the team experimented with a specific category of be-
haviors that Sysmon logs, Image/Dynamically Linked Library (DLL) loads into processes.
This category of events is logged by Sysmon into the standard Sysmon Windows Event
Log as Event ID 7. Using the Sysmon configuration file (XML), operators can enable this
category of logging and receive a log any time any process (or a specific process defined
in the configuration file) loads a DLL into the process memory space. From an attacker
perspective, once an attacker has code running inside a process on a target system, the
operational goal may be to execute subsequent techniques that require additional DLLs
to be loaded into the process. Matsuda’s research team designed experiments to identify
which DLLs are loaded into attacker-controlled processes when executing specific known
malicious tools.
18
The tools used for experimentation were China Chopper, Mimikatz, PowerShell Em-
pire, and HUC Packet Transmitter. During the experimentation, the team utilized a set
of baseline experimentation results to compare common DLLs loaded by processes against
additional DLLs loaded by the malicious process [15]. Using this information, the team
was able to come up with a list of DLLs and detection signatures using the Elastic Stack
[16]. This project assisted the research presented in this document by illustrating that is
possible to use a single data source, such as DLL loads, to identify that a process may
be acting maliciously and that a malicious tool might be present on the system. Going
into this research, the proposed features for the software artifact were to have the ability
to specify criteria, such as known-malicious DLLs, to detect attack techniques and the
project by Matsuda et al. supplied influence to expand this capability to ETW data
sources.
2.2 Process Behavior Monitoring Research
One of the first research projects the author conducted at GTRI that informed the research
presented in this document was on the subject of monitoring individual processes for
evidence of attack behaviors from June 2020 - July 2021. This research project was
named ”At-Scale Process Monitoring for Cyber Threat Identification” [17] and began
its first phase as an exploratory e↵ort to identify if Microsoft Sysmon could be used
to identify that a cyber attacker has executed a malicious behavior/technique within
individual processes. Sysmon provides operators with categories of events for specific
types of threat vectors. These categories are Process Creations (Event ID: 1), Process
Terminations (Event ID: 5), Network Connections (Event ID: 3), Driver Loads (Event
ID: 6), Image/DLL Loads (Event ID: 7), Remote Thread Creations (Event ID: 8), Raw
File Access reads (Event ID: 9), Process Accesses/Opens (Event ID: 10), File Creations
(Event ID: 11), Registry Changes (Event ID: 12), Named Pipe Actions (Event ID: 17-18),
19
WMI Actions (Event ID: 19-21), DNS Requests (Event ID: 22), and Process Tampering
(Event ID: 25) among others. As noted by the Russinovich and Garnier [13], Sysmon
data generated in these categories can be used to detect malicious activity on systems if
they are consumed and analyzed. Sysmon data collection in large scales became the focus
of the second phase of this research project.
In the first phase of this research project, the author executed specific attack techniques
that sufficiently stimulated the built-in Sysmon event categories in order to analyze the
resultant data for a variety of criteria. First, an analysis was conducted on whether
events generated by Sysmon included specific enough data to detect the specific technique.
During this process, each event was analyzed in the Windows Event Log to identify what
field names were included in each category of event, and which field names included
values with the artifacts/evidence of the techniques executed. One of the most important
lessons learned by iteratively analyzing techniques against event data was the concept of
correlation, which greatly shaped and informed the direction of the research presented in
this document. For each attack technique that was executed, individual events from a
specific category of events that Sysmon provides rarely illustrated evidence of technique
execution on their own. It was identified that in order to identify all components of a
specific technique that was executed, many event categories and fields would need to be
correlated together to reach certainty that the technique was executed. For example,
the author downloaded and executed a simulated malware binary on a test system that
established a Command and Control (C2) channel. In order to fully detect that these
actions occurred, event data from the file creation (Event ID: 11), process creation (Event
ID: 1), DNS (Event ID 22), and network connection (Event ID: 3) categories all need to
be correlated together. However, the capability of monitoring each process for malicious
behaviors was proven feasible and the research could progress into the next phase.
The second phase of the research focused on methods to consume Sysmon log data
resident in the Windows Event Log from endpoints in an enterprise and send them to
20
a centralized logging platform for analysis and alerting of attack techniques. A series
of experiments were conducted using the Splunk Enterprise [18] log aggregator and the
Splunk Universal Forwarder [19] to collect events from the Windows Event Log. These
tools and techniques to collect Windows Event Logs with the Splunk Universal Forwarder
are common amongst enterprises. The results of the experimentation revealed that events
generated by Sysmon could be collected, sent, and consumed by enterprise logging plat-
forms at an enterprise scale, however, the amount of data sent to the logging platform from
each endpoint would depend on the specific Sysmon event categories that were enabled
through the Sysmon configuration file. Sysmon provides an XML-based configuration file
to enable event categories as well as configure filters to filter out specific events. Figure
2.1 illustrates an example configuration file developed [20] by SwiftOnSecurity for the
purposes of attack technique detection.
Figure 2.1: SwiftOnSecurity Sysmon Configuration File [20]
At the end of the research project, Sysmon proved to be a high-fidelity data source
for consumption into a centralized log aggregation tool, but revealed a number of key
constraints that ultimately drove the research detailed in this document in a di↵erent
direction with respect to telemetry sources to detect attack techniques. First, Sysmon
21
logs all events into the Windows Event Log by default and this cannot be changed easily.
Second, Sysmon event types/categories are finite and no additional telemetry sources in
Windows can be enabled. For example, Windows API calls to certain high-value func-
tions, logging of PowerShell commands, logging of .NET Framework Common Language
Runtime (CLR) execution, and logging of authentication and authorization events can-
not be enabled and logged through Sysmon. These events are sometimes logged in the
Windows Event Log through other logging channels and would have to be collected and
correlated against Sysmon events. Third, the process of filtering what Sysmon events
are normal and abnormal in the Sysmon XML configuration file is cumbersome, prone
to user error, prone to system instability, and time consuming. Finally, Sysmon event
logging cannot easily be integrated into custom software for programmatic analysis of
events without the introduction of convoluted data consumption, parsing, and reliability
challenges with the Windows Event Log. All of these facts ultimately drove the decision
to use Event Tracing for Windows (ETW) as the core telemetry source for the research
presented in this document.
2.3 Process Behavior Classification Research
Following the process behavior monitoring research, the author also conducted an internal
R&D project at GTRI from July 2021 - June 2022 named ”Process Behavior Analysis and
Classification for Threat Detection” [14]. The purpose of this research was to investigate
attack techniques typically executed during cyber-attacks and identify subordinate atomic
techniques that are included for each. Once atomic techniques for an example attack
chain were identified, the research then classified each of the techniques into categories of
behaviors. The final component to the research was to map identified atomic techniques
to Event Tracing for Windows (ETW) providers, field names, and field values. Similar to
the process behavior monitoring research project, this project developed an attack chain
22
that was used for experimentation and analysis. This attack chain included techniques
across the most impactful categories of the MITRE ATT&CK framework from software
exploits, to command execution, and ending with credential access.
The primary component of this project that aided the development of the software
artifact in the research detailed in this document was the extractions of atomic techniques
and mapping them to ETW event providers/fields. In order to identify the atomic tech-
niques for each overall technique, iterative reverse engineering was required to identify
how the technique functioned on a target system. Figures 2.2 & 2.3 illustrates the map-
ping that was completed for the attack technique chain. Each atomic technique that was
identified in the chain was categorized based on the most logical behavior class and then
mapped to telemetry sources that are available for detection of the atomic technique.
For each identified telemetry source, the event category where evidence of the technique
was found was noted. Finally, within each event category, the applicable fields where
evidence of the technique existed were noted. These atomic attack techniques, behavior
classes, behavior categories, ETW telemetry sources, and identified fields served as input
to developing the software artifact in the research detailed in this document.
23
Figure 2.2: Atomic Attack Technique Telemetry Mapping Steps 1-3 [14]
24
Figure 2.3: Atomic Attack Technique Telemetry Mapping Steps 2-3 [14]
2.4 Endpoint Defense Solution Research
As of this writing, the author and Assistant Professor Bhaskar Rimal of Dakota State
University are conducting two research projects on the subjects of current-generation
endpoint defense solutions (EPP, EDR, XDR) and various issues they have in real op-
eration. These issues are fault-tolerance against intentional evasions of detection com-
ponents, fault-tolerance against errors/bugs, technique/signature detection coverage, and
semantically appropriate alert messages.
25
2.4.1 EDR User-Mode Hook Research
The first research project being conducted by the author and Rimal is focused strictly on
Endpoint Detection and Response (EDR) solutions and their usage of user-mode hooking
on Windows API functions to detect attack techniques from processes. The overarching
hypothesis of this project is that removing EDR user-mode hooks in an attacker-controlled
process will result in no detection of subsequently executed attack techniques in a model
attack chain. User-mode hooking has been a common practice for Anti-Virus/Endpoint
vendors for over a decade due to the relative ease of implementation and high-yield in
detection capability. In summary, a user-mode hook is a stub of assembly code that
is inserted before each Windows API function mapped into user-mode processes that
redirects any call to that function to code in a Dynamically Linked Library (DLL) owned
by the endpoint solution for detection. If a function call is determined to be malicious by
the endpoint vendor, the call is blocked. If the call is determined to be not malicious, the
call is forwarded back to the original call chain for Windows to process. From an attack
perspective, the goal would be to remove this hook for a specific function or prevent
the endpoint DLL from being loaded into the attacker-controlled process. Figure 2.4
illustrates what a user-mode hook looks like inside of a debugger. The debugger is attached
to the attacker-controlled process and focused on the NtReadVirtualMemory Windows
API function location within the ntdll.dll DLL.
Figure 2.4: Debugger Illustrating an EDR User-Mode hook
26
The empirical method chosen for the EDR user-mode hook research was the Single-
Case Mechanism Experiment (SCME) of the Design Science Research (DSR) methodology.
In this methodology, experiments are designed and executed against a software artifact
with measurement criteria to determine how the artifact performs under real-world con-
ditions. The chosen artifact to experiment with was OpenEDR, an open-source EDR
solution. The reason for choosing this solution as opposed to a commercial solution is
that commercial EDR solutions typically restrict publication of any performance/vulner-
ability information. OpenEDR is open-source and does not currently have any of these
restrictions in the End User License Agreement (EULA). For the design of the experi-
ments, a model attack chain was created that contains real-world attack techniques across
various categories of the MITRE ATT&CK framework.
The chosen techniques to experiment with include 2 software exploits, 1 custom shell-
code execution technique, 1 Metasploit Meterpreter C2 channel technique, and 1 cre-
dential scrape technique using Mimikatz [11]. The experimentation process starts with
baseline experiments to identify which techniques are detected by OpenEDR under non-
evasive conditions. Once baseline experimentation is complete, the same experiments are
executed, except, user-mode hooks for the attacker-controlled process are removed by
overwriting the assembly code stub in front of each Windows API function. The results
of the experiment are measured to identify which attack techniques in the given attack
chain result in no detections after the user-mode hooks are removed.
This EDR user-mode hook research project is planned to be published in the form of
a paper once final experimentation results are collected. While this particular project is
designed to study the e↵ects of intentional evasions against EDR solutions, it is significant
for the research presented in this document because initial experimentation has revealed
that OpenEDR only detects 1/8 of the overarching techniques and 1/20 of the entire body
of atomic techniques (including supporting techniques) during baseline experimentation.
These initial results were used as influence and input to the research presented in this
27
document because the hypothesis indicates that the developed software artifact will detect
more atomic techniques than traditional EDR solutions.
2.4.2 Endpoint Solution Resiliency Research
The second research project being conducted by the author and Rimal as of this writing is
focused on surveying the open-source domain for resiliency issues that current-generation
commercial endpoint defense solutions have in the areas of fault-tolerance, signature de-
tection, and alert semantics. At the start of this research project, the author and Rimal
identified key problem categories that endpoint defense solutions have that can lead to
an attacker’s techniques not being detected. These problem categories were mapped to
specific failure paths that the author has observed using commercial endpoint solutions
during attack experiments. Figure 2.5 illustrates the identified failure paths for each prob-
lem category against an example attack technique chain. The overarching goal for this
research is to illustrate that commercial endpoint solutions are vastly complex in their
internal architectures to the point that any single failure in a component or step in the
pipeline can result in a degraded alert condition or no alert at all for a particular tech-
nique. The e↵ects of these degraded alert conditions are that highly observant attackers
can take advantage of these circumstances to guide their attack campaigns in such a way
that allows them to operate undetected.
28
Figure 2.5: Identified Endpoint Solution Problem Categories and Failure Paths
2.4.3 Endpoint Solution Research
Among the related research identified during the literature review, one research paper was
identified that focuses on Endpoint Detection and Response (EDR) solutions specifically,
the issues these solutions have in practice, and identified where improvements can be
made. In this paper, Hassan et al. identified [21] that while EDR solutions are heavily
used in commercial organizations, they su↵er from three challenges in practice, two of
which are directly applicable to the research presented in this document. First, EDR
solutions reference and use industry standard frameworks, such as the MITRE ATT&CK
framework, however, these frameworks often do not distinguish techniques that are in-
nocuous or normal in nature, but malicious when an attacker uses them. The example
given by Hassa et al. is the File Deletion (T1107) technique in the MITRE ATT&CK
29
framework. This technique is used as an evasion technique by attackers, however, file
deletions occur routinely on systems. This makes it difficult for EDR vendors to conclude
when specific techniques are malicious and when they are not, thereby, exacerbating
the problem illustrated in previous sections with signature detection gaps. The second
challenge identified by Hassan et al. relates to the alert semantics problem identified
in Chapter 1. Cyber defenders have to manually correlate information contained within
endpoint solution alerts to reach defensive conclusions and response actions. Hassan et al.
developed a novel way to address these challenges by developing [21] Tactical Provenance
Graphs (TPGs) that illustrate causal dependencies between alerts generated by EDRs in
a compact visualization to reduce defender investigation time.
30
Chapter 3
Research Methodology & Proposed

Solution
3.1 Overall Research Design
As defined in the introduction, the primary goal of this research was to design a software
artifact that is used to detect cyber attack techniques on endpoints, but also design a
software artifact that solves a few key issues that traditional endpoint defense solutions
(EPP, EDR, XDR) have. The current issues that a↵ect traditional endpoint solutions are:
current-generation endpoint defense solutions do not allow the ability to change the core
telemetry source(s) used for detection criteria, they do not allow the flexibility to change
the programmatic interpretation of events/data coming from these core system telemetry
sources, they do not allow the ability to change the alert semantics for built-in detection
signatures, they have a low level of technique detection coverage for supporting/atomic
behaviors/techniques, and they are not fault-tolerant or fault-aware that an underlying
component of the solution has failed. By breaking down the core issues that traditional
endpoint defense solutions have, a new software artifact can be specially designed to fix
some of these issues. The hope for this research is that a software artifact developed
with the identified issues in mind will enable operators to detect a broader class of attack
techniques on a system, but also give operators the flexibility to adapt to new threats in
the future. Sections 3.1.1-3.1.5 detail the overall design of the research presented in this
document and the phased approach that was taken to achieve the desired results.
31
3.1.1 Chosen Methodology
Given that this research focused on creating a software artifact that is intended to add
capability to an existing scientific field, the most appropriate choice for a research method-
ology was the Design Science Research (DSR) methodology. Wieringa states [22] that one
of the purposes of DSR is to investigate an artifact within a specific problem context, and
artifacts designed with this methodology should improve something within that context.
The problem context for this research is detecting cyber attack techniques on endpoints
where traditional defenses may fail to detect some or all of the techniques in a given attack
chain. While this problem context is broad, the hope is by creating a software artifact that
strictly solves each of the identified problems, operators of the artifact will immediately
see an improvement in the problem context. In order to assess whether an improvement
has been made in a problem context, software artifacts must be validated against real-
world conditions. The chosen method to accomplish this in this research was to execute
iterative attack technique experiments against the developed artifact to validate that it
is able to detect the executed techniques. Since this research will require a significant
amount of experimentation to validate the artifact, the most logical empirical research
method to choose within DSR is the Single-Case Mechanism Experiment (SCME). This
empirical research method allows for iterative experimentation and analysis of results to
determine success/failure.
3.1.2 Hypothesis and Research Questions
In order to build a software artifact that solves the problems that traditional endpoint
defense solutions have, a number of research questions, functionality requirements, and
software architecture requirements needed to be defined to ensure the research had a
distinct scope that enabled on-time completion. The primary research questions for this
research were:
32
1. Can real-time process behavior events from multiple Event Tracing for Windows
(ETW) event providers be simultaneously consumed and analyzed programmatically
without user intervention?
2. Can a data schema of ”system behaviors” be created in such a way that A) syntax
of the schema maps directly to incoming event data from ETW event providers, and
B) the schema syntax is easily configurable by operators?
3. Can the user-defined schema data be consumed by the software in such a way that
the software can quickly identify matching behaviors within incoming events for
alerting?
4. Are there any gaps in ETW event provider data that do not allow user-defined
schema data to sufficiently identify the activity?
The primary research questions focus on answering whether it is possible to A) con-

sume ETW event data in a fast and concurrent way, B) be able to process this data
against a user-defined schema of behavior, and C) be able to alert/log abnormal behavior
to operators quickly. As such, the hypothesis for this research should also reflect these
specific goals/details. The established hypothesis for this research was: A software ar-
tifact that is able to successfully analyze low-level kernel/user events against a highly
detailed schema of data defining granular boundaries of behavior from these events will
result in a more complete level of detection of attack techniques in an attack chain than
traditional endpoint defense systems.
3.1.3 Background/Supporting Research Phase
In order to successfully develop a software artifact that consumes ETW events and de-
tects attack techniques based on a user-defined configuration file, a few problems needed
to be solved in supporting research areas first. These problems were identifying whether
33
there are gaps in ETW event data for any attack techniques that would a↵ect overall
development of the artifact or the user-defined configuration file, how to reach this con-
clusion, and how to reach this conclusion quickly. To solve this particular problem, two
automation scripts were created to execute a body of attack techniques and iteratively
monitor all ETW event providers on a system during the execution of the attack tech-
niques. The logged event data from each ETW provider was then analyzed to identify
artifacts/evidence of the executed attack techniques. Once this analysis was completed, a
conclusion could be reached on whether there were any gaps within ETW event data for
any of the executed attack techniques. This background research is discussed in Section
3.2.
3.1.4 Artifact Development Phase
Once the initial background research was complete, development of the user-defined con-
figuration file and main artifact could commence. During this phase of the research, it
was paramount to maintain the primary focus of developing capabilities that would solve
problems within the identified problem context. For example, the problems identified
with current-generation endpoint defense solutions range from technical problems to us-
ability problems. As the development progressed, creating a solution that both could be
usable for real defensive operators and be e↵ective at detecting attack techniques drove
specific design/functionality choices. The development phase of the research is discussed
in Section 3.3.
3.1.5 Artifact Validation Phase
The final component to this research was the validation phase. In this phase, the devel-
oped software artifact and user-defined configuration file were tested against the real-world
attack techniques developed in the background research phase. In order to execute the
validation phase against the developed software artifact, three core components to the
34
research needed to be developed. First, a proper validation architecture needed to be
designed and implemented. In the context of validating a software artifact, the validation
architecture needed to encompass real-world configurations of systems to execute on, and
all of the required networking to support attack experiments. Second, the experiments
themselves needed to be designed in such a way that they sufficiently test the software
artifact’s capabilities, but also be representative of real-world conditions. Third, a set of
measurement criteria needed to be developed that would be used during the experimenta-
tion. Given that the purpose of the software artifact is to detect cyber attack techniques,
the measurement criteria developed for this research was a positive or negative detection
on specific atomic techniques across the developed attack chain. The validation phase of
this research is discussed in Chapter 4.
3.2 Background & Supporting Research Phase
3.2.1 Event Tracing for Windows (ETW) Introduction
Every version of Windows since Windows 2000 [23] has had Event Tracing for Windows
(ETW) available for software developers to consume events coming from the Windows
Kernel for troubleshooting and application performance analysis [24]. Figure 3.1 illus-
trates how all of the available ETW providers (⇠1,100) can be enumerated in Windows
by using the ”logman query providers” command. Each ETW provider is assigned a
unique GUID that is used to subscribe to ETW events in either Windows performance
analysis tools or directly in developed code using available Windows APIs.
35
Figure 3.1: Windows 10 21H1 ETW Providers
According to Microsoft [24], event tracing is provided by three core components called
Providers, Controllers, and Consumers (or Subscribers). Providers are components of the
Windows operating system that generate events on activity that has happened in that
component. Users cannot manipulate how the providers fetch, bu↵er, or deliver events.
However, they can control when providers should start sending events and whether the
events should be sent to a log file or directly to a consumer. Consumers are the components
that consume events from providers through the use of event tracing session bu↵ers. These
bu↵ers are maintained by the operating system and are the mechanism for bu↵ering events
of variable rates into data structures for applications to then consume the events. If users
choose to subscribe to ETW providers using a custom developed application, they would
write code to initiate a subscription to a given provider (controller), and then write code
to handle the ingestion and processing of events from the provider (consumer). The
overarching ETW event subscription and delivery pipeline is depicted in Figure 3.2.
36
Figure 3.2: Windows Event Tracing Model [24]
In recent years, ETW has become the subject of endpoint defense research for identi-
fying evidence of attacker techniques because of the significant amount of data available
in the 1,100 ETW event providers. Each event provider in the operating system has
numerous event classes that represent individual atomic behaviors/actions that were ex-
ecuted within that operating system component. During the literature review, numerous
research projects were found to be utilizing ETW events as the telemetry source for iden-
tifying attack techniques. One such project, named SilkETW, is a very well-known tool
for security research using ETW event data. Boonen illustrates [25] the benefits of using
this tool and ETW as a telemetry source for security research. Boonen is also the author
of SilkETW and illustrates [26] on Mandiant’s Github page that SilkETW can be used to
consume image load events that indicate a process has loaded an image known to be used
by well-known credential theft tool Mimikatz [11]. Evidence of this activity is located
within the JSON field XmlEventData.FileName as depicted in Figure 3.3. SilkETW cap-
tures ETW events in their native XML format and converts all of the fields to JSON for
easy consumption and manipulation by other tools.
37
Figure 3.3: SilkETW Capturing Events with Evidence of Mimikatz Behaviors [26]
While this level of visibility into the operating system provides tremendous defensive
capability, this example represents one single artifact of one single attack technique. At-
tack chains on a Windows system can be very complex and include many techniques that
are executed throughout the Windows operating system. This introduced the additional
question of ”can ETW providers be used to identify many di↵erent categories of tech-
niques in the MITRE ATT&CK framework, or are only some techniques identifiable?”
that needed to be added to the overall list of questions in this research.
3.2.2 ETW Event Data Completeness/Attack Technique Cov-
erage Problem
During the beginning phases of this research, it was determined that if the hypothesis
for the research defined that the developed software artifact will alert a user to a ”more
complete” set of attack techniques using ETW as a telemetry source, initial research
was needed in the area of ETW event providers to identify any gaps in attack technique
coverage across a model attack chain. For example, if the chain of attack techniques chosen
for validation of the developed software artifact included a software exploit, a macro-
enabled word document, and an HTTP Command and Control (C2) channel, evidence
38
of each of these techniques will likely exist in di↵erent ETW providers, event names,
and various field names. A methodology and framework were needed to answer these
questions:
• Which ETW provider is the most appropriate for detecting a specific attack tech-
nique or sub-technique?
• Within a specific event provider, which event names and field names are the most
appropriate for detecting a specific attack technique?
During the literature review, no existing research in this area of attack technique coverage
in ETW providers was found in either academic sources or open sources. Since each
Windows installation includes ⇠1,100 ETW providers and the chosen attack technique
chain used for validation included 12 techniques, automation was needed to accelerate
this discovery process.
3.2.3 Automated ETW Event Generation
The overarching goals for this branch of the primary research were to develop an automa-
tion script that subscribes to all of the available ETW providers on a system, executes
each attack technique in a chosen attack chain, and logs any events captured in any of
the ETW providers to a JSON file for analysis later. Since SilkETW is a stand-alone
executable and is able to output events to a JSON file, it was fitting to use this tool
as part of the developed automation script to capture events across all ETW providers
during attack stimulus.
Of special note, one of the primary goals for development of this script was for the
script to be idempotent. Ansible, an industry-standard IT automation framework, defines
idempotent [27] as ”An operation is idempotent if the result of performing it once is exactly
the same as the result of performing it repeatedly without any intervening actions.” For
this automation script to be able to execute repeatedly without causing any issues on
39
the executing system and still product the desired results, multiple checks had to be
implemented throughout the code to adhere to proper ETW subscription procedures. For
example, the developed automation script is programmed to execute these items in order
to support the idempotency requirement:
1. Delete old JSON log files in the collected logs directory.
2. Stop any prior orphaned SilkETW sessions in the operating system to prevent reach-
ing an eventual resource limit (64 concurrent ETW sessions).
3. Capture a list of all available ETW providers on the executing system.
4. Iteratively launch the SilkETW executable to monitor a single ETW provider in the
captured list and pause at 15 concurrent instances of SilkETW.
5. Execute a separate attack chain script with all chosen techniques.
6. Stop all SilkETW processes.
7. Stop all SilkETW sessions in the operating system to prevent reaching an eventual
resource limit (64 concurrent ETW sessions).
8. Repeat steps 4-7 in batches of 15 providers from the master ETW provider list until
the end of the list is reached.
Figure 3.4 details the automation script execution flow. At the time of this writing,
the Automated ETW Event Generation script is closed-source, however, the intent is to
open-source this script once an export-control review is completed. Upon completion
of export-control review, the script will be available from the Georgia Tech Research
Institute (GTRI) Github page [28].
40
Figure 3.4: Automated ETW Event Generation Script Execution Flow
3.2.4 Automated Attack Technique Generation/Execution
At this point in the ETW provider coverage research, the automation script for subscribing
to all possible ETW event providers was complete, but a similar script for executing attack
techniques was needed to execute as part of the original script. This attack technique
automation script proved to be more difficult to develop as there were numerous design
goals, functional requirements, and performance requirements. The overarching design
goals were:
1. The script needed to be written in PowerShell and all techniques needed to be
41
launchable using PowerShell code.
2. The script needed to execute once on the same system as the ETW generation script,
and without user interaction, sequentially execute a set of techniques in a specific
folder or set of folders.
3. If a specific technique required multiple interactions from an attacking system and

victim system, the script needed to be able to handle all of this interaction auto-
matically. Multi-system interactive techniques could include Command and Control
(C2), credential access, lateral movement, etc.
4. Ability for the user to quickly add and remove techniques as needed.
5. The script needed to extract artifacts/strings of each technique executed and store
in a text file for later analysis.
There are a number of open-source projects [29][30][31][32][33][34] that specialize in ex-

ecuting MITRE ATT&CK framework techniques for the purposes of red teaming, pen-
etration testing, and defensive tool testing. Upon analyzing the available open-source
projects, it was determined that operationalizing them into the already created ETW
event generation script would be too time consuming as the projects did not meet all
of the requirements above. Therefore, it was decided that a custom script and attack
technique execution framework would need to be developed instead.
3.2.4.1 Automated Attack Technique Execution Framework Development
The first step in developing the automation script for executing attack techniques was to
identify the techniques themselves. Since the primary research requires validation of the
developed software artifact, it was fitting to proceed and establish the attack techniques
for the later validation at this stage in the research. The hypothesis dictated that the
developed software artifact would alert to a ”more complete” set of attack techniques than
42
traditional endpoint defense solutions (EPP, EDR, XDR). In order for this hypothesis to
be proven true, the validation of the artifact needed to include a broad set of attack
techniques. This formed the basis for developing the chosen attack technique chain that
would be used for both the automation script in this background research and the final
artifact validation.
A core reference point for breadth with respect to attack techniques in this research
is the MITRE ATT&CK framework built by the MITRE Corporation [3]. As defined
by Crowdstrike [4], the MITRE ATT&CK framework is a ”curated knowledge base that
tracks cyber adversary tactics and techniques used by threat actors across the entire
attack life-cycle.” On the main ATT&CK web page [3], users can browse through all of
the atomic attack techniques within various ”categories” that represent typical attack
campaign progression. Figure 3.5 depicts this web page and all of the available attack
categories and techniques within each category.
Figure 3.5: MITRE ATT&CK Framework Matrix
As illustrated in Chapter 2, the author has conducted prior research on attack tech-
nique detection coverage with EDR platforms through the user-mode hook and EDR
43
resiliency research. In the user-mode hook research, the author developed a custom at-
tack chain that included a software exploit, followed by shellcode execution, followed by
PowerShell command execution, followed by Command and Control (C2), followed by
credential access with Mimikatz. Running this attack chain against OpenEDR resulted in
a detection on the credential access technique of the chain, but no other technique in the
chain. These results are not surprising as they match observations the author has made
on various commercial endpoint defense solutions during consulting engagements. The
combinations of these facts helped shape and inform the attack chain developed for this
research. It should be noted that the intent of this research and the developed artifact
is not to detect every possible attack technique listed in the MITRE ATT&CK frame-
work. The intent of this research is to illustrate that using ETW as a telemetry source
along with configurability can result in more coverage across the ATT&CK framework
than traditional endpoint defense solutions. Therefore, the developed attack chain only
includes a small number of techniques from each category that are highly representative
of techniques used in real attacks or are popular amongst defenders for detection. Table
3.1 details all of the techniques chosen for validation of the artifact in this research as well
as the automated attack execution script for ETW event coverage.
44
Table 3.1: Attack Techniques Chosen for Validation in Automation Script
Experiment
# ATT&CK Category Technique - Sub-technique Notes
1 Initial Access T1190.000 - Exploit Public Facing Ap- Stack bu↵er overflow on network facing
plication application (modified Vulnserver [35])
2 Initial Access T1566.001 - Phishing - Spear-Phishing Macro-enabled Word document, decodes
Attachment EXE with MessageBoxA() shellcode on
disk and executes it
3 Execution T1059.001 - Command/Script Inter- Encoded PowerShell command, decodes
preter - PowerShell and connects to localhost
4 Execution T1204.002 - User Execution - Malicious Executable that executes MessageBoxA()
file shellcode
5 Persistence T1543.003 - Create/Modify System Create a Windows service that executes a
Process - Windows Service PowerShell command
6 Persistence T1547.001 - Boot or Logon Autostart - Creates a registry run key to execute a
Registry Run Keys PowerShell command
7 Privilege Escalation T1068 - Exploitation for Privilege Es- Software exploited in experiment 1 was
calation running as an administrator
8 Credential Access T1003.001 - OS Credential Dumping - System privilege C2 channel is established
LSASS through lateral movement, Mimikatz is
loaded into the memory of the C2 chan-
nel and credentials scraped out of LSASS
9 Credential Access T1003.002 - OS Credential Dumping - System privilege C2 channel is established
Dumping SAM through lateral movement, Metasploit
module for SAM dumping (post/win-
dows/gather/smart hashdump) is exe-
cuted through C2 channel
10 Lateral Movement T1021.002 - Remote Services - SM- PSExec-type lateral movement inbound to
B/Windows Admin Shares ADMIN$ using a compromised local ad-
ministrator (500) account
11 Command and Control T1071.001 - Application Layer Proto- Msfvenom generated executable with
(C2) cols - Web Protocols (HTTP) Metasploit Meterpreter HTTP shellcode
executes and establishes an HTTP C2
channel
12 Command and Control T1071.001 - Application Layer Proto- Msfvenom generated executable with
(C2) cols - Web Protocols (HTTPS) Metasploit Meterpreter HTTPS shellcode
executes and establishes an HTTPS C2
channel
Once each of the identified techniques were tested in an isolated virtual network con-
sisting of a Kali Linux virtual machine (simulated attacker) and a Windows 10 virtual
machine (simulated victim), the execution of the techniques could then be automated.
As noted in Table 3.1, some techniques exist and are executed on the victim virtual ma-
chine such as PowerShell commands and stand-alone executables, and some techniques
require interaction with the attacking virtual machine such as network connected soft-
ware exploits, Command and Control (C2), lateral movement, and credential access. At
a high-level the attack technique automation script executes on the victim virtual ma-
45
chine, iterates through all technique folders, and executes individual techniques either
locally on the victim virtual machine or reaches out to the Kali virtual machine using
the plink.exe SSH client [36] and executes techniques within the Metasploit Framework
(MSF) [37]. Table 3.2 details how the automation script functions for each technique in
the chosen attack chain.
Table 3.2: Attack Technique Automation Details

Exp. #
Attack Technique Automation Notes
1 Init. Access - Vulnserver Script runs local vulnserver exe locally, then reaches out to Kali using plink.exe
Exploit SSH client to execute python exploit script
2 Init.Access - Macro Word Script runs Microsoft Word and opens document locally
Doc
3 Execution - PowerShell Script runs PowerShell command contained in text file locally
Command
4 Execution - Simulated Script runs custom binary locally that executes MessageBoxA shellcode
Malware Binary
5 Persistence - Windows Script reaches out to Kali, executes MSF psexec module to establish C2, runs
Service exploit/windows/local/persistence service module to install new service that
executes a simulated malware binary
6 Persistence - Registry Script reaches out to Kali, executes MSF psexec module to establish C2, runs
Run Key exploit/windows/local/persistence module to install new service that executes
a simulated malware binary
7 Privilege Escalation Privilege escalation occurs as part of experiment 1, so no technique execution
needed
8 Credential Access - Script reaches out to Kali, executes MSF psexec module to establish C2, loads
LSASS Mimikatz into the process that has the C2 channel, and extracts credentials
out of LSASS
9 Credential Access - SAM Script reaches out to Kali, executes MSF psexec module to establish C2, loads
Hashes Mimikatz into the process that has the C2 channel, and extracts local password
hashes out of the operating system
10 Lateral Movement - SMB Script reaches out to Kali, executes MSF psexec module which connects in-
Psexec bound to the victim over SMB to the ADMIN$ share, creates a Windows
service, and executes a PowerShell Meterpreter C2 payload
11 C2 - HTTP Script reaches out to Kali, executes the payload handler module for the Meter-
preter HTTP payload, then executes local Meterpreter HTTP binary generated
during setup
12 C2 - HTTPS Script reaches out to Kali, executes the payload handler module for the Me-
terpreter HTTPS payload, then executes local Meterpreter HTTPS binary
generated during setup
At the time of this writing, the Automated Attack Technique Execution script is
closed-source, however, the intent is to open-source this script and the framework to
execute techniques in an automated way once an export-control review is completed.
Upon completion of export-control review, the script will be available from the Georgia
46
Tech Research Institute (GTRI) Github page [28]. As stated before, the intent of the
Automated Attack Technique Execution script is to be able to execute an entire body of
techniques across various MITRE ATT&CK framework categories or be able to execute
specific techniques within a category by executing a single PowerShell script. Figure 3.6
illustrates the Automated Attack Technique Execution script at a high-level along with
all of the internal functions that are executed during the process.
Within the self-contained PowerShell script named Attack-Techniques.ps1, there are
three core functions to execute these processes. The functions are named Setup-AttackTechniques,
Store-Artifacts, and Run-AllTechniques and are executed in that order. The setup func-
tion will prepare the system for executing the attack techniques. This includes setting
all IP addresses in configuration files and exploit scripts, generating specific C2 payloads
based on environment properties, downloading resultant payload executables, etc. As a
result of the setup process, many artifacts are also generated that can be used for searching
in collected ETW event data later. This includes IP addresses, TCP port numbers, pro-
cess names, etc. The Store-Artifacts function iterates through all of the technique folders
and extracts these artifacts and stores them in a file for reference later during searching.
The final function, Run-AllTechniques iteratively executes each technique within each
folder sequentially.
47
Figure 3.6: Automated Attack Technique Execution Flow
The script was developed with the capability to perform various setup and execution
functions for each technique folder that is iterated through. To accomplish this, specific
rules were developed that dictate how the user should setup files within the technique fold-
ers so that the script knows how to prepare for attack technique execution. For example,
if a technique is a standalone executable and no interaction with Kali is needed, the file
name should be named with ” local run.exe” appended to the file name. This would tell
the script to execute that executable alone for a period of time before executing the next
48
technique. If a technique is a command that needs to be executed on the victim, a text
file should be created with the command inside the text file and the text file be stored in
the technique folder with ” local exec.txt” appended to the file name. This would cause
the script to read the contents of the text file and execute the specific command within
the text file. If the technique requires interaction with Kali and Metasploit, this is accom-
plished using separate files. One file needs to be a Metasploit Resource Script [38] with
the ”.rc” extension to tell Metasploit which modules to run. For techniques that need
local executables to be executed that then interact with Metasploit modules on Kali, such
as Command and Control (C2), the local executable needs to be generated dynamically
during the setup process. To accomplish this, the user would create a file within the tech-
nique folder name ”c2 http generate.txt” and put the appropriate Msfvenom command
for the payload desired between brackets ”[ ]” and the script will automatically generate
this payload on the chosen Kali system and download the executable for use during the
technique execution process. Figures 3.7 and 3.8 illustrate the overall folder structure
that users work with in creating techniques as well as the process of specifying a dynamic
payload to generate during the setup process.
49
Figure 3.7: Automated Attack Technique Execution Script - Technique File/Folder Struc-
ture
50
Figure 3.8: Automated Attack Technique Execution Script - Dynamic Payload Generation
Setup
Once all of the desired techniques and required files are placed in their respective
folders, the user can then initiate the setup process by executing the automation script
(AttackTechniques.ps1) with a parameter of ”setup attacks”. This parameter will call the
internal function Setup-AttackTechniques(). The script will automatically generate any
required payload executables on the Kali system, download them into their respective
technique folders and upload any necessary files to the Kali system such as Metasploit
Resource Scripts (.rc). Figures 3.9 and 3.10 depict the user interface output that the
script provides during the setup process.
51
Figure 3.9: Automated Attack Technique Execution Script - Setup UI
Figure 3.10: Automated Attack Technique Execution Script - Setup UI
Once the setup function has been executed, attack technique execution can commence.
As stated earlier, users have the ability execute all techniques in sequence, or execute
individual categories of techniques. To accomplish this, the user would simply run the
Automated Attack Technique Execution script (AttackTechniques.ps1) with either the
name of the category of techniques or ”all attacks” as parameters to the script. Figure
52
3.11 depicts the initial options the user has when running the script.
Figure 3.11: Automated Attack Technique Execution Script - Initial Menu
When executing attack techniques with the automation script, the user will be given
output of each technique that is being executed as well as visually see all of the interactions
that are occurring for each technique. For example, if the script is executing a macro-
enabled word document technique, the user will see Microsoft Word launch and any e↵ects
that occur as a result of the macro execution. If a technique requires interaction with
the Kali system for Command and Control (C2) or other techniques, the user will see an
SSH client window open showing the back-end interaction with Kali as well as any local
executions that need to occur to interact with the Kali system (payload executables, etc.).
Figure 3.12 illustrates the attack execution automation script executing and the resultant
interactions that the user sees as a result of executing a specific attack technique.
53
Figure 3.12: Automated Attack Technique Execution Script - C2 Technique Execution
3.2.5 Iterative Analysis of ETW Event Data Against Attack
Techniques
To further accelerate this discovery process of attack technique artifacts within ETW
event data, all of the JSON files generated during the Automated ETW Event Gen-
eration and Automated Attack Technique Execution process were ingested into Splunk
Enterprise located on a di↵erent virtual machine on the same network for quicker search-
54
ing and identification of attack technique artifacts. Splunk Enterprise is a data ingestion,
aggregation, and searching tool used by many organizations for security analytics and
dashboarding [18]. The overarching workflow for this discovery research once all the au-
tomation scripts were finalized involved executing the automated ETW event generation
script a single time on the victim virtual machine, which also executed the automated
attack technique execution script. Once the automated ETW event generation scripted
finished iterating through all of the available ETW providers on the victim virtual ma-
chine, all of the logs contained in the CollectedLogs directory were then ingested into the
Splunk Enterprise virtual machine for analysis. As noted in Figure 3.6, the automated
attack technique execution script also has a built-in feature to extract artifacts that can
be used for searching within Splunk such as binary file names, port numbers for network
connections, Uniform Resource Identifiers (URI) for HTTP connections, etc. Examples
of the artifacts extracted for some of the techniques are depicted in Figure 3.13.
Figure 3.13: Automated Attack Technique Artifacts
3.2.5.1 Attack Technique Artifact Identification in ETW Event Data
Section 3.2.2 details that the primary problem this initial discovery/background research
was trying to solve was to identify the answers to the questions ”which ETW provider
is the most appropriate for detecting a specific attack technique or sub-technique?” and
55
”which event names and field names are the most appropriate for detecting a specific
attack technique?” Answers to these questions would immediately shape and inform how
the User-Defined Context configuration file would be structured and used by users to
detect the techniques in the attack chain. The following sections detail the process taken
to identify artifacts for each attack technique. All techniques that were identified in ETW
event data were identified either through ETW event data located in Splunk as a result of
the Automated ETW Event Generation script process or through manual experimentation
with ETW event data using tools such as SilkETW.
3.2.5.2 Technique Identification - Initial Access - Exploit Network Facing

Application (T1190.000)
As noted in Table 3.1, the first attack technique in the chain is a software exploit against
the popular test software Vulnserver [35]. Vulnserver is an open-source project of network
socket software that is designed to be vulnerable to a number of typical memory corruption
software vulnerabilities such as bu↵er overflows. The entire application is written in C
and is compiled with no compile-time protection mechanisms, such as, memory address
randomization, bu↵er overflow protections, etc. that are o↵ered by languages such as
C++. This provides a very representative example attack surface as the author can
confirm through professional experience in penetration testing the existence of C-based
legacy applications with similar vulnerabilities installed in many organizations across
many industries. For the purposes of the attack chain in this research, Vulnserver is
listening for network connection on the victim virtual machine through TCP port 9999
and waits for interaction from users across network sockets. The attacking virtual machine
in the Automated Attack Technique automation script executes a python script that
connects to TCP port 9999 on the victim virtual machine, sends more bytes than the
network socket bu↵er is allocated, triggers a bu↵er overflow, redirects execution flow by
injecting a known-memory address of a JMP ESP instruction, and executes shellcode
56
embedded in the bu↵er that was sent.
From a technique identification perspective, this particular technique does not provide
many opportunities for confirming that a software exploit has occurred using ETW event
data. There are no standard ETW event providers that provide visibility into bu↵er allo-
cations, memory write operations, program execution, etc. However, this does not mean
that detection of software exploitation scenarios is impossible. Upstream and downstream
techniques to the primary technique can be used as detection that an abnormal behavior
for the specific software of concern is occurring. Table 3.3 illustrates all of the atomic
techniques that occur within the software exploit experiment and whether these atomic
techniques are detectable using ETW event data.
Table 3.3: Experiment 1 - Software Exploit Atomic Techniques

Atomic Attack Technique
Detectable Using ETW ETW Provider - EventName(s) - Field
Event Data? Name(s)
Inbound network connection to Yes - Check if source IP is normal Kernel Network TCPIP - TcpIp/Accept -
9999 saddr
Socket writes to memory bu↵er No N/A
EIP overwrite, execution flow de- No N/A
viates
Shellcode begins execution No N/A
Actions of shellcode (C2, etc.) *Yes - *Each action may or may Depends on the action taken
not
While the majority of techniques within the software exploit are not detectable using
ETW event data, identifying which systems should be allowed to connect to the software
over the network provides the greatest defense capability. For example, if a particular
system only ever needs inbound connections from specific IP addresses, alerting on any
deviations from this allowed list should be able to be accomplished using the User-Defined
Context file. Using the ETW event logs collected during the background research, network
connections inbound to the Vulnserver process were most reliably found within the kernel
ETW provider NetworkTCPIP within the event name ”TcpIp/Accept”. Figure 3.14 illus-
trates identifying the attacking IP within a kernel TcpIp/Accept event that could reliably
57
be used in the User-Defined Context file for detection.
Figure 3.14: Identifying Malicious Connections to Vulnserver in ETW Event Data
A small distinction was identified during the evaluation of the event data presented in
Figure 3.13. The event was generated after the attacking IP connected to the Vulnserver
port and began exploitation. Additionally, the EventName indicates that the event refers
to acceptance of a TCP connection, however, the source IP, destination IP, source port,
and destination port are reversed from the typical interpretation. The attacking IP of
”172.16.255.139” appears in the ”daddr” field indicating ”destination address.” While this
appears reversed, this event is likely being generated after a TCP three-way handshake is
complete, and therefore, the event is generated for the response back to the originating IP.
The response to the connection back to the originating IP would cause the IP address and
58
port numbers to be reversed from their normal source/destination interpretation. This
does not invalidate the detection of the activity as it is still indicative of a connection to
the concerning port number in the first place since responses to connections will typically
have the original inbound destination port (9999) as the source port.
3.2.5.3 Technique Identification - Initial Access - Spear-Phishing Attachment

- Word Document Macro (T1566.001)
The second attack technique experiment chosen for validation of the artifact in this re-
search was a macro-enabled Word document being used under the guise of a notional
spear-phishing scenario. Spear-phishing and malicious attachments are typically a pri-
mary attack vector for many cyber threat groups due to the success of these vectors and
the strategic position that having control of endpoints inside an organization provides an
attacker. From a current threat landscape perspective, one of the most prolific malware
campaigns that uses malicious Office documents within phishing emails is called Emotet.
Fortinet indicates [39] that the Emotet malware campaign has infected more than one
million devices worldwide throughout the campaign’s existence. Additionally, the author
can confirm from professional experience in penetration testing, red teaming, and cyber
threat emulation engagements across many industries that spear-phishing and malicious
Office documents have proven to be very successful attack vectors that resulted in the
most impactful compromises in organizational assessments.
The overarching flow of this attack technique starts with Microsoft Word launching a
custom-made Word document that contains an executable embedded as hexidecimal code
within the document contents. The Word document launches a VB script macro that is
embedded into the document upon launch. The macro is programmed to iterate through
each paragraph in the document, copy the bytes for each paragraph, write the copied bytes
into a executable file in the directory C:\Windows\Temp\malware macro.exe, and then
launch this executable. Figure 3.15 illustrates how the malware macro.exe executable is
59
embedded into the Word document for the macro to extract.
Figure 3.15: Executable Embedded in Word Document
The executable itself then executes Windows API MessageBoxA() shellcode to spawn
a message box. It should be noted that for the purposes of attack simulation in this exper-
iment, a number of configurations and circumstances were pre-built into the experiment
to enable the attack technique to succeed. For example, macro execution was allowed to
execute upon opening the document simulating a user was socially engineered to enable
macro execution. This also enabled the automation of the attack technique to succeed
without user interaction. From a technique identification perspective, there are numerous
artifacts available for detecting that a macro-enabled Word document has been opened
on an endpoint. Table 3.4 illustrates all of the atomic techniques for this experiment and
the identified ETW event data required to detect them.
60
Table 3.4: Experiment 2 - Macro-Enabled Word Document Atomic Techniques
Event Data? Name(s)
Launch WINWORD.exe Yes - Check if username is normal Kernel Process - Process/Start - Process-
Name, Username
WINWORD opens macro- Yes - Check Word document path Kernel Process - Process/Start - Process-
enabled Word document Name, CmdLine
Macro executes and iterates No N/A
through paragraphs
Macro writes bytes from para- Yes - Check WINWORD writes Microsoft-Windows-FileInfoMinifilter -
graphs into EXE file EXE write to suspicious path fi:FileNameCreate
Macro executes EXE file No N/A
in C:\Windows\Temp with
ShellExecute()
Malware macro.exe launched Yes - Check EXE launched by Yes - Check EXE launched by WINWORD
by WINWORD from WINWORD
C:\Windows\Temp
The primary detective capability for the majority of the atomic techniques in this
experiment lies in the kernel Process ETW provider and Process/Start EventName. Event
data within these events provides a significant amount of information that can be used
to deduce numerous facts and circumstances about a process’ launch. Depending on
the system’s defined purpose and expected behavior, these facts and circumstances can
be interpreted in such a way that clear distinction between normal and abnormal can
be achieved. For example, Figure 3.16 illustrates a process start event for this attack
technique where identification of where the Word document was located on the endpoint,
which username opened the word document, and the fact that the Word document is
macro-enabled (.docm) can be identified. Figure 3.17 illustrates a FileInfoMiniFilter event
that WINWORD.exe wrote an executable file to a temporary directory. Figure 3.18 finally
illustrates WINWORD.exe launching that same executable file that was written by Word
as a child process. These three events occurring in a sequence allows a defensive operator
to be able to deduce that the Word document likely contained a macro and the macro
was executed to completion with a final stage malware executing at the end.
61
Figure 3.16: Identifying Macro-Enabled Word Document Launch in ETW Event Data
62
Figure 3.17: Identifying Word Document Writing EXE to Disk in ETW Event Data
63
Figure 3.18: Identifying EXE Being Launched by Word in ETW Event Data
3.2.5.4 Technique Identification - Execution - Command/Script Interpreter

- PowerShell (T1059.001)
The third technique chosen for validation moves on to the Execution category of the
MITRE ATT&CK framework. This category represents techniques that typically happen
when an attacker has already gained access to an endpoint and needs to execute addi-
tional code or malware that enables them greater access to the system. For the purposes
of validation of the developed artifact in this research, the first Execution category tech-
nique chosen was a PowerShell command. PowerShell has been a very powerful scripting
language and command execution capability for system administrators on Windows sys-
64
tems since Windows XP SP2 and Windows Vista. From a threat landscape perspective
PowerShell has been used by many high-profile cyber threat groups due to the inherent
capabilities that PowerShell provides without the need to install additional tools or mal-
ware to achieve their objectives. Dunwoody illustrates [40] that Russian cyber threat
group APT29 uses PowerShell commands as a ”living o↵ the land” technique to appear
legitimate on the systems they are controlling. The particular backdoor used in the cam-
paign Dunwoody describes [40] is named ”POSHSPY” and uses a series of techniques
to execute an encoded PowerShell command persistently on compromised systems. This
attack campaign served as appropriate influence for the type of PowerShell command
executed for this research.
The experiment for this research uses a single PowerShell command that contains
additional PowerShell commands encoded within a base64 encoded string. The Power-
Shell command was generated from msfvenom [41], a payload generation tool built into
Kali Linux and the Metasploit Framework, on the Kali virtual machine and uses the
windows/meterpreter/reverse tcp payload with a destination host of the Kali IP address.
The command attempts to iteratively decode an underlying set of PowerShell commands
that load necessary Windows DLLs into the powershell.exe process, call various Windows
API functions to load embedded Meterpreter C2 code, and finally execute the loaded Me-
terpreter C2 code that will reach out to the Kali virtual machine IP address and establish
a Meterpreter TCP C2 channel with the victim virtual machine. Figure 3.19 illustrates
what the PowerShell command looks like from an attacker perspective when executing the
command on a compromised endpoint. Table 3.5 details all of the atomic techniques that
occur as a result of executing the single PowerShell command as well as which techniques
are detectable using ETW event data.
65
Figure 3.19: Base64 Encoded PowerShell Command to Execute Meterpreter TCP C2
Channel
Table 3.5: Experiment 3 - PowerShell Command Atomic Techniques

Atomic Attack Technique Detectable Using ETW ETW Provider - EventName(s) - Field
Event Data? Name(s)
Powershell.exe is launched Yes - Check for PowershellCon- Kernel Process - Process/Start OR
soleStartup event or process start Microsoft-Windows-PowerShell - Power-
shellConsoleStartup
PowerShell interpreter decodes Yes - Check for ”Sys- Microsoft-Windows-PowerShell - any event -
encoded commands tem.Convert::FromBase64String” System.Convert::FromBase64String
PowerShell calls WinAPI to allo- Yes - Check for ”kernel32.dll Vir- Microsoft-Windows-PowerShell - any event -
cate memory (VirtualAlloc) tualAlloc” kernel32.dll VirtualAlloc
PowerShell copies payload bytes Yes - Check for ”Marshal]::Copy” Microsoft-Windows-PowerShell - any event -
into allocated memory Marshal]::Copy
PowerShell calls WinAPI to set Yes - Check for ”kernel32.dll Vir- Microsoft-Windows-PowerShell - any event -
protection on memory (Virtual- tualProtect” kernel32.dll VirtualProtect
Protect)
PowerShell calls WinAPI to exe- Yes - Check for ”kernel32.dll Cre- Microsoft-Windows-PowerShell - any event -
cute thread of copied code (Cre- ateThread kernel32.dll CreateThread
ateThread)
Command and Control (C2) con- Yes - Check process name and Kernel NetworkTCPIP - TcpIp/Send - Pro-
nection is made from Power- destination IP cessName, daddr
shell.exe
From a technique identification perspective, there are a significant amount of artifacts

available for detecting various behaviors of PowerShell commands as PowerShell com-
mands are executed through the PowerShell interpreter in Windows. All actions executed
through the Windows PowerShell interpreter pipeline are logged within the Microsoft-
66
Windows-PowerShell ETW provider. Additionally, if there are encoded commands nested
within encoded commands in a single PowerShell command, the interpreter will log the
commands being executed in their encoded state as well as their decoded state itera-
tively until all encoding is decoded. Within the Microsoft-Windows-PowerShell ETW
provider, there are numerous events with di↵erent EventNames that occur when a user
opens a PowerShell command prompt and executes a command. Figure 3.20 illustrates
the various EventNames that occurred when the command chosen for Experiment #3 was
executed. Each EventName has a di↵erent logging purpose as well as di↵erent information
that can be used for detection of the technique. For example, to detect basic behaviors
such as when a specific username opens a PowerShell prompt, the events within the
EventName ”PowerShellConsoleStartup/Startup” can be analyzed. These events contain
no command-level information. To detect more advanced behaviors within a PowerShell
command, such as which Windows APIs were called, events within the EventName ”Ex-
ecuteaRemoteCommand/OnCreateCalls” could be analyzed.
Figure 3.20: Microsoft-Windows-PowerShell ETW Provider EventNames for Experiment

#3
67
While defenders could simply alert on the execution of an encoded PowerShell com-
mand as many commercial EDR solutions do, doing so does not provide enough context
on the extent of malicious behavior the encoded command executed on the endpoint.
To analyze the initial encoded PowerShell command more in depth, careful inspection
of events within the EventName ”ExecutaRemoteCommand/OnCreateCalls” is needed.
Figure 3.21 illustrates an example ETW event after Experiment #3 was executed. The
PowerShell interpreter decodes the original encoded command and logs the decoded con-
tents within this event. Numerous artifacts exist within this event to identify what
type of actions the PowerShell command executed. For example, the existence of the
strings ”System.IO.Compression.GZipStream”, ”System.IO.MemoryStream”, and ”[Sys-
tem.Convert]::FromBase64String” together in the same script block indicate an intent to
decompress Base64 encoded code into memory to be executed later.
68
Figure 3.21: ETW Event Showing First Level Decoding of PowerShell Command to Ex-
ecute Meterpreter TCP C2 Channel
Since the PowerShell interpreter logs all commands and script blocks that are executed
regardless of how many levels of nesting they are within, the final actions of the PowerShell
script can be analyzed using events with the same ”ExecutaRemoteCommand/OnCreate-
Calls” EventName. Figures 3.22 and 3.23 illustrate the fully decoded script blocks that
69
load the ability to call native Windows API functions, allocate memory with VirtualAl-
loc(), copy the Meterpreter C2 payload into the allocated memory, and create a thread
to execute the Meterpreter payload. From a defensive perspective, having these details
within an ETW event provides a significant amount of context on whether a PowerShell
command is malicious or not. Depending on the system that this event occurs on, the
existence of this activity may immediately indicate malicious activity. For example, in a
standard organization, workstations that are not IT workstations typically never execute
PowerShell commands that decode code, allocate memory, copy decoded code, and create
threads of written code to memory. All of these artifacts are the types of artifacts that
this research sought to make configurable in the User-Defined Context file and able to be
alerted on with the developed software artifact.
Figure 3.22: ETW Event Showing PowerShell Code to Load the Ability to Call Windows
API Functions
Figure 3.23: ETW Event Showing PowerShell Code to Allocate Memory, Copy Payload
into Memory, and Create a Thread to Execute Copied Payload
3.2.5.5 Technique Identification - Execution - User Execution - Malicious

File (T1204.002)
The fourth experiment to be used in this research for validation of the developed soft-
ware artifact was simulating a malware executable that was downloaded by a user from
70
a spear-phishing scenario. Malware files can include an enormous amount of di↵erent
capabilities, therefore, the goal of this technique was to exercise a select few techniques
that are inherently difficult to detect to identify whether any of the behaviors can be
detected with ETW event data. One of the most difficult techniques to detect using any
security product is allocation and writing bytes to memory within the same process. The
reason is because these techniques are techniques that software developers use for legit-
imate functionality in legitimate applications making it difficult for endpoint solutions
to identify malicious memory allocations from legitimate ones. For Experiment #4, a
simulated malware executable was created in C++ that exercises the core behaviors of
allocating memory within its own process memory space, writing embedded shellcode into
the allocated memory, and then executing the shellcode that was written. The shellcode
chosen to execute is a set of simple No Operation (NOP) instructions followed by a re-
turn (RET) instruction. While this shellcode is simple in operation, any x86 shellcode
could be used and the goal for this experiment was to identify whether any of the internal
process memory/execution operations could be detected using ETW event data. Figure
3.24 illustrates the source code that was used for the simulated malware executable in
this experiment. Table 3.6 illustrates the atomic techniques identified for this experiment
and whether they are detectable using ETW event data.
71
Figure 3.24: Source Code of Simulated Malware Executable
Table 3.6: Experiment 4 - Simulated Malware Executable Atomic Techniques

Event Data? Name(s)
Simulated malware executable is Yes - Check various properties if Kernel Process - Process/Start - Process-
launched normal/concerning Name, Username, Path, etc.
VirtualAlloc() is called to al- Yes* *Kernel VirtualAlloc events are too volumi-
located memory inside process nous and
memory
Memcpy is called to copy embed- No N/A
ded shellcode bytes to allocated
memory
Call start of shellcode by function No N/A
pointer
NOP + RET shellcode execution No N/A
From a technique identification perspective, very few of the internal techniques exe-
cuted by the simulated malware file were able to be detected using ETW event data. This
is primarily caused by the fact that no ETW providers exist for logging internal process
memory operations. However, this does not mean that malware files cannot be detected
72
on endpoints. Using creative defensive logic, many other pieces of information can be used
within kernel Process/Start events to identify the introduction of suspicious executables
to the system. For example, if the system being monitored has a baseline established such
that all executables that should exist on the system were already known and establish
a policy that no additional executables should be installed on the system under normal
operation, the introduction of additional executables after this baseline is immediately
noteworthy to defenders. Alerting on the introduction of executable files onto a system
does not confirm that they are malware, however, supporting facts/contexts about the
system can aid a defender in identifying activity that is abnormal for that system. For
example, using the baseline of known executables on a system can be used in conjunction
with the username that launched it, the path that it was launched from, as well as the
parent process that launched it from kernel Process/Start event data, defenders could
achieve a high-degree of certainty that the executable that was introduced presents a
threat to respond to.
Additionally, if a defender is also consuming event data from the Microsoft-Windows-
FileInfoMinifilter provider, they can also identify which process downloaded the file to
determine where the file came from. Figures 3.25 and 3.26 illustrate identifying artifacts
from both the download of the executable file as well as the launching of the executable.
Using all of this data together from two di↵erent ETW providers allows defenders to
conclude that a user likely used Microsoft Edge to download an executable file that was
previously not on the system and then execute that file. No ETW event data is available
to determine what the executable did after launch, however, the information achieved
from the data available is enough to make an informed decision about a response action.
Additionally, while this particular experiment highlights the worst-case scenario that a
defender loses visibility into malicious activity, this does not mean that every malware
executable will exhibit only these actions. Malware executables typically execute many
techniques across the MITRE ATT&CK framework, which may be detectable using other
73
ETW providers as this research has identified.
Figure 3.25: ETW Event Showing Malware File was Downloaded by Microsoft Edge
74
Figure 3.26: ETW Event Showing Malware File was Launched by User in Windows File
Explorer
3.2.5.6 Technique Identification - Persistence - Create/Mod Windows Ser-

vice (T1543.003)
The fifth experiment used for validation of the software artifact in this research moves to
the persistence category of the MITRE ATT&CK framework. Persistence techniques are
techniques that attackers use to maintain access to systems that they have compromised
in the event connection to those systems is disconnected from system restarts, network
connectivity problems, etc. This technique was chosen for validation because of the preva-
lent use of the technique in high-profile cyber-attacks by well-known cyber threat groups.
For example, the Cybersecurity Infrastructure and Security Agency (CISA) reported [42]
in 2020 that a North Korean intelligence service controlled hacker group named Beagle-
Boyz used Windows services as a persistence mechanism in many of the group’s known
75
attack campaigns. To simulate the e↵ects of creating a Windows service that persistently
executes a malware executable, the Metasploit Framework was used by the Automated At-
tack Technique execution script to execute the ”windows/local/persistence service” mod-
ule. Table 3.7 illustrates the atomic techniques that are executed as part of executing this
technique within the Metasploit Framework.
Table 3.7: Experiment 5 - Persistent Windows Service Atomic Techniques

Event Data? Name(s)
Attacker process writes ser- Yes - Check if executable/path is Microsoft-Windows-FileInfoMinifilter -
vice/payload executable to known, etc. fi:FileNameCreate
C:\Windows\Temp\
Payload/service executable Yes - Check if executable is Kernel Process - Process/Start - Process-
launches known, etc. Name, Username, etc.
Windows service is created by *Yes - API call not detectable, Microsoft-Windows-Services - ServiceSta-
calling CreateService() API func- service creation is tusChange/StatusChange - ServiceName
tion
New C2 channel is opened from Yes - Check if destination IP/port Kernel NetworkTCPIP - TcpIp/Send -
payload executable is allowed daddr, dport, ProcessName, etc.
From a technique identification perspective, the most appropriate ETW provider to de-
tect additions/changes to Windows services is the ”Microsoft-Windows-Services” provider.
Endpoints in enterprise environments typically experience very little changes to Windows
services throughout normal operations, therefore, this ETW provider provides sufficient
fidelity for detecting that a new service has been installed on a system. For positive
identification of this technique, three fields within ETW event data need to be analyzed.
Figure 3.27 illustrates that the most appropriate event name for service creations is ”Ser-
viceStatusChange/StatusChange” located in the EventName field. Additionally, the field
XmlEventData.ImageName lists any executable that is executed by the service when the
service starts. This field is typically where the attacker’s executable will be listed along
with the full file path. Finally, the field ServiceName lists the officially registered ser-
vice name. Similar to the malware executable experiment, a static list of known service
names can be used to identify services that are not known to be on the system. Since
Windows service names are typically static and cannot be duplicative on systems, using a
76
static list of known services will provide a reliable way to detect deviations from normal
expectations.
Figure 3.27: ETW Event Showing a Windows Service was Installed to Execute Malware
Persistently
77
3.2.5.7 Technique Identification - Persistence - Autostart/Exec Registry Run
Key (T1547.001)
The sixth experiment used for validation of the software artifact in this research executes
another typical technique for persistence. In Windows operating systems the Windows
Registry is a central hierarchical database to store information for applications and ser-
vices to use [43]. Within the Windows registry exists specific registry keys named Run
and RunOnce that are designed to make a program launch whenever a user logs in or the
system is booted [44]. From an attacker perspective this is advantageous for maintaining
persistence on a system if an existing Command and Control (C2) channel is lost at some
point during the campaign. To accomplish this, an attacker simply needs to modify the
Run/RunOnce key to include an executable that they want to launch either during user
login or system boot. The only prerequisite is that the process the attacker has control
of needs to be running with the right permission level to edit keys within the top-level
registry key that is needed.
For program execution upon user login, the Run/RunOnce key needs to be set within
the top-level key HKEY CURRENT USER and current running user permissions are
required. For program execution upon system boot, the Run/RunOnce key needs to be set
within the top-level key HKEY LOCAL MACHINE and admin/system level permissions
are required. To simulate all of the registry Run key persistence e↵ects in the experiment
for this research, the Metasploit module ”windows/local/persistence” was used to exercise
all of the components of the registry persistence technique. Figures 3.28 and 3.29 illustrate
the Metasploit module execution as well as the resultant artifacts in the registry and file
system that are produced to achieve registry Run key persistence.
Figure 3.28: Metasploit Registry Persistence Module Execution
78
Figure 3.29: Results of Metasploit Registry Persistence Module Execution
As with the other techniques executed in the validation experiments, this technique has
additional sub-techniques that are executed as part of the main technique. Registry Run
keys allow users to input any application that they wish to execute when the predefined
conditions occur (user login/system boot). While the Metasploit module executed in this
experiment uses a VB script file in the C:\Windows\Temp directory to execute the final
Meterpreter payload, writing executable files to disk and referencing them in the registry
Run key is not the only possible technique to achieve persistence with the registry Run
key. For example, attackers could input self-contained encoded PowerShell commands in
the registry Run key and achieve the same level of persistence without writing a payload
executable to a directory. Table 3.8 details all of the sub-techniques that are executed as
part of this experiment and whether they can be detected with ETW event data.
Table 3.8: Experiment 6 - Persistent Registry Run Key Atomic Techniques

Atomic Attack Techniques Detectable Using ETW ETW Provider - Event Name(s) - Field
Event Data Name(s)
Attacker process writes payload Yes - Check process name, file Microsoft-Windows-FileInfoMinifilter -
file to C:\Windows\Temp name, path, etc. for allowed fi:FileNameCreate
Attacker process sets value for Yes - Check key name that is Kernel Registry - Registry/Open - KeyName
HKLM\Software\..\..\..\Run opened
From a technique identification perspective, the most appropriate ETW providers to

detect the techniques for this experiment are the Microsoft-Windows-FileInfoMinifilter
79
provider for the payload write to disk and the kernel Registry provider for the interaction
with the registry Run key. Figure 3.30 details the fields located within a ”Registry/Open”
event in the kernel registry provider that are necessary to be analyzed to positively identify
this technique. Defensive alerting can be made on specific processes that open the registry
Run key or any processes that open this key if this key is not often modified on the system.
Figure 3.30: ETW Event Showing an Attacker Process Accessing Registry Run Key
3.2.5.8 Technique Identification - Credential Access - OS Dumping Creden-

tial LSASS (T1003.001)
The eighth experiment used for validation of the software artifact in this research moves
to the Credential Access category of the MITRE ATT&CK framework. This category
represents some of the most impactful techniques for an attacker as gaining access to
credentials often dictates how quickly an attacker can move laterally to other systems
within an organization. The specific credential access technique chosen for this exper-
iment was dumping/scraping cached credentials from operating system memory in the
LSASS.exe process. Mandiant indicates [45] in a report about APT1 affiliated with the
80
Chinese People’s Liberation Army (PLA) that this cyber threat group has used many
tools to execute LSASS scraping credential access techniques. One of the tools used by
this group is Mimikatz, an open-source computer security tool. For experiment #8 in
this research, Mimikatz is also used to simulate the same credential access technique used
by APT1. The overarching process for executing an LSASS credential access technique
using Mimikatz on an endpoint involves gaining access to an endpoint at the system
privilege level (process running as NT AUTHORITY\SYSTEM), loading Mimikatz code
within the attacker-controlled process, and executing one of several built-in functions
in the Mimikatz code to read credentials out of the LSASS.exe process by calling the
OpenProcess() and ReadProcessMemory() Windows API functions. For the experiment
in this research, Metasploit and Meterpreter C2 were the platform and C2 channel to
load Mimikatz and execute the read process. This represents the most stealthy method
of executing the Mimikatz/LSASS credential access technique as no executable files are
placed on the disk of the victim virtual machine and all scraping occurs through Windows
API calls in memory. Additionally, using Metasploit and Meterpreter for the platform
and initial access channel are comparable tools/techniques to those used by real cyber
threat groups.
The process taken to execute this technique in this experiment was to establish a
high-privilege Meterpreter C2 session using a lateral movement technique (Metasploit
psexec module) and a simulated stolen administrative credential, load Mimikatz into
the established Meterpreter session, and call the ”creds all” Mimikatz function to read
credentials from the LSASS process. Table 3.9 illustrates the atomic techniques executed
in this validation experiment and the relevant ETW event data that can be used to detect
it.
81
Table 3.9: Experiment 8 - Credential Access (LSASS) Atomic Techniques
Atomic Attack Techniques Detectable Using ETW ETW Provider - Event Name(s) - Field
Event Data Name(s)
Attacker process loads vaultcli.dll Yes - Check image name for vault- Kernel ImageLoad - Image/Load - Image-
into process for Mimikatz to uti- cli.dll Name
lize
Attacker process loads samlib.dll Yes - Check image name for sam- Kernel ImageLoad - Image/Load - Image-
into process for Mimikatz to uti- lib.dll Name
lize
Attacker process loads crypt- Yes - Check image name for Kernel ImageLoad - Image/Load - Image-
dll.dll into process for Mimikatz cryptdll.dll Name
to utilize
Attacker process calls OpenPro- Yes - Check process name, Tar- Microsoft-Windows-Kernel-Audit-Api-Calls
cess() WinAPI function with getProcessId for allowed - EventId(5) - TargetProcessId
LSASS.exe as the target
From a technique identification perspective, inspecting Windows API function calls

is typically not possible using solely ETW event data as there are no providers available
for such function calls. One of the exceptions is in the Microsoft-Windows-Kernel-Audit-
Api-Calls provider that provides logging for calls to certain important kernel-level func-
tions. Official documentation for the specific functions that are logged within this ETW
provider do not exist, however, security researchers have reverse engineered many of the
ETW providers to identify their capabilities. Damari has performed [46] reverse engineer-
ing of the Microsoft-Windows-Kernel-Audit-Api-Calls ETW provider and relevant kernel
symbols within the Windows kernel image (ntoskrnl.exe) to identify that this provider
produces events for the Windows API functions depicted in Table 3.10 and Figure 3.31.
82
Table 3.10: Microsoft-Windows-Kernel-Audit-Api-Calls ETW Provider - Au-
dited Functions
Microsoft-Windows-Kernel-Audit- EventName in ETW Events

Api-Calls Functions
PsSetLoadImageNotifyRoutine() EventId(1)
TerminateProcess() EventId(2)
CreateSymbolicLinkObject() EventId(3)
SetContextThread() EventId(4)
OpenProcess() EventId(5)
OpenThread() EventId(6)
IoRegisterLastChanceShutdownNotification() EventId(7)
IoRegisterShutdownNotification() EventId(8)
83
Figure 3.31: Reverse Engineering of Kernel Function Audit Event Properties [46]
Inspecting events from the Microsoft-Windows-Kernel-Audit-Api-Calls ETW provider

during the experiment confirmed that calls to the OpenProcess() API function are logged
within this provider and have the EventName of ”EventId(5)” within the event data.
Additionally, events for EventId(5) include the level of access that was requested (hex-
adecimal bit mask of desired permissions [47]), a return value on the success or failure
of the call, as well as the target process ID that was accessed. Figure 3.32 illustrates all
of the fields that can be used to detect malicious read operations to the LSASS process.
Using all of these fields, defenders can reach a reasonably high certainty that a process
read contents from the LSASS process. While this is not definitive that a credential access
84
technique was executed, read operations to the LSASS process only typically occur from
specific processes, have other desired access values, etc., therefore, this baseline can be
used to identify abnormal reads from normal reads.
Figure 3.32: ETW Event Showing Evidence of PowerShell Reading Memory from LSASS
3.2.5.9 Technique Identification - Credential Access - OS Credential Dump-

ing SAM (T1003.002)
The ninth experiment used for validation of the software artifact in this research stays in
the credential access category of the MITRE ATT&CK framework but moves to dump-
ing password hashes out of the Security Accounts Manager (SAM) database file. Every
Windows system maintains a SAM file to store password hashes of local user accounts on
the system. Accessing password hashes of local user accounts on a system is significant
for an attacker because Windows allows authentication over the network using password
hashes for accounts as opposed to using plain-text credentials. In the event that an at-
tacker is not able to execute an LSASS read-based credential access technique but is able
to access password hashes on the local system, there is a chance that these local account
password hashes (example: built-in Administrator account in Windows) are configured
85
elsewhere in an organization and can be used to authenticate to other systems with a
lateral movement technique. This technique of using password hashes to move laterally
is called ”Pass-the-Hash”, and is illustrated in a document by Microsoft [48] on how to
mitigate the e↵ects of Pass-the-Hash attacks.
For the validation experiment in this research, the specific hash extraction technique
chosen was extracting cached password hashes from the Windows registry. Mandiant
indicated [45] in the China PLA APT1 report that this threat actor is known to use
a tool called ”cachedump” to extract local account password hashes from the Windows
registry. Cachedump, along with other password hash dumping tools, utilize a complex
method to decrypt password hashes located in the registry because the SAM database
file that stores password hashes is locked by the operating system once the system boots.
The typical location of the SAM database is in C:\Windows\System32\Config on disk
and HKLM\SAM in the registry. Security researcher tjil illustrated [49] in a blog post
the process that is taken to extract local account password hashes from the registry af-
ter Windows 10 build 1607. The process involves extracting AES encrypted password
hashes from specific registry keys and decrypting the password hashes with AES, fol-
lowed by decrypting them with DES. Similar to the LSASS credential access experiment,
this experiment utilizes an existing high-privilege Meterpreter C2 session to execute the
”post/windows/gather/smart hashdump” Metasploit module which uses the same tech-
niques to read password hashes from the registry as the tool cachedump. The process of
extracting password hashes from the registry and all the required decryption processes is
illustrated in Figure 3.33.
86
Figure 3.33: ETW Event Count for Kernel Registry Provider
From a technique identification perspective, the most appropriate ETW provider

for detecting possible hash retrieval techniques from the registry is the kernel Registry
provider. As shown in Figure 3.33, there are numerous interactions with di↵erent reg-
istry keys during the process of retrieving the encrypted hashes and supporting encryp-
tion information. However, the most identifiable registry key that can be used to iden-
tify a process is attempting to access local account password hash information is the
”HKLM\SAM\SAM\Domains\Accounts\Users” registry key. Figure 3.34 illustrates a
Registry/Open event within the kernel Registry ETW provider that identifies a process is
attempting to open the registry key where password hashes are stored. It should be noted
that the process name that is accessing the registry key is listed as ”lsass” in the ETW
event. As stated earlier, the SAM file information is only accessible by the operating
system after the system boots. Although the attacking process (powershell.exe) is run-
ning the hash dump tool to extract the hashes, the interface that is used to perform this
extraction causes the source process to change to ”lsass”, which is a core process for cre-
87
dentials managed by the Windows operating system. As is the case for other techniques,
the supporting indicator that this technique is occurring is the fact that this event does
not happen regularly and therefore a defensive conclusion can be made that the activity
in the event is abnormal for the system.
Figure 3.34: ETW Event Showing Password Hashes Retrieved from the Registry
3.2.5.10 Technique Identification - Lateral Movement - Windows SMB AD-

MIN Shares (T1021.002)
The tenth experiment used for validation of the software artifact in this research moves
to the lateral movement category of the MITRE ATT&CK framework. The lateral move-
ment category encompasses many di↵erent techniques but the core e↵ect of all of the
techniques is that an attacker is able to use an existing compromise point to achieve an
additional compromise point on another system. The specific lateral movement method
chosen for this experiment is using Windows SMB ADMIN file shares to execute a process
88
on a remote system. The most appropriate tool to use to simulate the e↵ects of Windows
SMB-based lateral movement is PsExec [50]. PsExec is a tool designed for system admin-
istrators to remotely execute processes on network-connected Windows systems. From an
attacker perspective, this capability is highly advantageous because it enables the ability
to remotely execute commands on systems across internal networks, but also appear as
though the activity is legitimate because the tool is used by system administrators. The
Cybersecurity and Infrastructure Security Agency (CISA) indicates [51] in alert bulletin
TA17-181A that the highly impactful NotPetya ransomware utilizes PsExec to move lat-
erally within infected environments. For execution in this experiment, the Metasploit
framework includes a module named ”exploit/windows/smb/psexec” that uses all of the
techniques exhibited by the PsExec tool. Figure 3.35 illustrates this module in operation
to achieve Windows SMB-based lateral movement and subsequent Meterpreter C2 chan-
nel establishment on a victim virtual machine. Table 3.11 illustrates all of the atomic
techniques that are executed as part of this process as well as the identified ETW event
data that can be used to detect the techniques.
Figure 3.35: Metasploit Psexec Module Executing Lateral Movement Over SMB
89
Table 3.11: Experiment 10 - SMB Lateral Movement Atomic Techniques
Event Data? Name(s)
Inbound network connection to Yes - Check if source IP is allowed Kernel NetworkTCPIP - TcpIp/Accept -
System process over TCP 445 ProcessName, daddr, sport, etc.
Authentication to SMB server as Yes - Check if username is allowed Microsoft-Windows-SMBServer -
administrator over network Smb2SessionAuthenticated - DomainName,
Username
SMB server receives request to in- Yes - Check if name is svcctl Microsoft-Windows-SMBServer -
teract with Service Control Man- Smb2FileOpen/Start - Name
ager (”svcctl” endpoint)
SMB server receives request to Yes - Check if name is powershell Microsoft-Windows-SMBServer -
open/start Powershell.exe Smb2FileOpen/Start - Name
Service Control Manager creates Yes - Check if service name exists Microsoft-Windows-Services - ServiceSta-
Windows service or is new tusChange/StatusChange - ServiceName
Powershell.exe launches Yes Kernel Process - Process/Start - Process-
Name, Username, etc.
PowerShell executes commands Yes Microsoft-Windows-PowerShell - any event
to establish Meterpreter C2 for string of known malicious commands
Outbound network connection for Yes - Check if destination IP/port Kernel NetworkTCPIP - TcpIp/Send -
C2 is allowed daddr, dport, etc.
From a technique identification perspective, the most appropriate ETW providers

that can be used to detect all of the attack techniques executed in this experiment are the
Microsoft-Windows-SMBServer, Microsoft-Windows-Services, and kernel NetworkTCPIP
providers. Since this overarching technique has multiple interactions with the Windows
SMB server, Service Control Manager, and has inbound/outbound network connections,
all three providers are needed. Within these providers, however, exists a significant
amount of data to detect all aspects of the SMB-based lateral movement technique. The
author can confirm from experience executing this exact technique against many com-
mercial endpoint defense solutions that these solutions typically only alert on one or two
of the techniques identified in Table 3.11. Figures 3.36, 3.37, and 3.38 illustrate exam-
ple ETW events from the Microsoft-Windows-SMBServer ETW provider that indicate
an administrative user successfully authenticated to the SMB server, Powershell.exe was
requested over the SMB connection, and the Windows Service Control Manager named
pipe (svcctl) was also requested over the SMB connection.
90
Figure 3.36: ETW Event Showing Successful Administrative Authentication to SMB
Server
91
Figure 3.37: ETW Event Showing Request to Open/Start Powershell.exe Over SMB
92
Figure 3.38: ETW Event Showing Request to Open/Start Interaction with the Windows
Service Control Manager
3.2.5.11 Technique Identification - Command and Control (C2) - Application

Layer Protocols HTTP/HTTPS (T1071.001)
The eleventh and twelfth experiments used for validation of the software artifact in this
research move to the Command and Control (C2) category of the MITRE ATT&CK
framework. Command and Control (C2) techniques are the primary techniques that
allow an attacker to control an endpoint that they have compromised. Similar to lateral
movement, there are numerous techniques and protocols that can be used for establishing
Command and Control (C2) with a victim. Since this technique is highly dependent on
93
network connectivity between victims and attacker Command and Control (C2) servers,
the chosen service/protocol that is used can be di↵erent for di↵erent environments that
are compromised.
For the validation experiment in this research, HTTP and HTTPS C2 connections
were used as these are the most common ports/protocols allowed outbound to the Inter-
net in organizations. As with other validation experiments, these two experiments utilize
Metasploit Meterpreter payloads to simulate HTTP/HTTPS C2 connections. The specific
mechanism that established the C2 channel in this experiment was a msfvenom gener-
ated executable that simulated a malware binary that was downloaded to the endpoint.
This simulated malware binary connects to the Kali virtual machine and establishes the
Meterpreter HTTP/HTTPS C2 channel. The specific payload used to generate the sim-
ulated malware executable was the ”windows/meterpreter/reverse http(s)” payload. It
should be noted that the payload used in both Experiments #11 and #12 utilize the
same underlying WinINet components for making HTTP connections, therefore, their
atomic techniques and resultant ETW artifacts are also the same. The only distinction
is experiment #12 utilizes an HTTPS connection as opposed to an unencrypted HTTP
connection. It is important to identify if using SSL/TLS on top of HTTP connections
precludes identification of artifacts in ETW data, therefore, both HTTP and HTTPS
C2 channels were chosen for artifact validation. Table 3.12 illustrates all of the atomic
techniques executed during this experiment.
94
Table 3.12: Experiment 11 - Command and Control (C2) HTTP Atomic Tech-
niques
Event Data? Name(s)
Malware executable launches Yes - Check if executable is Kernel Process - Process/Start - Process-
known, etc. Name, Username, path, etc.
Shellcode calls LoadLibrary() Yes - Check for loads of Kernel ImageLoad - Image/Load - any field
WinAPI function to load wininet.dll with wininet.dll
Wininet.dll
Shellcode calls InternetOpenA() No - Function simply sets up pa- N/A
WinAPI function rameters
Shellcode calls HttpOpenRe- No - Function simply creates a N/A
questA() WinAPI function handle
Shellcode calls Yes - Check hostnames/URIs if Microsoft-Windows-WinINet -
HttpSendRequestA() WinAPI they are allowed wininet connect handle created - Service,
function ServicePort, AddressName
Outbound network connection for Yes - Check if IP/port/process Kernel NetworkTCPIP - TcpIp/Send -
C2 names are allowed daddr, dport, ProcessName, etc.
From a technique identification perspective, the most appropriate ETW provider that
can be used to detect the techniques in this experiment is the Microsoft-Windows-WinINet
provider. This ETW provider logs all actions executed by applications using WinInet APIs
and logs a significant amount of data for each HTTP/HTTPS connection made including
all necessary DNS request information, HTTP URL/URI paths, connection start/stop
information, and SSL/TLS encryption information. All of this detail can aid defenders in
identifying many di↵erent malicious behaviors that are executed by processes in the C2
category of techniques. Figures 3.39 and 3.40 illustrate example ETW events for a DNS
request and HTTP request made during this experiment.
95
Figure 3.39: ETW Event Showing DNS Request of C2 Server Before HTTP(S) Connection
96
Figure 3.40: ETW Event Showing Full URL/URI in C2 Channel Connection
3.2.5.12 Technique Identification - Final Collection of Artifacts
As noted at the beginning of this section, the purpose of this background research in attack
technique identification/coverage within ETW event data was to identify any gaps with
detecting specific techniques or categories of techniques prior to the development of the
User-Defined Context file and the main software artifact. Knowing any gaps in detection
would shape and inform how the User-Defined Context file would be structured and what
kinds of features would be developed for the software artifact as the hypothesis for the
overall research indicated that the developed software artifact would be able to detect
a ”more complete” set of attack techniques than traditional endpoint defense solutions.
The phrase ”more complete” refers to breadth within the MITRE ATT&CK framework
and breadth within atomic sub-techniques for specific techniques within a category. To
97
determine whether there were any gaps for identifying techniques within ETW event data,
two primary questions were asked during each experiment. These questions were:
• Which ETW provider is the most appropriate for detecting a specific attack tech-
nique or sub-technique?
• Within a specific event provider, which event names and field names are the most
appropriate for detecting a specific attack technique?
At the end of this background research phase, it was identified that a significant
majority of the techniques chosen for validation experiments were able to be detected
using solely ETW event data in some way. Some techniques can only be detected using
supporting contextual information, such as baselines of system behaviors. Being able to
define this type contextual information on a system-by-system basis to enable distinctions
between normal and abnormal was a primary goal of this research. Table 3.13 illustrates
all of the validation experiments for this research, a high-level indication of whether the
experiment is detectable using solely ETW event data, as well as details on which event
criteria can be used to detect the technique.
98
Table 3.13: ETW Event Coverage - All Validation Experiments/Techniques
Exp. #
ATT&CK Category - Attack Detectable Using ETW Detection Details
Technique Event Data?
1 Init. Access - Vulnserver Network Yes - Kernel network provider Check inbound source IPs for normal/abnor-
Exploit mal
2 Init. Access - Macro Word Doc Yes - File create, kernel process Check file write process name, parent process
start providers start for normal/abnormal
3 Execution - PowerShell Com- Yes - PowerShell provider Check PowerShell command for normal/ab-
mand normal command strings
4 Execution - Simulated Malware Yes - Process start provider Check if file is normal/abnormal for system
Binary
5 Persistence - Windows Service Yes - Windows service provider Check if process name is normal/abnormal
6 Persistence - Registry Run Key Yes - Kernel registry provider Check if Run key is accessed
7 Privilege Escalation - Result of Yes - Kernel process start Check username/privilege level of process
Exploit provider
8 Cred. Access - LSASS Yes - Kernel API audit provider Check source process, desired access level,
target process ID for normal/abnormal
9 Cred. Access - SAM Hashes Yes - Kernel registry provider Check if registry key with hashes is accessed
10 Lateral Move - SMB Psexec Yes - SMB server provider Check if admin is authenticated, svcctl is ac-
cessed, powershell is accessed
11 C2 - HTTP Yes - Wininet, kernel network Check URL/URI, domain name, destination
providers IP for normal/abnormal/known malicious
12 C2 - HTTPS Yes - Wininet, kernel network Check URL/URI, domain name, destination
providers IP for normal/abnormal/known malicious
With all of the artifacts for all of the techniques identified, the research could then
progress to developing the User-Defined Context file structure and the main software
artifact to detect all of the chosen attack techniques. It should be noted that while
the majority of attack techniques in the validation experiments were able to be detected
using ETW event data, it is likely that not all attack techniques in the MITRE ATT&CK
framework can be detected using ETW event data. Defenders that wish to detect an
individual attack technique not covered in this research would need to go through the
iterative process laid out in the prior sections to identify if ETW event data can be used
to detect a technique or a sub-technique of an overarching technique.
99
3.3 Artifact Design & Development Phase
3.3.1 Artifact Research Goals/Requirements
At the beginning of this research, an initial set of research goals and requirements were
established for the software artifact to ensure alignment with the proposed hypothesis
and ability to answer the research questions. These original artifact research goals and
requirements were:
1. The software artifact must be able to consume real-time data from multiple Event
Tracing for Windows (ETW) event providers without user interaction and without
the software artifact failing.
2. The software artifact must be able to import and understand a schema of data
called ”user-defined contexts” and map each defined line item to 1) an ETW event
provider, 2) a behavior within this ETW provider, and 3) what action to take on
that data.
3. The software artifact must also be able to understand nested/chained user-defined

contexts that allow users to specify atomic behaviors, chains of behaviors, and gen-
eral definitions of facts that are used in other context definitions.
Throughout the development of the software artifact and User-Defined Context file,
these goals were prioritized, however, changes to these goals were needed as additional
research on the subject of endpoint threat detection was conducted. The research con-
ducted by the author and Bhaskar Rimal (DSU) on endpoint solution resiliency identified
that endpoint defense solutions need to also allow users the ability to define the semantic
meaning of each alert for specific attacker techniques. As such, the goals were updated to
add this goal:
100
1. The software artifact must allow users to configure the meaning of alerts on each
configured behavior.
3.3.2 Artifact Development Goals/Requirements & Initial De-
sign
Based on the artifact research goals/requirements, an initial set of design/development

requirements for the software artifact and an initial artifact design were also established
to guide the code development. These requirements were as follows:
• The artifact should be written in C# and utilize the .NET Framework for all needed
functionality.
• The artifact should operate as a single executable.
• The artifact should accept input and display output through the standard Windows
command prompt console.
• The artifact should include multiple internal classes to support all of the functional
requirements.
• The artifact should support multi-threaded operations to handle concurrent tasks

such as consumption of ETW events and searching through ETW event data for
evidence of attack techniques.
• The artifact should use as many Microsoft developed libraries as possible and only
use third-party libraries as needed to increase the strength of the artifact against
any future software supply chain compromises.
Based on all of these requirements, an initial design diagram was created to illustrate
all of the interactions of the artifact with the endpoint it is running on. Figure 3.41
illustrates this diagram.
101
Figure 3.41: EDAC Initial Artifact Design
As stated in the artifact requirements, the artifact must be able to import and under-
stand a schema of data called ”user-defined contexts” and use these definitions to identify
attack techniques when ETW events are consumed. The structure and syntax of this
configuration file were critical to identify first before parsing code could be developed
in the artifact. This is because the User-Defined Context file is the primary component
that the user interacts with in order to achieve attack technique detection coverage across
multiple MITRE ATT&CK categories.
3.3.2.1 User-Defined Context File Design Goals
As proven in prior sections, ETW provides a significant amount of detail within event data
that can be used for identifying attack techniques on endpoints, however, consumption of
ETW events alone does not ”detect” or ”alert” users that a particular attacker technique
was executed on an endpoint. Each ETW event and field within an event have to be
carefully assessed by some capability before a detection or an alert can be generated. For
102
the purposes of this research, this means that some level of configurability was needed
to compliment ETW event consumption to be able to detect attack techniques across
various ETW event providers, event names, and event fields. As noted on the Mandiant
Github page for SilkETW [26], SilkETW is able to use YARA rules to detect defined
criteria within ETW event data and display the matches in the console window. This
is a powerful capability, however, with SilkETW being a stand-alone executable that
subscribes to a single ETW event provider, the abilities to scale this process of detecting
attack techniques across multiple event providers, event names, and event fields would be
difficult. This is one area that the artifact developed in this research adds novelty and
capability to the science.
One of the primary influences for achieving configurability in the developed software
artifact going into this research was the concept of Infrastructure as Code (IaC). The
author has professional experience working on DEVOPS teams to develop IaC and IT
automation code for system deployment and configuration. During this experience, the
author used a combination of products such as Ansible, Terraform, and Packer to deploy
Windows systems, install software, set configuration settings, etc. Microsoft defines [52]
Infrastructure as Code as having three core properties. The core property of IaC in-
cludes a machine-readable definition file with declarative definitions of properties. These
declarative definitions describe how the system should be provisioned. The definitions
can also be atomic/direct assignment or reference properties or chains of properties. For
example, the Ansible Project illustrates [53] these characteristics in an example figure of
configuring an inventory configuration file (.ini). Figure 3.42 defines a top-level group of
systems, assigns these systems to additional group names, and finally assigns variables
to these systems through referenced/inherited names in an INI configuration file syntax.
Figure 3.43 illustrates configuring the same elements and properties but within a YAML
file format/syntax.
103
Figure 3.42: Ansible Inventory Definition with Variable Assignment (.ini file) [53]
Figure 3.43: Ansible Inventory Definition with Variable Assignment (.yaml file) [53]
YAML is an open-source project that is meant to provide a human-friendly language

that is also able to be serialized by all programming languages [54]. INI is a long-standing
configuration file format/syntax that has been used in the Windows operating system
104
since Windows 3.0 [55]. Both configuration formats in Figures 3.42 and 3.43 are valid
for Ansible to function properly, however, they di↵er greatly in complexity of the syntax.
Some key di↵erences in the syntax are that hosts are defined underneath the higher-level
groups they are a part of in the YAML file, but are defined in separate blocks in the INI
file. These examples of IaC configuration with INI and YAML files served as input to
this research to establish the following requirements for the syntax of the User-Defined
Context file:
• The syntax needs to be declarative and have inheritance similar to YAML or INI in
which entities can be defined, properties can be defined for those entities, and the
entities can be referred to and used in the configuration by other entities.
• The syntax needs to support defining di↵erent telemetry sources that map to mul-
tiple ETW event providers for the artifact to consume events from.
• The syntax needs to support defining ”constructs” that represent facts or circum-
stances in the system that can be referred to or considered prior to generating an
alert on activity.
• The syntax needs to support defining ”behaviors” (collections of field values in an

ETW event) an operator wants the artifact to consider as normal or abnormal.
• The syntax needs to support defining the inverse of criteria (negate) to identify
attack techniques.
• The syntax for all of the above features needs to be written in such a way that an
operator who is not familiar with YAML, INI, data serialization, or IaC concepts
can easily understand the intent and e↵ect of the configuration within a single line
or a few lines of ”code.”
Syntax complexity was a primary focus area for determining how the User-Defined
Context file would be constructed because it can drastically a↵ect user adoption and
105
usability. While YAML and INI configuration structure/syntax are considered industry
standards and provide significant data serialization capabilities, the syntax does not satisfy
all of the requirements set for this research. It was identified that a custom file structure
and syntax would be needed to fit all of the defined requirements. At the beginning
stages of this research, a prototype syntax structure was defined that matched all of
the requirements. Figure 3.44 illustrates this initial design of the User-Defined Context
syntax structure. As noted in prior sections, each atomic attack technique in an attack
chain may include multiple atomic sub-techniques in di↵erent ETW event providers and
EventNames, or a single ETW event provider and a single EventName. Figure 3.41
illustrates this initial vision of being able to quickly define, within one single configuration
line, an individual field value within a specific ETW event that should be considered
abnormal and thus generate an alert in the console.
Figure 3.44: Initial EDAC User-Defined Context File Syntax
The overarching structure of the User-Defined Context file was initially visualized as
being grouped into sections of related configurations. For example, in order to detect spe-
cific techniques within identified ETW providers, users would need the ability to configure
106
telemetry sources, behaviors containing criteria matching identified ETW event field val-
ues, as well as other contexts that may assist in detection. Each of these items should be
configured in their own sections of the User-Defined Context file to maintain the highest
level of readability of the file. Figure 3.45 illustrates these particular sections and their
overall placement within the User-Defined Context file structure.
Figure 3.45: User-Defined Context File - Overall Structure
107
3.3.2.2 User-Defined Context File Syntax Development - Telemetry Section
Since the User-Defined Context file was chosen to implement a custom structure, the
development of the syntax was a highly iterative process that took place along with the
development of the parser code that imports and understands what the user is configuring
the artifact to do. Figure 3.46 depicts the top of User-Defined Context file in the overall
structure. The file itself is a plain-text file that is imported by the software artifact
and processed line-by-line. The artifact iterates through each line of the file and decides
based on pre-defined criteria of how to interpret all of the elements in the written line.
Similar to YAML, any line with a ”#” character at the beginning is treated as a line
comment and will not be parsed. Each line is meant to define and set a specific property
of something by specifying what to define to the left of the ”=” sign and what value to
set that property to on the right side of the ”=” sign. Additionally, the structure of the
User-Defined Context file is broken up into two core sections defined by comment lines
that should not be changed. These sections are the telemetry definition section, and the
construct/behavior definition section. Each section requires the artifact to handle those
configuration lines di↵erently, therefore, they are broken up into di↵erent sections for both
user comprehension and ease of parser code development/maintenance.
108
Figure 3.46: User-Defined Context File - Telemetry Definition Section
Another defining characteristic with the chosen syntax structure was the dotted sep-
aration of words to set specific properties. For example, each section of the User-Defined
Context file has di↵erent top-level words that are allowed to clearly define what the config-
uration line does and where it goes within the text file. Additionally, within those allowed
top-level elements, only certain second level, third level, fourth level, etc. elements are al-
lowed. Creating the syntax this way allowed for greater levels of control in the readability
of the configuration lines for the user. As shown in Figure 3.47, the telemetry section has
a top-level allowed element of ”program” indicating that these are program-level settings.
Two more defining characteristics about the telemetry section syntax are the fourth and
fifth level elements. The fourth level element is interpreted by the parser as a custom
name that the user provides. This allows the user to reference this telemetry source name
later in the User-Defined Context file during the behavior definitions.
109
Figure 3.47: User-Defined Context File - Telemetry Definition Allowed Elements
The fifth level element of the telemetry definition syntax indicates that there are three
required properties that need to be set for any telemetry sources that are defined. These
properties are source, officialname, and eventfilter. Source is the property that defines
what type of telemetry source is being defined (ETW, Event Log, etc.). As of this writing
the artifact is only configured to handle ETW telemetry sources but will be programmed
to allow others in the future. Officialname is the property that requires the user to specify
the name of the ETW provider that will be consumed for this telemetry source (Example:
Microsoft-Windows-PowerShell). The officialname property value should match an avail-
able event provider that the operating system includes. Finally, the eventfilter property
allows the user to specify a comma delimited list of EventNames that exist within the
specified event provider that will filter out EventNames that do not match what was spec-
ified during the event consumption operation of the artifact. Optionally, the user can also
specify ”none” and all EventNames will be consumed with no filtering. Figure 3.48 illus-
trates two telemetry source configurations that have been set in an example User-Defined
110
Context file.
Figure 3.48: User-Defined Context File - Telemetry Definition User Providers vs. Kernel
Providers
It should be noted that Figure 3.48 depicts two sections for user ETW providers and
kernel ETW providers. This is a typical distinction in ETW event consumption tools
as Windows provides di↵erent mechanisms for subscribing to event providers that are
considered ”user-level” providers and event providers that are considered ”kernel-level”
providers. The SilkETW project illustrates [26] that to subscribe to a user provider the
user must specify the full provider name or GUID to subscribe to a user provider. To
subscribe to a kernel provider, the user must specify a kernel ”keyword” from a body
of known keywords that the operating system publishes. The artifact in this research
has a similar implementation of subscribing to these di↵erent classes of event providers
and thus implements this distinction in the User-Defined Context file in the officialname
property. Figure 3.48 also illustrates in the kernel telemetry provider definition that the
officialname is ”kernel:Process” whereas the user provider definition is the typical ETW
provider format. This format is a custom defined format where the parser identifies the
distinction that this line is referring to a kernel provider and takes the second half of the
string after the ”:” character and applies it as the kernel keyword for subscription later.
In this case, the ”Process” kernel keyword subscribes to the ETW provider that sends all
process start/stop events that the kernel handles.
111
3.3.2.3 User-Defined Context File Syntax Development - Construct Section
The construct section of the User-Defined Context file is meant for the user to define facts
and circumstances about the system that is being monitored. The intent is then for the
construct that the user defines to be referred to in behaviors later in the file. From a syntax
development perspective, it was important to make the top-level element of the construct
section distinct from both the telemetry definition and behavior definition sections. This
further ensures user readability and reduces errors in the User-Defined Context file. Figure
3.49 illustrates the syntax structure for the construct section and allowed elements at each
level. Similar to the telemetry definition section, the user defines a custom name for the
construct which the artifact automatically stores as a known variable name for when the
behavior section is parsed.
Figure 3.49: User-Defined Context File - Construct Definition Allowed Elements
For the purposes of identifying attack techniques within ETW event data, there are
numerous situations where adding context such as username, file path, IP address, or
process name can make identification more certain. For example, if a user wants to know
when a specific username or set of usernames launch powershell.exe, the user can define a
112
construct to use with the behavior definition section. Defining a construct with a comma
delimited list of usernames would be treated as an ”OR” condition when being assessed
for matching criteria in the behavior analysis operation. Constructs can also be used
together in multiple fields for a behavior definition that would create an ”AND” condi-
tion. An example of this could be the user wants to know when any set of usernames
launch any set of process names. Each permutation of the username construct and pro-
cess name construct would be compared together with an ”AND” condition. Allowing
users to specify these constructs with groups of items enables the ability to create a sig-
nificant amount of permutations of conditions to compare events against, which further
strengthens the identification of attack techniques. Figure 3.50 illustrates some example
construct definitions for local administrator usernames, concerning usernames, concerning
app names, web browser process names on the system, trusted IP addresses the system
should contact, etc.
Figure 3.50: User-Defined Context File - Construct Definition Section
3.3.2.4 User-Defined Context File Syntax Development - Behavior Section
The final section of the User-Defined Context file is the behavior definition section. This
is the section where users configure specific ”behaviors” or attack techniques that they
want to the artifact to alert on using all of the already configured telemetry sources and
constructs in the rest of the context file. Similar to the telemetry and construct sections,
113
the behavior section has a di↵erent set of top and second level elements in the syntax to
maintain distinct understanding from the user perspective. As with the syntax trees for
the other sections, the behavior section has an element that is customized by the user.
For the purposes of behaviors, this allows users to assign unique names to the specific
behavior that they are configuring to be detected. For example, if the user is configur-
ing the artifact to alert when a concerning user from a construct launches a concerning
app like PowerShell, the user can specify the behavior name as ”concerningUserLaunch-
esPowerShell” as the third-level element. Allowing customized behavior names serves two
purposes. First, each behavior needs at least four lines of code to properly configure
it, therefore, a unique name is needed for the parser to understand how to apply mul-
tiple configuration lines. Second, the User-Defined Context file will be a large text file
once all behaviors are configured and having these unique names in the syntax allows
for greater readability of the context file to enable greater understanding of what attack
techniques/behaviors are being monitored for. Figure 3.51 illustrates the syntax tree and
allowed elements for the behavior section.
Figure 3.51: User-Defined Context File - Behavior Definition Allowed Elements
The fourth level element of the behavior syntax tree represents the four minimum
114
properties that need to be configured for each behavior before the artifact will fully de-
tect a defined behavior. Each behavior needs a telemetry source to search within (teleme-
trysource), a specific field name and value of criteria to search for in ETW event data
(field.FieldName or field.any), a string of the alert text that should be used when the
criteria is found within ETW event data, and a color that the alert should be displayed in
on the alert console. For the field property, a real field name within ETW events from the
telemetry source defined should be specified on the left side of the ”=” sign and the spe-
cific string of text representing the artifact of the technique that is desired to be searched
for within that telemetry source is defined on the right side of the ”=” sign. Multiple
field names can be specified by adding lines of code for that behavior which will create
”AND” conditions for the defined criteria of each field name. Conversely, if the user does
not know what the field name is within the chosen ETW telemetry source, or the specific
artifact to search for lands in the middle of a large string in the ETW event, the user
can specify ”field.any” and the artifact will know to search the entire ETW event for the
specific string.
For example, Figure 3.52 illustrates how to configure a behavior to look for in the
User-Defined Context file. The example behavior is intended to detect when Microsoft
Word launches an executable as a child process, which is a common detection for mali-
cious macros that extract or launch a process. The telemetry source is the kernel-level
telemetry source for process starts/stops that was defined earlier in the telemetry section
named ”process starts”. In order to detect this behavior, any process start event in the
process starts telemetry source that has ”WINWORD” (the standard name for Microsoft
Word’s process name) listed in the ParentProcessName field is indicative of Microsoft
Word launching a child process. Under testing and evaluation of this behavior, Microsoft
Word did not normally launch any child processes, therefore this proved to be sufficient
criteria to detect this behavior. The alertmeaning field explicitly states what happened
and how this relates to attack techniques, which immediately gives users necessary context
115
to respond further. Finally, given that this is high-fidelity criteria, the log color chosen
was red, to indicate a level of severity or importance to the user. However, as stated
previously, this color can be set to any color allowed by Windows Console Colors that fits
the user’s needs.
Figure 3.52: User-Defined Context File - Behavior Definition Example
It should be noted that Figure 3.52 represents one single behavior/technique of poten-
tially many other atomic techniques that could be included within the larger technique for
a malicious Word document. The defined behavior is not a guarantee that if this were to
generate an alert on the artifact, that a malicious Word document is confirmed. It is sim-
ply a behavior that defenders would typically configure that adds suspicion. As proven in
the Automated ETW Event Generation script section, some attack techniques only need
a small number of behaviors to confirm their existence, and some are more difficult to
detect with certainty. The idea for the behavior section in the User-Defined Context file
is for the user to iteratively test and evaluate current and future techniques of concern
for the specific system being monitored and define as many behaviors as needed. Ideally,
behaviors should overlap with each other and multiple behaviors be used to detect a single
technique. This adds redundancy in the event a specific sub-component of a technique
does not show up in an expected ETW event. Detecting attack techniques is best done
using a totality of circumstance with many pieces of supporting information that allows
users to conclude what is occurring on a system.
In order to develop a more complete set of behaviors to detect all of the chosen
116
attack techniques for validation in this research, the output from the prior background
research on attack technique coverage within ETW event data was used in an iterative
way to develop the specific behavior definition lines within the User-Defined Context
file. Each attack technique chosen for validation in Table 3.1 was assessed for any sub-
components/techniques within that overall technique and documented. Each of the atomic
techniques for the overall technique were assessed for any identifying characteristics or
artifacts that could be searched for in Splunk where all of the collected ETW event data
is located. If a reliable artifact was found in a specific ETW event, the EventName
and any relevant field names where the artifact was located were documented. Finally,
once reliable field names and artifacts were found, these were used to develop specific
behaviors in the User-Defined Context file before a second execution of the technique
occurred to validate proper function of the configured behavior. Figure 3.53 illustrates
this iterative process for developing behavior definition lines against individual atomic
techniques for a chosen validation experiment. Each atomic technique documented in the
prior attack technique coverage section was assessed for coverage within ETW event data
and each field needed for detection was put together in the desired behavior definition
syntax. In this example, a behavior could be developed in the User-Defined Context file
from the example ETW event data to detect that a macro-enabled Word document was
opened, which represents the first atomic technique in the larger body of techniques for
detecting attacker-created Word documents and their malicious actions. This analysis
process shaped and informed the direction that the parser code development process took
that is detailed in the next section.
117
Figure 3.53: User-Defined Context File Configuration Line Development
3.3.3 Final Artifact Design, Class Structure, & User Workflows
At a high-level, the final artifact design matches the original artifact design developed at
the beginning of this research with a few exceptions. The overall user workflow of the
artifact remains the same where users configure a User-Defined Context file, import and
parse the file using the main artifact executable, subscribe to defined ETW providers,
and generate alerts to a console window based on defined criteria. Since the artifact is a
console utility where the user interacts and controls the execution of the program through
a command-line interface, all of these user workflows needed to be constructed together
with easy to understand and logically flowing user interfaces. The identified workflows
are a sequential progression both in concept and in internal program execution, therefore,
a menu-based interface was chosen as the primary method in which users interact with
the program. Upon launching the standalone artifact executable, the user is presented
with a single menu containing options for either importing a User-Defined Context file
or monitoring the system. As of this writing, the artifact has been developed to require
importing and parsing of the User-Defined Context file prior to being able to monitor the
system (subscribing to ETW providers). Figure 3.54 illustrates the initial menu that the
user is presented on program launch.
118
Figure 3.54: EDAC Main Menu
Within the artifact code base, there are numerous internal C# classes that were built
to support all of the menu functionality, console window behavior, User-Defined Context
file parsing, processing of context file configurations, and final ETW event subscription
functions. The first primary action that users taken when launching the artifact executable
is importing and process of the User-Defined Context file. The context file parsing process
and code are illustrated in the next section.
3.3.3.1 User-Defined Context File Parser & Import Process
Once the desired User-Defined Context file structure and syntax were developed from a
usability perspective, a parser needed to be written in the software artifact that would
import the context file, iterate through each configuration line, understand what the user
is attempting to configure, and store any necessary information for future use during event
consumption and monitoring. Since the developed syntax for the User-Defined Context
file was decided to be a custom structure, a custom parser was also needed to be able to
understand the structure of the file during the import process. As noted in prior sections,
the User-Defined Context file is a plain-text file format, therefore, the parser was written
to be able to consume plain-text files and iteratively parse individual configuration lines
119
that the user has configured in the established syntax/schema.
At a high-level the parser code first stores all lines of the User-Defined Context file
into a string array. Once all lines are stored in a string array, the parser then iterates
through the lines and stores lines specific to the telemetry definition section in a .NET List
and stores lines specific to the construct/behavior section in a di↵erent .NET List. The
parser knows which list to store items in because of the established anchor structure of
”TELEMETRY SECTION” and ”CONSTRUCT/BEHAVIOR” section headers written
as line comments in the file sections. As defined in the Microsoft .NET documentation
[56], Lists are collections of strongly typed objects that can be accessed by index and
can be used to store objects of any type similar to other types of data structures (arrays,
dictionaries, etc.). For the software artifact in this research, Lists are used heavily because
they can quickly be expanded and contracted in size as needed and do not require fixed
sizes during instantiation. Figure 3.55 illustrates the process the parser takes to iterate
through each configuration line and store lines for each section in respective lists to work
on later.
Figure 3.55: User-Defined Context File Parser Code - Section Parsing
120
Once all lines in the original string array have been divided into their relevant section
lists, each section is then parsed individually. As noted in prior sections, the syntax of the
telemetry definition, construct definition, and behavior definition sections are di↵erent to
maintain the highest level of readability and understandability for conveying what the
software artifact will do with the configured lines. As such, individual parser code was
written to parse and make use of each section’s configured lines. For a specific telemetry
source in the telemetry definition section, the user first defines what the telemetry source
name will be for the rest of the User-Defined Context file. For example, if a user wishes
to define a telemetry source of ”powershell” the user will start a new line and put ”pro-
gram.define.telemetry = powershell” in the telemetry section. In subsequent configuration
lines for this telemetry source, the user will change the top two levels of the configura-
tion line to say ”program.set.[user defined name].[next property to set] = value” until all
required properties are configured.
The parser code is written to iterate through each syntax tree element of individual
configuration lines delimited by a ”.” to understand additional branches of parsing code to
take and then use the value that the user supplies after the ”=” as the value that is being
configured for the configuration line. Because this software artifact includes the ability
for a user to supply input that needs to be parsed, great care was taken to minimize the
risks of malicious code or character injection. This is accomplished by using a series of
checks in various areas of the parsing code to verify that input received from the user
is what is expected for the specific area being processed and nothing more. Regarding
syntax tree element naming conventions, although the established syntax tree elements
are set with specific names in this research, new names for elements can be made in the
future. The parsing code is written in such a way that each element refers to an index of a
definition of elements for each level to determine which branch to take rather than writing
their names explicitly throughout the parsing code. Figures 3.56 and 3.57 illustrate how
the parsing code defines allowed telemetry section elements that can be changed for each
121
level of the syntax tree and how the parsing code iterates through syntax tree elements
to identify code branches to take.
Figure 3.56: User-Defined Context File Parser Code - Allowed Syntax Elements
Figure 3.57: User-Defined Context File Parser Code - Syntax Tree Iteration
122
Writing the parsing code in this way with nested ”IF” statements for each level of the
syntax tree allows for easier error checking of user input in the User-Defined Context file.
If a user misspells a syntax element or is missing a required property to be set, the error can
be caught in the corresponding ”ELSE” condition of the relevant ”IF” block and produce
very detailed error messages so that users know exactly which element in the syntax tree
that was input incorrectly. As the parser iterates through each line defined in the User-
Defined Context file, the parsing code keeps track of required properties that are needed
for a specific telemetry source. For example, telemetry source definitions require three
properties to be configured before the telemetry source can be stored for use later in the
program execution (source, officialname/guid, eventnamefilter). Because the parser needs
to be able to handle parsing individual lines as well as blocks of lines, the chosen method
to aggregate telemetry sources and required properties into a single object to be referenced
later during processing was to use .NET Dictionaries. A .NET Dictionary is a collection of
key/value pairs of specific data types that can be referenced by key [57]. For the telemetry
source parsing, this dictionary is called ”UserDefinedTelemetryToWorkOn”. Within this
dictionary, the telemetry name that the user defines is the key for the key/value pair, and
the value includes all of the required properties for that telemetry source. Figure 3.58
illustrates the parsing code that iteratively stores each property of a defined telemetry
source into a single object in the dictionary referenced by the key (user defined telemetry
name).
123
Figure 3.58: User-Defined Context File Parser Code - Collect Required Telemetry Prop-
erties
Once all required properties for a telemetry source are stored in the UserDefinedTeleme-
tryToWorkOn dictionary, the parser will iterate through the next telemetry source and
perform these same steps until all lines in the telemetry section are parsed. After the
telemetry section is parsed, the parser moves on to parsing the construct and behavior
section of the context file. The construct/behavior parsing code works much in the same
way as the telemetry section parser, where, the parser will iterate through individual
lines, keep track of required properties that are needed for each block, and store each con-
struct/behavior into a dictionary that is used later during processing. The key distinction
between parsing constructs and parsing behaviors is in the number of lines that are parsed
for each. Each construct is designed to only be a single line that a user inputs in the User-
Defined Context file, and each behavior has multiple properties/lines that are written as
blocks in the context file. As an example, if a user wants to configure a construct that
stores the values of specific usernames they are concerned about, the user can write in
the User-Defined Context file ”global.construct.concerningUsers = bob,nancy,alice” and
the construct parser will parse only that line and no other lines. Figure 3.59 illustrates
the construct parsing code that is used for parsing constructs within the construct sec-
tion. It should be noted that although constructs are meant to be supplied with comma
delimited values, constructs are stored in the GlobalConstructs dictionary as-is from the
124
User-Defined Context file with no modifications or processing. During the behavior ex-
traction and class generation process that is described in the next sections, construct
values are referenced and extracted from the comma delimiters as individual criteria.
Figure 3.59: User-Defined Context File Parser Code - Construct Parsing
Once constructs are parsed, the parsing code then proceeds to parsing behaviors that
the user has configured in the User-Defined Context file. The behavior parsing code
functions in a similar way as the telemetry parsing code where multiple configuration
lines are aggregated together into a single string containing all of the configured properties,
and then stored into a .NET Dictionary named ”BehaviorsToAddToClass” for processing
later. Figure 3.60 illustrates the entry point to the behavior parsing code and storage of
behaviors into a dictionary.
125
Figure 3.60: User-Defined Context File Parser Code - Behavior Parsing
The key di↵erence to behavior parsing from telemetry source parsing is that some
initial processing is done for certain properties of configured behaviors prior to storing
the behaviors in the .NET dictionary for full processing later. The primary property
this applies to is field names that are configured in the User-Defined Context file. As
stated in the syntax definition section, users can input specific field names appended to
the property ”field” (ex: ”field.Username”) to indicate which field name inside ETW
event data to look for a specific value for that behavior. There are special circumstances
where additional functionality is needed from a user configuration perspective that also
need special handling in parsing code. If a user specifies that a field should be any value
contained within a specific construct that is defined, the parsing code needs to be able to
retrieve all of the values defined in that construct and store them in the behavior string
that is stored in the BehaviorsToAddToClass dictionary for full processing later. Negation
of field values is also allowed in the User-Defined Context file and users can negate the
values of a field or a construct by prepending a ”!” to the value or construct. Figure
3.61 illustrates how the user would configure this in the User-Defined Context file which
illustrates the need for this special handling in the parsing code.
126
Figure 3.61: User-Defined Context File - Construct Reference
To accomplish this special condition in the User-Defined Context file, the user is re-
quired to prepend the string ”global:” in front of the construct name they are trying to
reference. Having this string prepended allows the parsing code to know when a user
is attempting to reference an already defined construct and retrieve the values stored
in the construct prior to storing the value of the field into the BehaviorsToAddToClass
dictionary. As noted earlier, constructs are stored within a .NET Dictionary named
”GlobalConstructs”. Since .NET Dictionaries are collections of key/value pairs, the pars-
ing code simply does a lookup using the user supplied context name after the ”:” (ex:
”global:concerningUsers”) as the key to get the value of the defined construct. Figure
3.62 illustrates the behavior parsing code that handles construct value fetching.
127
Figure 3.62: User-Defined Context File Parser Code - Construct Value Fetch
Once all lines for a configured behavior have been parsed, the behavior is stored into
the BehaviorsToAddToClass dictionary as a single line of properties for a given key (be-
havior name). Once the parser is done parsing and storing all properties of a configured
behavior, the parsing code continues to iterate through all other stored lines in the con-
struct/behavior section and repeats the extraction and storage process. From a user
interface perspective, the parsing process requires little user interaction besides specify-
ing the location of the User-Defined Context file to import. Figures 3.63, 3.64, 3.65, and
3.66 illustrate the user interface workflow for importing and parsing a context file.
Figure 3.63: EDAC User Interface - Main Menu
128
Figure 3.64: EDAC User Interface - Import UDC Menu
Figure 3.65: EDAC User Interface - Specify File Path to UDC File
129
Figure 3.66: EDAC User Interface - UDC File Parse Results
Once all configured behaviors are parsed and stored in the BehaviorsToAddToClass
dictionary, the parser has finished parsing the User-Defined Context file and execution
can continue on to the next stage in the process, ETW class generation for each telemetry
source.
3.3.3.2 At Run-Time Class Generation & Behavior Detection
During the development of the behavior syntax tree in the User-Defined Context file,
multiple problems were identified regarding the method that needed to be used to ingest
the defined behavior criteria (field names, field values, negations, and/or conditions, etc.)
and operationalize them into code so that the behaviors can be ”detected” when processing
a stream of incoming ETW events that have been subscribed to. For example, Figure 3.67
illustrates an example behavior that a user could configure to detect when an unknown
binary on the system makes an HTTP request to an unknown domain name. This behavior
takes advantage of using constructs with large amounts of values in the ProcessName and
130
ServerName fields to identify unknown binaries and unknown domain names respectively.
To identify ”unknown” binaries (not known to be on the system), a user would use a
PowerShell script created for this research that creates a comma delimited list of all .exe
files on the system to copy and paste into the defined construct in the User-Defined
Context file. For the unknown domain names construct, a user would create a comma
delimited list of domain names that are known to be trusted for this system to connect
to.
Figure 3.67: EDAC User Interface - UDC File Parse Results
These behavior criteria are very sound in defensive logic because it will allow a user
to conclude that A) an unknown binary on the system made a network connection, B)
an unknown binary on the system made an HTTP connection, C) an unknown binary on
the system made an HTTP connection to a untrusted domain. All of these conditions
together immediately raise the severity of the action as there are multiple conditions that
have occurred that are defined as abnormal on the system (as defined in the User-Defined
Context file). However, challenges became apparent when considering how to create these
conditions in C# code or data structures for the detection and alert to occur because the
intent of the User-Defined Context file is that the user can define any amount of behaviors
of any complexity. For example, all fields specified by a user in a behavior need to be
treated as an ”AND” condition and all of the values retrieved for the constructs need to
be treated as ”OR” conditions. When putting these conditions together in a global .NET
dictionary that all telemetry sources use for example, the cycles needed to check every
131
incoming event from every subscribed telemetry source against every defined behavior
would be too costly in performance. It was decided that behaviors should only be checked
if they are relevant to a specific telemetry source. This drove development of the ETW
provider subscription functionality to be individual C#/.NET classes so that behaviors
could be configured and alerted on inside of each class and no wasted cycles would occur
iterating through behavior criteria that does not apply to a specific telemetry source.
This still introduced an additional challenge of how to detect each behavior inside of
a telemetry source’s class file. Many iterations of test and evaluation occurred during
the development of these individual ETW class files to identify if behaviors should be
detected using a dictionary lookup mechanism or an integrated method as events are
processed. Since behaviors can consist of numerous AND/OR conditions from field values
and constructs, it was determined the quickest way to detect a behavior would be in the
form of an IF statement block with all of the user-defined conditions contained in the
IF statement criteria. This way, as each ETW event being consumed is ingested by the
consumer function, the conditions can be immediately checked and alerted on if necessary.
The last challenge identified during this process of developing behavior detection code
was how to generate these behavior detection IF statement blocks while the program is
running. Since behaviors that a user defines in the User-Defined Context file are not
ingested into the program until run-time, the artifact needed a way to generate all of the
behavior detection IF statement blocks prior to consuming ETW events. It was decided at
this point that both the ETW subscription classes for each telemetry source and all of the
user defined behavior IF statements would need to be created at the same time because of
the dependency on each other. However, creating C# code at run-time is only part of the
equation. This dynamically generated code would also need to be compiled, loaded, and
executed at run-time to prevent users from having to run the artifact twice. This drove
the development of the artifact to use .NET Reflection as a method to accomplish this.
.NET Reflection is a dynamic programming capability in the .NET Framework where
132
assemblies (classes) can be loaded at run-time and objects accessed after they are loaded
[58].
Using .NET Reflection as the core capability to load classes while the artifact is running
introduced multiple downstream benefits that made development of the ETW subscription
and behavior detection easier. Instead of creating static class files for every possible
ETW provider identified in the attack technique coverage section to detect the techniques
in the validation experiments, one single class file could be created as a template and
changes could be made to the template as needed before the artifact begins consuming
ETW events and detecting behaviors. The inspiration for this came from the author’s
experience conduction penetration testing engagements using a tool called Veil-Evasion
for generating Metasploit payloads that evade Anti-Virus/endpoint solutions [59]. One of
the capabilities built into Veil-Evasion is a tool called the Arya Crypter, built by HarmJ0y,
for obfuscating C#/.NET based Metasploit payloads [60]. The specific method used by
Arya to perform this obfuscation involves loading C#/.NET classes at run-time using
.NET Reflection. The C#/.NET Metasploit payload code that is loaded into the .NET
Reflection compilation function is represented as a single string object. This was the
primary influence to use the same method for the template of the ETW monitoring class
files in this research. Figure 3.68 illustrates the process Arya uses to load C#.NET code
represented as a string object and execute functions within that code after it is loaded at
run-time.
133
Figure 3.68: Arya .NET Crypter Using .NET Reflection to Load C# Code at Run-Time
The Arya Crypter is meant to obfuscate C# code in addition to dynamically loading

it at run-time, however, the obfuscation capabilities were not needed for this research.
Only the class loading from a string object capability was needed. The primary class
template used for ETW monitoring in this research is stored in a string object within
the ETW User Subscribers.cs class file in the main artifact’s solution. This ETW class
template represented as a string contains variable placeholders for all items that need to
be changed during the class generation process. These variables that are filled during
the class generation process include function names, telemetry source names, behavior
IF block placeholders, etc. The chosen method for quick searching and replacement of
variables within the class template was to use ”%” symbols wrapped around a unique
name. Figure 3.69 illustrates this string object containing C# code along with some of
the variables that are replaced during the class generation process.
134
Figure 3.69: ETW Monitoring Class Template
The class generation process for the software artifact is started by the user once the
parsing of the User-Defined Context file finishes and is depicted in the user interface
screenshot in Figure 3.70. There are two primary functions in the software artifact that
handle generating the class strings that get loaded/compiled and executed during run-
time. These functions are GenerateUserProviderClasses and GenerateKernelProvider-
Class. It was determined during test and evaluation of subscribing to ETW events in
.NET code that kernel provider subscription cannot be in two di↵erent classes within the
same solution due to conflicting backend subscription sessions, therefore, kernel providers
and behaviors that the user configures in the User-Defined Context file are placed in a sin-
gle kernel provider class that is generated at run-time. All other telemetry sources/ETW
providers defined by the user in the context file are contained within their own class that
is generated at run-time.
135
Figure 3.70: EDAC User Interface - ETW Class Generation Process
At a high-level, the user provider class generation and kernel class generation functions
operate in similar ways. They both iterate through user defined telemetry sources, then
iterate through each behavior defined for that telemetry source, then create IF statement
blocks containing all necessary criteria for each behavior, then store the finished class
string into a dictionary for compilation in the next step. In code, much of these itera-
tions are contained within nested ”foreach” loops that work through all telemetry sources
defined in the UserDefinedTelemetryToWorkOn dictionary and the BehaviorsToWorkOn
dictionary. Figure 3.71 illustrates all of these interactions from the two main class gen-
eration functions working with the various dictionaries and generating a final class string
that can be compiled with .NET Reflection.
136
Figure 3.71: ETW Class Generation Process
The most important part of the class generation process is the generation of the
behavior detection blocks. As noted earlier, behaviors can include numerous field names
and field criteria to detect, therefore, the class generation code needed to be written in
such a way that all possible combinations of user configurations are able to be generated
dynamically without issue. The method chosen to accomplish this was to first generate
behavior detection IF statement blocks using variables in a similar way to the overall
class string, and then place generated IF statement blocks into the class string iteratively
until all behaviors have been generated. In order to generate a behavior IF block and
put it into the ETW class template, the class generation code first creates the criteria
that is put inside the ”if()” parentheses. Based on the criteria specified by the user
in the User-Defined Context file (field names and field values), the criteria is generated
dynamically using a template of code that would normally go into the IF statement.
Figure 3.72 illustrates the criteria template that is used to develop detection criteria for
each named field specified for a behavior. For example, if a user specifies ”field.Username”
137
in a behavior, this is considered a named field because the field will exist in the ETW
event as an JSON object. If a user specifies ”field.any” to search for a value within any
field in an ETW event, this is considered a non-named field and has di↵erent IF block
criteria. Figure 3.73 illustrates the IF statement criteria that is used if a user specified
”field.any” in a behavior in the User-Defined Context file.
Figure 3.72: ETW Class Template - Behavior IF Criteria Template (Named Field)
Figure 3.73: ETW Class Template - Behavior IF Criteria Template (Any Field)
Once the IF statement criteria is generated for a single field that a user specifies in
a configured behavior, the generation code continues creating criteria and building a full
criteria string until all fields that a user has configured for a behavior have been added
to the string. When building a multi-field criteria string, the generation code simply uses
the necessary criteria template (named field or non-named field) repeatedly and places
”&&” symbols between each for the C# ”AND” condition. If a user specifies to use a
specific construct for a field, these values and resulting criteria templates are wrapped in
parentheses with ” ” symbols placed between them because construct values are treated
as ”OR” conditions. For the cases when a user has negated a specific field value in a
138
behavior, the generation code simply uses a di↵erent criteria template with a ”!” symbol
injected. Accounting for all of these conditions allows the generation code to build complex
IF statement criteria dynamically based on any combination of field definitions the user
configures for a behavior in the User-Defined Context file. Once a full criteria string
is built, the string is placed into an IF statement block template. The IF statement
templates represent the entire behavior that a user has configured and include all logging
and alert noise suppression functionality. Figure 3.74 illustrates the IF statement template
prior to being generated and Figure 3.75 illustrates the same IF statement block after it
has gone through the class generation process and contains criteria for a behavior defined
in the User-Defined Context file.
Figure 3.74: ETW Class Template - Behavior IF Block Template
139
Figure 3.75: ETW Class Template - Behavior IF Block Template
Once an IF statement block has been created from the template and is ready to be
put into the ETW class template, the class generation code locates the string ”//AN-
CHOR:BEHAVIORS” within the class string and places the generated behavior IF state-
ment block beneath this string. These strings are used as anchors so that the generation
code can reliably find their placement within the class string as other elements are added
to the string during the generation process. This behavior anchor is located within a be-
havior checking function where all ETW events are sent after they are consumed from the
subscription pipeline. Figure 3.76 illustrates the behavior checking function that exists
within the overall ETW class template section where behavior IF blocks are placed during
the behavior generation process.
140
Figure 3.76: ETW Class Template - Behavior Check Function Template
Once all of the necessary variables and behavior IF blocks have been added to an
ETW class template that is under construction, the template string is added to a final
.NET Dictionary named ETWProviderClassesToGenerate that is used during the .NET
Reflection class compilation and execution process. As noted earlier, user provider classes
are individually generated for each user provider that a user specifies in the User-Defined
Context file and only a single kernel provider class is generated for all kernel-level teleme-
try sources a user configures. The single kernel provider class that is generated is also
added to this dictionary so that class compilation of all dynamically generated ETW
classes can occur in one process. Figure 3.77 illustrates the end of the class generation
process and the resultant entry of a fully generated class string entered into the ETW-
ProviderClassesToGenerate dictionary.
141
Figure 3.77: ETW Class Generation Process - Add to Dictionary for Compilation
Once all classes are loaded into the ETWProviderClassesToGenerate dictionary, the
artifact is ready to call the .NET Reflection code to compile, load, and execute the classes
as individual threads. Within the software artifact code-base, a separate class is used
to execute this process with a function named CompileAndLoadETWClasses within the
EDAC Dynamic Code Gen.cs class. The code in this class file was developed from official
Microsoft documentation on .NET Reflection methods as well as open-source examples
on how to compile C# code from a source code string. The primary open-source example
used was one developed by Strahl [61]. Once each class is compiled successfully without
errors, the classes are executed by instantiating an instance of the Go() function within
the compiled class. This Go() function serves to maintain a common entry point for all
generated ETW classes and starts the primary ETW subscription function in a separate
thread. Once the ETW subscription pipeline is active, the compiled class will start
sending events to the dynamically generated behavior IF statements generated earlier for
behavior checking. Figures 3.78 and 3.79 illustrate the Go() function that is called within
the loaded ETW class as well as the CompileAndLoadETWClasses function to compile,
load, and execute the Go() function of the ETW class.
142
Figure 3.78: ETW Class Template - Go() Function
Figure 3.79: ETW Class Generation Process - Compilation and Execution
From a user interface perspective, the user does not need to perform any additional
actions to complete the ETW class compilation process other than confirm which classes
will be compiled and verify the output sent to the console after the compilation/loading
process completes. The compilation, loading, and Go() function processes each have
143
distinct log messages that indicate whether the process for each compiled class succeeded
or failed. Figures 3.80 and 3.81 illustrate the final two user interfaces before and after
class compilation.
Figure 3.80: EDAC User Interface - ETW Classes Ready to Compile
Figure 3.81: EDAC User Interface - ETW Classes Compiled and Executed Successfully
3.3.3.3 ETW Event Subscription in .NET - User & Kernel Providers
The primary code used to subscribe to user providers and kernel providers defined in
the User-Defined Context file was adapted from examples provided by Microsoft [62][63]
144
within their Perfview performance analysis tool Github repository on how to subscribe
to ETW providers using C#/.NET code. It was identified during development that
between the provided examples, user-level ETW providers and kernel-level ETW providers
have di↵erent syntax to subscribe to events. This was one of the contributing factors to
keeping user and kernel ETW provider subscription within separate class files during the
class generation process. For either case however, both ETW class templates have a
single ETW subscription function named SubscribeToEvents() that contains the specific
user/kernel subscription code in it along with template variables that are filled in during
the class generation process. This function’s purpose is to subscribe to the user-defined
ETW provider(s), receive incoming ETW events, and pass them o↵ to a separate parsing
function discussed in the next section. Figures 3.82 & 3.83 illustrate the code used for
subscribing to user providers and kernel providers respectively.
Figure 3.82: EDAC ETW Provider Subscription Code (User)
145
Figure 3.83: EDAC ETW Provider Subscription Code (Kernel)
3.3.3.4 ETW Event Processing - User & Kernel Providers
Once events are consumed by their respective functions detailed in the previous section,
they are sent to a function named XMLParser() to parse their native XML format coming
from the Windows operating system into a JSON object containing all event fields and
values for behavior checking. The XMLParser function code was adapted from the im-
plementation used in the SilkETW project [64] where incoming event data is first parsed
using built-in .NET XML syntax parsing functions, then parsed into a JSON object using
a third-party JSON parsing library named Json.NET built by NewtonSoft [65]. Addi-
tional functionality developed in this function includes normalization of JSON fields that
are expected for behavior checking. For example, not every ETW event that is consumed
146
from user or kernel providers includes a ”username” field, therefore, this field is added to
the created JSON object so that users can define username behavior criteria in the User-
Defined Context file. Similarly, some fields coming from ETW event providers are parsed
by the JSON parser into a specific level in the JSON object tree and need to be copied/-
moved into a known level of the tree for faster behavior checking (example: jsonObject-
Name[”Username”] versus jsonObjectName[”XmlEventData”][”Username”]). The code
for the developed XML parsing function is too large to depict in a single image, therefore,
a diagram was created to illustrate all of the actions that are taken on each ETW event
that is sent to the parser, as well as what happens to the event once it is passed to the
BehaviorChecker() function for alerting. This diagram is depicted in Figure 3.84.
147
Figure 3.84: EDAC ETW Event Processing Pipeline
148
3.3.4 Artifact Performance Analysis
As with all software artifacts developed during Design Science Research (DSR) method-
ology projects, performance metrics of the artifact under validation experiments and real-
world conditions are critical to determining whether the artifact solves the stated problem
and can be used by real users in the scientific field. During the development of the soft-
ware artifact, performance was a primary focus in all aspects of the artifact’s functionality.
The body of functionality needing performance analysis encompassed User-Defined Con-
text file parsing, ETW class generation, ETW provider subscription/event processing,
and behavior detection. The most appropriate measurement of performance criteria for
each of these operations as determined by initial tests were measurements of the time
in seconds to complete the entire process based on specific system and input conditions,
CPU utilization, and RAM utilization. The software artifact was developed using Visual
Studio Community 2022. Visual Studio, a Microsoft standard Integrated Development
Environment (IDE), has native functionality to measure the identified performance met-
rics on software developed within the IDE. In order to fully exercise the identified internal
operations of the artifact, one of the validation experiments was chosen to execute with
di↵erent configurations of the constructs and behaviors in the User-Defined Context file.
The chosen validation experiment was Experiment 10 (SMB lateral movement). The rea-
son for choosing Experiment 10 was because it is the validation experiment with the most
complicated configuration in the User-Defined Context file.
The User-Defined Context file for this experiment has five telemetry sources, four con-
structs, and eight behaviors configured to detect all of the techniques in the experiment.
Measuring the artifact performance criteria against a User-Defined Context file with five
telemetry sources, four constructs, and eight behaviors will illustrate how the artifact
performs in the most complicated configuration for the research, however, it does not
test for failure points of the artifact. To test the artifact more thoroughly, an additional
149
User-Defined Context file configuration was used on top of the configuration used for
Experiment 10 to increase the amount of complexity that the identified operations have
to execute. For example, instead of using comma delimited lists of a small number of
objects ( 10) in constructs, the constructs were changed to have a large body of objects
( 1721). The large body of objects chosen to exercise the identified operations was a
baseline of all known executable files on the system prior to executing Experiment 10.
As noted in prior sections, in order to detect potentially malicious executables on the
system, the artifact can check against a list of known executables on the system. On
the Windows 10 endpoints used for validation, this list is roughly 1721 executables long.
Increasing the size of this list that is input into a construct in the User-Defined Con-
text file increases parsing, class generation, and ETW event processing/behavior checking
computational complexity, and would reveal any execution issues in the artifact during
performance analysis of Experiment 10. For the purposes of the performance analysis,
these two configurations were classified as ”basic” and ”advanced” configurations. Each
of the defined workflow operations and their observed performance under the the two
experiment conditions are described in the following sections.
3.3.4.1 Performance of Parsing Operations
The first major operation that the artifact executes during the typical user workflow is the
import and parsing of the User-Defined Context file. As noted in Chapter 3, this process
accepts a file path to the User-Defined Context file, reads all lines of the context file, parses
each line of each section of the file, and stores information in various .NET dictionaries
for processing later in the workflow. In total, CPU utilization for parsing of both basic
and advanced configurations spiked to 25%, RAM utilization peaked at 25MB, and time
elapsed for each operation never exceeded one second. Based on the author’s experience
in working in Information Technology departments in enterprises across industries, all of
these metrics are well within acceptable standards for software agents running on Windows
150
endpoints in enterprises. Table 3.14 illustrates all of the performance measurements and
the collected results.
Table 3.14: Experiment 10 - Artifact Performance Analysis - UDC Parse

Operation with Specific Condi- Time to Com- CPU Utilization for Ex- RAM Utilization In-
tions plete ecuting Function (%) crease
Read all lines of User-Defined Context <1 second 0.00% observed (not enough 0MB increase from 12MB
file (Exp. 10 - basic) complexity to measure) program start baseline
Read all lines of User-Defined Context <1 second 0.00% observed (not enough 0MB increase from 12MB
file (Exp. 10 - advanced) complexity to measure) program start baseline
Parse & store Telemetry section <1 second 1.38% (function measure- 0MB increase from 12MB
(ParseTelemetrySection()) with 5 ment by VS2022) program start baseline
providers (Exp. 10 - basic)
Parse & store Telemetry section <1 second 0.66% (function measure- 0MB increase from 12MB
(ParseTelemetrySection()) with 5 ment by VS2022) program start baseline
providers (Exp. 10 - advanced)
Parse & store Construct section (Par- <1 second 10.34% (function measure- 9MB increase from 12MB
seConstructSection()) with 4 con- ment by VS2022) program start baseline
structs containing  10 items (Exp. 10
- basic)
Parse & store Construct section (Par- 1 second 20.53% (function measure- 13MB increase from 12MB
seConstructSection()) with 4 con- ment by VS2022) program start baseline
structs containing  1721 items (Exp.
10 - advanced)
Parse & store Behavior section (Par- <1 second 10.34% (same function as 0MB increase from 21MB
seConstructSection()) with 8 behaviors construct parsing) total in last step
referencing construct lists of  10 items
(Exp. 10 - basic)
Parse & store Behavior section (Par- 1 second 20.53% (same function as 0MB increase from 25MB
seConstructSection()) with 8 behaviors construct parsing) total in last step
referencing construct lists of  1721
items (Exp. 10 - advanced)
3.3.4.2 Performance of ETW Monitoring Class Generation & Compilation
The second major operation that the artifact executes during the typical user workflow
is the ETW class generation process. During this process, configurations from the User-
Defined Context file are used to dynamically generate behavior checking conditions within
C# class templates to be compiled. The class templates are then compiled and executed
to begin the process of receiving events and checking events against defined behaviors. In
total, CPU utilization for generation of user provider classes and the single kernel provider
class in both the basic and advanced configurations spiked to 53.88%. RAM utilization
peaked at 34MB, and time elapsed for each operation never exceeded one second. All
151
of these metrics are still well within acceptable standards for software agents running on
Windows endpoints in enterprises. Short duration spikes in CPU utilization are often
not noticeable by end users, however, sustained spikes in CPU utilization can become
noticeable. Table 3.15 illustrates all of the performance measurements and the collected
results.
Table 3.15: Experiment 10 - Artifact Performance Analysis - Class Generation

Operation with Specific Con- Time to Com- CPU Utilization for Ex- RAM Utilization In-
ditions plete ecuting Function (%) crease
Generate and store user provider <1 seconds 5.10% (function measure- 6MB increase from 21MB
classes (GenerateUserProvider- ment by VS2022) baseline (after parsing)
Classes()) (Exp. 10 - basic)
Generate and store user provider 2 seconds 53.88% (function measure- 12MB increase from 21MB
classes (GenerateUserProvider- ment by VS2022) baseline (after parsing)
Classes()) (Exp. 10 - advanced)
Generate and store kernel <1 second 4.46% (function measure- 2MB decrease from 27MB
provider class (GenerateKer- ment by VS2022) baseline (after user class
nelProviderClass()) (Exp. 10 - generation)
basic)
Generate and store kernel 1 second 22.75% (function measure- 0MB increase from 32MB
provider class (GenerateKer- ment by VS2022) baseline (after user class
nelProviderClass()) (Exp. 10 - generation)
advanced)
3.3.4.3 Performance of ETW Class Compilation, Event Consumption, & Be-

havior Checking
The third major operation that the artifact executes during the typical user workflow
is the subscription, consumption, processing, and behavior checking of events streaming
from the defined ETW provider. These operations represent the bulk of the artifact’s
capability. It should be noted that research by the author and Bhaskar Rimal (DSU) on
improving the performance of behavior checking against large lists of construct items is
occurring concurrently with the research presented in this document. Initial performance
tests of subscribing to a single ETW provider, and checking events against a list of 1700
items in a construct resulted in sustained CPU utilization greater than 50% for longer
than 30 seconds. This was caused by the exponential computational complexity created by
152
putting 1700 items into the criteria that is dynamically generated for IF statement blocks
in the class generation process. The code responsible for creating IF statement blocks
in the class generation process creates IF statement criteria that is O(N) in complexity.
Therefore, construct lists that are large in size will increase the computational complexity
to O(N), where N is the size of the construct list. Performance metrics such as this are not
usually acceptable for production environments, therefore, an additional research project
was created by the author and Rimal to explore ways to improve these measurements.
This branched research project will be completed and published after this writing.
Given the existing performance issue of large construct behavior checking, the ad-
vanced configuration performance test was not executed during the performance analysis
period. Only the basic configuration, as executed during validation Experiment 10, was
executed and measurements collected. This is still useful to the research as it illustrates
the performance of subscribing to multiple ETW providers, processing events from these
providers, checking all incoming events for defined behavior criteria, and generating alerts
to the console under a real-world attack stimulus. Two points should be noted for the
performance results of Experiment 10 in a basic configuration. First, attack stimuli for
Experiment 10 occur in quick succession and may result in di↵ering CPU utilization mea-
surements because each telemetry source and included behavior checking functions are
executing within their own thread concurrently with other telemetry sources. Second,
CPU utilization measurements represent the total peak utilization during the analysis
period. This means that if events coming into a particular telemetry source class are pro-
cessed at the same time, the measured CPU performance for checking the events against
defined behaviors may be higher than if the events came in at separate times from the
operating system. The results should be considered against these factors and taken as a
general measurement of peak performance requirements and not an exhaustive measure-
ment of all permutations of system conditions. In total, CPU utilization for execution
of all atomic techniques in Experiment 10 spiked to 37.57%, RAM utilization peaked
153
at 47MB, and time elapsed for each operation never exceeded one second. All of these
metrics are well within acceptable standards for software agents running on Windows
endpoints in enterprises. Table 3.16 illustrates all of the performance measurements and
the collected results.
Table 3.16: Experiment 10 - Artifact Performance Analysis - ETW Consump-

tion/Behavior Check
Operation with Specific Con- Time to Com- CPU Utilization Exe- RAM Utilization In-
ditions plete cuting Function (%) crease
Compile & execute 4 user 4 seconds 10.37% (function measure- 15MB increase from 32MB
provider classes and 1 kernel ment by VS2022) baseline (after class genera-
provider class tion)
Kernel ETW Class - XML- 1 second 30.60% (function measure- 0MB increase from class
Parser() - Parse all events ment by VS2022) compilation/execution
Kernel ETW Class - Behav- <1 second 0.27% (function measure- 0MB increase from class
iorChecker() - Check for 3 behav- ment by VS2022) compilation/execution
iors
Kernel ETW Class - Logger() - <1 second 0.07% (function measure- 0MB increase from class
Log confirmed behaviors ment by VS2022) compilation/execution
SMB ETW Class - XMLParser() 1 second 8.95% (function measure- 0MB increase from class
- Parse all events ment by VS2022) compilation/execution
SMB ETW Class - Behav- <1 second 0.12% (function measure- 0MB increase from class
iorChecker() - Check for 3 ment by VS2022) compilation/execution
behaviors
SMB ETW Class - Logger() - Log <1 second 0.12% (function measure- 0MB increase from class
confirmed behaviors ment by VS2022) compilation/execution
PowerShell ETW Class - XML- 1 second 37.57% (function measure- 0MB increase from class
PowerShell ETW Class - Behav- <1 second 0.42% (function measure- 0MB increase from class
ior
PowerShell ETW Class - Logger() <1 second 0.08% (function measure- 0MB increase from class
- Log confirmed behaviors ment by VS2022) compilation/execution
Services ETW Class - XML- 1 second 9.91% (function measure- 0MB increase from class
Services ETW Class - Behav- <1 second 0.11% (function measure- 0MB increase from class
ior
Services ETW Class - Logger() - <1 second 0.11% (function measure- 0MB increase from class
Log confirmed behaviors ment by VS2022) compilation/execution
3.3.4.4 Artifact Performance Discussion
As noted in the prior sections, all operations executed by the artifact during validation
experiments did not experience any significant performance issues other than the identified
154
issue with large list constructs that is being fixed in the performance research conducted
by the author and Bhaskar Rimal (DSU). For each of the atomic operations measured,
no operation exceeded one second in time to complete, and no operation exceeded 50% in
sustained CPU utilization. While some operations experienced spikes in CPU utilization,
these spikes quickly reduced after the time to complete had elapsed. For a software artifact
that is consuming hundreds of ETW events in real-time and making decisions on whether
field values within each event matches complex criteria, these performance results are very
promising for the future of attack technique detection using kernel/user event tracing.
3.3.4.5 Special Notes on Resiliency Against ETW Evasions
As stated in the design requirements for the artifact in this research, the artifact needed to
be fault-aware with respect to ETW event pipeline health as any active evasion attempt
against the software could result in a failed detection of a particular technique. It should
be noted for users of the developed artifact that o↵ensive techniques that evade or disable
components of defensive software are developed frequently. Therefore, the resiliency com-
ponents currently built into the software artifact are meant to illustrate how resiliency
against current/known techniques can be built and not a guarantee that the current com-
ponents will be resilient against all possible current or future techniques. While evasion
resistance was not one of the original criteria that was designated for validation of the
software artifact, the initial capabilities developed are important to note as defensive tool
evasion is a popular topic amongst o↵ensive technique research.
As with all defensive cyber capabilities and techniques that are developed, there are
o↵ensive techniques developed to counter the developed defensive techniques. ETW event
consumption for the purposes of defense is not a new topic and has been accomplished
by many security researchers noted in Chapter 2. One of the most common techniques
to defeat ETW event consumption is to prevent applications from sending events to the
Windows operating system. User-mode applications have a few mechanisms that are used
155
to send ETW events to the operating system. As noted by MDSec [66] in a blog post
about how to hide malicious .NET code from being detected with the .NET Runtime
ETW provider, the ”EtwEventWrite” API function located in ntdll.dll is a function that
is used by user-mode applications to send events to the operating system. MDSec also
noted that this function can be overwritten with a return (RET) instruction in ntdll.dll in
each attacker-controlled process to prevent the user-mode application from writing ETW
events to the operating system to distribute to subscribers.
As an illustrative example on how to counter this o↵ensive technique, a function was
written in the developed software artifact that iterates through every running process on
the system, identifies the address of the EtwEventWrite function inside ntdll.dll, checks
the assembly opcode bytes at the starting address, compares the bytes with a list of
known opcode bytes that are expected to be at that address, and alerts to the console if
there are any deviations from what is expected. This function executes periodically to
ensure that during the operation of the artifact, the defensive telemetry landscape does
not change. Figure 3.85 illustrates the code to the function that was written to perform
this health checking. To simulate the o↵ensive technique developed by MDSec [66], a
debugger was used to locate the address of the EtwEventWrite function in an example
PowerShell process, and the bytes at the starting address were changed to a return (RET).
Figure 3.86 illustrates this process. Once the bytes were changed at the function address,
the health function identified the change on the next health check interval and alerted
the user on the main console. Figure 3.87 illustrates the alert in the console. The design
philosophy with this was to make ETW logging pipeline health messages as important as
alerts on attack techniques. The reason is because without the user knowing the health
status of the ETW logging pipeline, the same implicit trust discussed in Chapter 1 can
occur and attack techniques could be missed.
156
Figure 3.85: EDAC ETW Health Check Function
Figure 3.86: Example Overwrite of Bytes at EtwEventWrite
157
Figure 3.87: EDAC User Interface - Tamper Warning for EtwEventWrite Overwrite
158
Chapter 4
Validation & Results
4.1 Validation Goals
Given that this research uses the Design Science Research (DSR) methodology, one of
the most important aspects of this methodology besides developing an artifact that is a
useful contribution to the field is the validation of a developed artifact. The overarching
goals of validation within DSR are to ensure that developed artifacts are tested and eval-
uated against real-world conditions before they are released into production. The chosen
empirical research method for this research was the Single-Case Mechanism Experiment
within DSR. This research method uses iterative experimentation on a software artifact
to identify whether specific experiments succeed or fail. For the validation experiments on
the artifact in this research, the success/failure measurement criteria focused on whether
the developed artifact, and relevant User-Defined Context file configuration, resulted in
a detection of the atomic techniques for each experiment. Having these results allowed
conclusions to be drawn on the artifact’s usefulness to the field and whether improvements
need to be made prior to releasing it into production environments.
4.2 Validation Design
This research focuses on developing an artifact that is designed to detect cyber attack
techniques across various categories of the MITRE ATT&CK framework. Therefore, the
most appropriate type of validation experiments that should be conducted are execution
159
of individual attack techniques with measurement of whether the artifact is able to alert
on them or not. As noted in the background research section, initial research on attack
technique artifact coverage within ETW event data was required prior to development
of the main artifact and User-Defined Context file syntax. As such, the attack tech-
niques that would be used in the validation experiments for the software artifact in this
phase of the research were developed during this initial background research phase. The
techniques identified and developed cover the majority of the MITRE ATT&CK frame-
work categories, such as, Initial Access, Execution, Persistence, Credential Access, Lateral
Movement, and Command and Control (C2). The specific techniques in each experiment
were chosen due to their impactful and strategic properties that contribute to a typical
attacker’s overall goals, and have been used by the author during real-world penetration
testing, red teaming, and cyber threat emulation engagements.
However, the attack techniques developed for the validation experiments represent
only one component of the validation phase. In order to sufficiently test the software
artifact against real-world conditions, the software artifact itself needed to be running
on an endpoint that represented real-world attack/defense conditions. As noted in the
introduction, the author has a significant amount of experience designing security ar-
chitectures in government and commercial systems and has observed numerous system
configurations that experience challenges in detecting attack techniques at the endpoint.
For example, current-generation endpoint defense solutions have a combination of local
detection features as well as cloud-level detection features that occur after an endpoint
has sent telemetry to the vendor for processing and analytics. While these capabilities
are useful for endpoints that are internet connected 100% of the time, many other system
architectures in government and commercial systems are not able to take advantage of
this level of connectivity due to policy or security requirements. This immediately reduces
the overall capability of the majority of commercial endpoint solutions and illustrates the
need for high-fidelity local detection of attack techniques. This experience working with
160
disconnected/air-gapped/special security systems formed the basis for development of the
validation architecture that was used for the validation experiments.
4.3 Validation Architecture
The attack techniques chosen for execution within the validation experiments are all
atomic techniques that get executed on an endpoint in some way. Some techniques are
executed solely on the endpoint such as PowerShell commands, simulated malware bina-
ries, etc., and other techniques require interaction with a Command and Control (C2)
server. For this research, a small self-contained network of virtual machines was setup
to accomplish all of the experimentation necessary for validation. On the attack side, a
single Kali 2022.4 virtual machine was used to serve the role of a Command and Control
(C2) server for all techniques that required C2 channels. This system also played the
role of a simulated rogue device planted in a network for all techniques where an inbound
lateral attack was necessary. On the victim side, three Windows 10 22H2 systems with
specific system configurations and characteristics were used to conduct all of the valida-
tion experiments against. Figure 4.1 illustrates the validation architecture used for this
phase of the research.
161
Figure 4.1: EDAC Validation Architecture
Given that the developed software artifact is designed to operate on Windows systems,
Windows was the operating system chosen for the validation experiments. Windows 10
was chosen as the operating system version because the majority of Windows deployments
world-wide still use Windows 10 and have not yet adopted Windows 11. A well-known
version statistic platform named StatCounter, who collects statistics of visiting operating
systems on 1.5 million websites, indicates that in 2022, roughly 70% of the operating
systems traversing their sensors were Windows 10 [67]. Figure 4.2 illustrates the Windows
10 statistic versus the other Windows versions across 2022. For the Windows 10 systems
used in the validation experiments, build 22H2 was used as this was the most recent build
as of this writing.
162
Figure 4.2: Windows Version Distribution 2021-2022 by StatCounter [67]
4.3.1 Representative System Configurations
As noted in the validation introduction, the Windows 10 systems used for the validation
experiments were specially configured to model representative installation environments.
These configurations cover many classes of system architectures across government and
commercial organizations of all sizes. Experimenting with these di↵erent classes of envi-
ronments will sufficiently determine if the artifact can be used in real-world configurations
and to what extent the artifact enables defense against the attack techniques. It should
be noted, however, that the Windows 10 configurations used in the validation experiments
will not be fully representative of any specific organization or group of organizations in the
government or commercial sectors. Each organization that uses any of the three classes
of systems represented in this research will likely have specific software that is installed
on systems, specific Windows Group Policy security configurations, and other system
security configurations tailored to their organizational needs.
To cover as many organizational deployments as possible and to reduce the likelihood
of interference in the detection of techniques, it was decided that all of the Windows 10
163
system configurations used for the experiments would have minimal security configurations
and software installed. For example, Windows Defender was turned o↵, the Windows
Firewall was turned o↵, and no Group Policies were configured on the systems. The
purpose of the experimentation was to determine if the software artifact can detect all or
part of an attack technique executed. If Windows Defender or any special Group Policy
configuration blocked all or part of a technique in an experiment, this would interfere with
the results of the experiment and a↵ect the final conclusions on the research. The purpose
of the system configurations is to exhibit specific high-level characteristics described in
the following sections that sufficiently change the context/purpose of the system, for the
purposes of detecting the chosen techniques.
4.3.1.1 Windows 10 Standard Endpoint - Fully Internet Connected
The first system configuration chosen for the validation experiments was a standard Win-
dows 10 endpoint that has a primary characteristic of being internet connected 100% of
the time. This system could be considered as the standard endpoint that is deployed in
an organization for most organizational units. Being connected to the internet 100% of
the time introduces significant challenges for defense and detection of techniques on the
system. First, the primary nature of the system is to be open from a network perspective,
which means that all applications and processes on the system may connect to internet
IPs at any time and it requires defense to then sift through all of these connections and
determine which are normal/abnormal. From an attacker perspective, these systems are
the easiest to command and control if they are compromised due to the open-nature of
the network connectivity.
An additional characteristic of this system is that the system was not joined to a
Microsoft Active Directory domain. Microsoft Active Directory is Microsoft’s native solu-
tion that organizations use for authentication, authorization, and accounting of user and
computer objects [68]. The reasons this system configuration was not joined to an Active
164
Directory domain are two-fold. First, no attack techniques in the attack experiments
focus on any aspect of authentication, authorization, or accounting at the domain-level.
While some techniques focus on authenticating to the system, such as lateral movement,
the Windows SMB ETW provider and the User-Defined Context file allow the user to
configure any set of local or domain usernames as criteria to detect techniques. Second,
the author has experience with many small and medium sized businesses that do not need
or use Active Directory due to the complexity required to manage it. This introduces a
very large distribution of Windows systems that are deployed in the real-world that uti-
lize local user accounts, have no security software installed, or have no security-focused
Group Policy configurations, just like this system configuration in this research. For the
purposes of the experiments on this system, a list of system parameters and their expect-
ed/abnormal behaviors was generated to develop the specific configuration that would go
into the User-Defined Context file. Table 4.1 illustrates these system parameters used for
this system configuration.
Table 4.1: Windows 10 Standard Endpoint - System Parameters

System Parameter Expected Behavior Abnormal Behavior
Internet connectivity Network/application connections to internet Connections to malicious domains/IPs are
IPs/domains are normal abnormal
LAN connectivity No internal lateral network connections are Any connections laterally to/from internal
made IPs are abnormal
Command/script execution System does not normally use commands or Execution of any command/script is abnor-
scripts for normal operation mal
System is built from a gold No auto-start mechanisms (services, sched- Any changes to auto-start mechanism (ser-
image uled tasks, registry run keys) are changed af- vices, scheduled tasks, registry run keys) are
ter system is built abnormal
System is built from a gold No additional executables should be intro- Any executables added to the system are ab-
image duced after system is built normal
Low privilege user logins Low privilege user logins only occur on the Logins from any other account or method
console with their local accounts other than the console are abnormal
High privilege user logins High privilege user logins only occur on the Logins from any other account or method
console with local administrative accounts other than the console are abnormal
4.3.1.2 Windows 10 Edge Compute Endpoint - DDIL Network
The second system configuration chosen for the validation experiments was influenced
by the author’s professional experience working with ”edge” computers and systems in
165
government and commercial organizations. The term ”edge” typically refers to computer
systems that are ”close to the user” wherever the user may be in the world physically.
RedHat indicates [69] in an article about edge computing for the U.S. Department of De-
fense that ”edge” computing is not a place, but rather, where the data is produced at the
end of the network. An ”end” of a network may be far away from a standard datacenter
geographically that connects back to this datacenter over a range of network connections
such as satellite or cellular. There is one primary characteristic of edge computing that
formed the basis of this experiment configuration named Denied, Disrupted, Interrupted,
and Limited (DDIL) connections. DDIL typically describes network connections for sys-
tems that may be denied, disrupted, interrupted, and/or limited.
In the context of edge computing, since edge computers and systems may exist in
adverse condition environments, their connections back to centralized datacenters may
experience DDIL conditions frequently. This presents challenges for detecting attack
techniques on edge systems as the network connection to cloud services used by tradi-
tional endpoint defense solutions (EDR, EPP, XDR) may not be reliable or available for
extended periods of time. Local defense on the system is the only feasible way to detect
attack techniques by highly integrated and positioned attackers. For the purposes of the
validation experiments, the Windows 10 system was the same configuration as the pre-
vious configuration, except the system operates under the guise that it was in a DDIL
environment and certain parameters existed that determine expected behaviors of the
system. These parameters and expected behaviors are detailed in Table 4.2.
166
Table 4.2: Windows 10 Edge Compute (DDIL) Endpoint - System Parameters
Internet connectivity Network/application connections to known Connections to unknown domains/IPs are
internet IPs/domains are normal abnormal
LAN connectivity Network connections to/from known server Connections to/from unknown internal IPs
IPs are normal are abnormal
Command/script execution System does not normally use commands or Execution of any command/script is abnor-
scripts for normal operation mal
4.3.1.3 Windows 10 Air-Gapped Endpoint - No Internet Connectivity
The third and final system configuration chosen for the validation experiments was influ-
enced by the author’s professional experience working with fully disconnected Windows
endpoints, also called ”air-gapped” systems. The term ”air-gap” refers to the operational
goal that these systems are physically disconnected from other networks. These types of
systems exist in government and commercial organizations for a variety of use-cases. The
primary characteristics of these systems are that they are fully disconnected from internet
connectivity, disconnected from other networks in an organization, and serve critical roles
where the consequences of compromise are higher than any other system configuration.
From an attacker perspective, these systems are the most sought after, but the most dif-
ficult to compromise due to their disconnected nature. One of the most common systems
that match these operational characteristics are those systems that operate in Industrial
Control Systems (ICS) such as manufacturing plants, chemical plants, oil & gas plants,
electrical utilities, and water utilities.
From a defense perspective, system configurations that are designed to do a single
or small set of functions typically have little user activity, little network activity, etc.
These characteristics result in easily identifiable normal activity from abnormal activity.
167
This is because the system context/purpose is so constrained that any deviation from
this pre-set configuration may be more apparent than other system configurations. With
respect to the internet connection characteristic, the Standard Endpoint configuration
would typically allow an attacker to establish a Command and Control (C2) channel
over many di↵erent protocols to any IP the attacker needs. For the Edge Compute and
Air-Gapped configurations, the systems have limited to no internet connectivity which
drives the direction of the attacker’s Command and Control (C2) capabilities to other
internal systems that can laterally communicate with this system. This immediately
allows defenders to shift focus of detection of these techniques to network connectivity
with other internally adjacent systems, since the primary connectivity path attackers use
for C2 is no longer available. Table 4.3 illustrates the parameters set for this system
configuration.
Table 4.3: Windows 10 Air-Gapped Endpoint - No Internet Connectivity

Internet connectivity No outbound network communications to Any outbound connections to internet IPs
any internet IP are abnormal
LAN connectivity No internal LAN communications to IPs Any connections laterally to/from internal
other than what is normal to operate the sys- IPs are abnormal
tem
Command/script execution System does not execute commands or Execution of any command/script is abnor-
scripts during normal operation mal
4.3.2 Validation Experiments
The design philosophy for the attack techniques in the validation experiments was to
select as many of the most impactful techniques in as many categories of the MITRE
ATT&CK framework as possible, but still maintain realism and relevance to the current
168
cyber threat landscape. This materialized into picking techniques that the author has
used in penetration testing professional experience, techniques that are known to be used
by high-profile cyber threat groups, and techniques that are typically not detected by
current-generation endpoint solutions. In addition to these criteria, an assessment was
made on the selected techniques to identify if they can be logically grouped together into
one cohesive chain. This ensured that the techniques and the overall chain of techniques
maintained a level of realism that a real attacker could capitalize on during a real attack.
Figure 4.3 illustrates all of the techniques chosen and where they lie within the MITRE
ATT&CK framework as well as the interconnected relationship of the other techniques.
Any technique that is connected by a line to another technique means that the next
technique could be executed as a result of accomplishing the prior technique, or the
prior chain of techniques. This progression models the real behavior of cyber threats
where technique chains follow a logical progression strategically, but do not necessarily
use a technique in every category of the framework. Many attack chains capitalize on
techniques that succeed in the moment and skip categories to the next technique that
succeeds and supports the overall mission goals.
169
Figure 4.3: Attack Technique Path Across the MITRE ATT&CK Framework
4.4 Validation Results
The following sections detail the results of each attack technique experiment and their
subordinate atomic techniques. Each section details specific User-Defined Context file
configurations needed for detection and whether an alert was received in the console on a
specific atomic technique. It was identified during the experimentation that each system
configuration experienced very similar detection/no detection results for each experiment.
This is not unexpected as the system parameters defined for each system configuration
were similar across all configurations. The parameters for each system do have some
di↵erences, however, they are mostly nuances with conditional logic or wording in the
expected behavior (”some” vs. ”none”, ”known bad” vs. ”unknown”, etc.). For exam-
ple, in the internet connectivity system parameters, the Standard Endpoint dictates that
network connectivity to the internet is normal, but connections to known malicious IPs/-
domains are abnormal. For the Edge Compute Endpoint and the Air-Gapped Endpoint,
connectivity to unknown IPs/domains are considered abnormal. From an attack tech-
170
nique perspective, if an attacker uses a known malicious IP for Command and Control
(C2), all three system configurations will result in a match for abnormal behavior based
on how the parameters are written. These facts changed how the results are conveyed
in the following sections to focusing on detection or no detection of atomic techniques
for all three system configurations rather than documenting the same results for each of
the three system configurations. If there were di↵erences in the experiment results for a
specific configuration, they were noted and are illustrated in the following sections.
4.4.1 Initial Access Category Results
4.4.1.1 Experiment 1 Results - Exploit Network Facing Application (T1190.000)
This section details the results of experimentation for Experiment 1 which includes soft-
ware exploitation over the network of the Vulnserver software on the experimentation
endpoints. In total, 1/5 of the atomic techniques defined for this particular experiment
were detectable by the software artifact using the User-Defined Context file. Figures 4.4,
4.5, and 4.6 illustrate the configuration lines necessary to detect the atomic techniques
as well as the EDAC user interface showing alerts. Table 4.4 illustrates all of the atomic
techniques executed in the experiment and the result of detection/no detection for each
atomic technique of the overall technique.
171
Figure 4.4: Experiment 1 - User-Defined Context File Telemetry Section
Figure 4.5: Experiment 1 - User-Defined Context File Construct Section
172
Table 4.4: Experiment 1 - Software Exploit Experiment Results
Atomic Technique Expected Outcome Observed Outcome
Inbound network connec- Alert - inbound network connections that are Alert received for connection from an IP that
tion to 9999 not 192.168.10.10 was not 192.168.10.10
Socket writes to memory No alert - internal process execution opera- No alert as expected
bu↵er tions are not detectable using ETW data
EIP overwrite, execution No alert - internal process execution opera- No alert as expected
flow deviates tions are not detectable using ETW data
Shellcode begins execution No alert - internal process execution opera- No alert as expected
tions are not detectable using ETW data
MessageBoxA() API func- No alert - most Windows API function calls No alert as expected
tion is called are not detectable using ETW data
Figure 4.6: Experiment 1 - EDAC Alert Console
4.4.1.2 Experiment 2 Results - Spear-Phishing Attachment - Macro Word

Document (T1566.001)
This section details the results of experimentation for Experiment 2 which includes the
execution of the macro-enabled Word Document on the experimentation endpoints. In
total, 4/6 of the atomic techniques defined for this particular experiment were detectable
by the software artifact using the User-Defined Context file. Figures 4.7, 4.8, and 4.9
illustrate the configuration lines necessary to detect the atomic techniques as well as the
EDAC user interface showing alerts. Table 4.5 illustrates all of the atomic techniques
executed in the experiment and the result of detection/no detection for each atomic
173
technique of the overall technique.
174
Figure 4.8: Experiment 2 - User-Defined Context File Construct Section
Table 4.5: Experiment 2 - Macro Word Document Experiment Results

Launch WINWORD.exe Alert - Microsoft Word opened by user Alert received for Microsoft Word launching
WINWORD opens macro- Alert - Microsoft Word opened a macro- Alert received for Microsoft Word opening a
enabled Word document enabled Word document macro-enabled document
Macro executes and iterates No alert - internal macro code execution is No alert as expected
through paragraphs not detectable using ETW data
Macro writes bytes from Alert - Microsoft Word wrote an EXE file to Alert received for Microsoft Word writing an
paragraphs into EXE file a folder EXE file to a folder
Macro executed EXE file No alert - internal macro code execution is No alert as expected
in C:\Windows\Temp with not detectable using ETW data
ShellExecute()
Malware macro.exe Alert - Microsoft Word launches an EXE as Alert received for Microsoft Word launching
launched by WINWORD a child process an EXE as a child process
from C:\Windows\Temp
175
4.4.2 Execution Category Results
4.4.2.1 Experiment 3 Results - Command/Script Interpreter - PowerShell

(T1059.001)
This section details the results of experimentation for Experiment 3 which includes execu-
tion of a PowerShell command that includes a Meterpreter C2 channel embedded within
it. In total, 7/7 of the atomic techniques defined for this particular experiment were de-
tectable by the software artifact using the User-Defined Context file. Figures 4.10, 4.11,
and 4.12 illustrate the configuration lines necessary to detect the atomic techniques as
well as the EDAC user interface showing alerts. Table 4.6 illustrates all of the atomic
176
177
Figure 4.11: Experiment 3 - User-Defined Context File Behaviors Section
178
Table 4.6: Experiment 3 - PowerShell Command Experiment Results
Powershell.exe is launched Alert - PowerShell opened by user Alert received for PowerShell launching
PowerShell interpreter de- Alert - PowerShell is decoding an encoded Alert received for PowerShell executing de-
codes encoded commands command code commands
PowerShell calls WinAPI Alert - PowerShell called the Windows API Alert received for PowerShell calling Virtu-
to allocate memory (Virtu- for VirtualAlloc alAlloc
alAlloc)
PowerShell copies payload Alert - PowerShell copied bytes from a man- Alert received for PowerShell calling Mar-
bytes into allocated memory aged array to an unmanaged memory pointer shal.Copy function
PowerShell calls WinAPI to Alert - PowerShell called the Windows API Alert received for PowerShell calling Virtu-
set protection on memory for VirtualProtect alAlloc
(VirtualProtect)
PowerShell calls WinAPI to Alert - PowerShell called the Windows API Alert received for PowerShell calling Cre-
execute thread of copied for CreateThread ateThread
code (CreateThread)
Command and Control (C2) Alert - Outbound network connection made Alert received for PowerShell making an out-
connection is made from from Powershell.exe bound network connection
Powershell.exe
179
4.4.2.2 Experiment 4 Results - User Execution - Malicious File (T1204.002)
This section details the results of experimentation for Experiment 4 which includes exe-
cution of the simulated malware binary on the experimentation endpoints. In total, 1/5
of the atomic techniques defined for this particular experiment were detectable by the
software artifact using the User-Defined Context file. Figures 4.13 & 4.14 illustrate the
configuration lines necessary to detect the atomic techniques as well as the EDAC user
interface showing alerts. Table 4.7 illustrates all of the atomic techniques executed in
the experiment and the result of detection/no detection for each atomic technique of the
180
overall technique.
Figure 4.13: Experiment 4 - User-Defined Context File
181
Table 4.7: Experiment 4 - Simulated Malware Binary Experiment Results
Simulated malware exe- Alert - Unknown binary has executed on the Alert received for an unknown binary exe-
cutable is launched system cuted on the system
VirtualAlloc() is called to No alert - internal process execution opera- No alert as expected
allocated memory inside tions are not detectable with ETW data
process memory
Memcpy is called to copy No alert - internal process execution opera- No alert as expected
embedded shellcode bytes tions are not detectable with ETW data
to allocated memory
Call start of shellcode by No alert - internal process execution opera- No alert as expected
function pointer tions are not detectable with ETW data
NOP + RET shellcode exe- No alert - internal process execution opera- No alert as expected
cutes tions are not detectable with ETW data
4.4.3 Persistence Category Results
4.4.3.1 Experiment 5 Results - Create/Mod System Service - Windows Ser-

vices (T1543.003)
This section details the results of experimentation for Experiment 5 which includes es-
tablishing persistence by creating a Windows Service on the experimentation endpoints.
In total, 4/4 of the atomic techniques defined for this particular experiment were de-
tectable by the software artifact using the User-Defined Context file. Figures 4.15, 4.16,
182
183
Figure 4.16: Experiment 5 - User-Defined Context File Construct/Behavior Section
Table 4.8: Experiment 5 - Service Persistence Experiment Results

Attacker process writes service/pay- Alert - EXE written to disk Alert received for an EXE being written to
load executable to C:\Windows\Temp disk
Service/payload executable launches Alert - Unknown binary has Alert received for an unknown binary execut-
launched ing
Windows service is created by calling Alert - A new service has Alert received for a new service being created
CreateService() API function been created
C2 channel is opened from payload ex- Alert - Network connection Alert received for network connection to un-
ecutable to unknown IP known IP
184
4.4.3.2 Experiment 6 Results - Autostart/Exec - Registry Run Key (T1547.001)
This section details the results of experimentation for Experiment 6 which includes estab-
lishing persistence by creating a value within the registry Run key on the experimentation
endpoints. In total, 2/2 of the atomic techniques defined for this particular experiment
were detectable by the software artifact using the User-Defined Context file. Figures 4.18,
4.19, and 4.20 illustrate the configuration lines necessary to detect the atomic techniques
as well as the EDAC user interface showing alerts. Table 4.9 illustrates all of the atomic
185
186
Figure 4.19: Experiment 6 - User-Defined Context File Behavior Section
Table 4.9: Experiment 6 - Registry Persistence Experiment Results

Attacker process writes payload exe- Alert - EXE written to disk Alert received for an EXE being written to
cutable to C:\Windows\Temp disk
Attacker process sets value for Alert - Run key accessed Alert received for the registry Run key being
HKLM\Software\..\..\..\Run accessed
187
4.4.4 Credential Access Category Results
4.4.4.1 Experiment 8 Results - OS Credential Dumping - LSASS (T1003.001)
This section details the results of experimentation for Experiment 8 which includes dump-
ing credentials out of the LSASS process using Mimikatz on the experimentation end-
points. In total, 4/4 of the atomic techniques defined for this particular experiment were
detectable by the software artifact using the User-Defined Context file. Figures 4.21, 4.22,
188
189
Table 4.10: Experiment 8 - Credential Dump (LSASS) Experiment Results

Attacker process loads vaultcli.dll into Alert - DLL known to be used by Alert received for vaultcli DLL being
process for Mimikatz to function Mimikatz loaded loaded
Attacker process loads samlib.dll into Alert - DLL known to be used by Alert received for samlib DLL being
Attacker process loads cryptdll.dll into Alert - DLL known to be used by Alert received for cryptdll DLL being
Attacker process calls OpenProcess() Alert - LSASS process memory ac- Alert received for an unknown process
WinAPI function with LSASS.exe as cessed reading LSASS process memory
target
190
4.4.4.2 Experiment 9 Results - OS Credential Dumping - SAM File (T1003.002)
This section details the results of experimentation for Experiment 9 which includes dump-
ing local password hashes out of the registry on the experimentation endpoints. In total,
1/1 of the atomic techniques defined for this particular experiment were detectable by the
software artifact using the User-Defined Context file. Figures 4.24 & 4.25 illustrate the
configuration lines necessary to detect the atomic techniques as well as the EDAC user
interface showing alerts. Table 4.11 illustrates all of the atomic techniques executed in
the experiment and the result of detection/no detection for each atomic technique of the
overall technique.
191
Figure 4.24: Experiment 9 - User-Defined Context File
Table 4.11: Experiment 9 - Credential Dump (SAM) Experiment Results

Attacker process writes payload exe- Alert - EXE written to disk Alert received for an EXE being writ-
cutable to C:\Windows\Temp ten to disk
192
4.4.5 Lateral Movement Category Results
4.4.5.1 Experiment 10 Results - Shared Services - Windows SMB Admin

Shares (T1021.002)
This section details the results of experimentation for Experiment 10 which includes exe-
cuting Windows SMB-based lateral movement against the experimentation endpoints. In
total, 8/8 of the atomic techniques defined for this particular experiment were detectable
by the software artifact using the User-Defined Context file. Figures 4.26, 4.27, 4.28, 4.29,
193
194
Figure 4.27: Experiment 10 - User-Defined Context File Constructs Section
195
196
Table 4.12: Experiment 10 - Lateral Movement (SMB) Experiment Results

Inbound network connection to System Alert - System process accepted con- Alert received for connection to System
process over TCP 445 nection over TCP 445 process over TCP 445
Authentication to SMB server as ad- Alert - Successful administrator au- Alert received for successful adminis-
ministrator thentication to SMB server trator login over SMB
SMB server receives request to interact Alert - SMB server received request for Alert received for the svcctl endpoint
with Service Control Manager (”svcctl” svcctl being access over SMB
endpoint)
SMB server receives request to open/s- Alert - SMB server received request to Alert received for Powershell.exe being
tart Powershell.exe open Powershell.exe accessed over SMB
Service Control Manager creates Win- Alert - Service Control Manager cre- Alert received for the Service Control
dows service ated a new Windows service Manager creating a service
Powershell.exe launches from Windows Alert - Powershell.exe launched from a Alert received for Powershell.exe
service Windows service launching from services.exe
Powershell.exe executes commands to *Alerts on all PowerShell techniques *Alerts on all PowerShell techniques
establish Meterpreter C2 from Experiment 3 from Experiment 3
Outbound network connection for C2 Alert - Outbound network connection Alert received for outbound network
to unknown IP connection to unknown IP
197
4.4.6 C2 Category Results
4.4.6.1 Experiment 11/12 Results - Application Layer Protocols - HTTP/HTTPS

(T1071.001)
This section details the results of experimentation for Experiments 11 & 12 which include
establishment of Command and Control (C2) channels over HTTP and HTTPS on the
experimentation endpoints. As noted in the ETW attack technique coverage section, ex-
198
periments 11 & 12 are executed together because the ETW provider Microsoft-Windows-
WinInet extracts HTTP requests from TLS encrypted HTTP sessions in addition to
standard unencrypted HTTP sessions. In total, 4/6 of the atomic techniques defined for
this particular experiment were detectable by the software artifact using the User-Defined
Context file. Figures 4.31, 4.32, 4.33, and 4.34 illustrate the configuration lines necessary
to detect the atomic techniques as well as the EDAC user interface showing alerts. Table
4.13 illustrates all of the atomic techniques executed in the experiment and the result of
detection/no detection for each atomic technique of the overall technique.
Figure 4.31: Experiment 11/12 - User-Defined Context File Telemetry Section
199
Figure 4.32: Experiment 11/12 - User-Defined Context File Constructs Section
Figure 4.33: Experiment 11/12 - User-Defined Context File Behavior Section
200
Table 4.13: Experiment 11 - C2 HTTP/HTTPS Experiment Results
Malware executable launches Alert - Unknown binary executed Alert received for unknown binary exe-
cuting
Shellcode calls LoadLibrary() WinAPI Alert - Process loaded DLL for Alert received for process loading
function to load Wininet.dll HTTP/FTP HTTP/FTP DLL
Shellcode calls InternetOpenA() No Alert - WinAPI function not moni- No alert as expected
WinAPI function to setup connection tored by ETW
parameters
Shellcode calls HttpOpenRequestA() No Alert - WinAPI function not moni- No alert as expected
WinAPI function to create a handle tored by ETW
Shellcode calls HttpSendRequestA() Alert - HTTP connection to unknown Alert received for HTTP connection to
WinAPI function to connect to C2 domain unknown domain
Outbound network connection for C2 Alert - Network connection to unknown Alert received for network connection
IP to unknown IP
Figure 4.34: Experiment 11/12 - EDAC Alert Console
201
4.4.7 Final Experiment Results Discussion
The results of the validation experimentation illustrate that the developed software arti-
fact and User-Defined Context file enable detection of techniques within all of the vali-
dation experiments. Including all atomic techniques within each overall technique there
are a total of 48 atomic techniques. This includes techniques that were expected to be
detected and techniques that were not expected to be detected. The techniques that were
not expected to be detected include techniques that do not have ETW providers that
provide artifacts for detection. Including these techniques, the overall detection ratio is
36/48, where 36 atomic techniques were detected in the validation experiments out of a
total of 48 atomic techniques. If the techniques that cannot be detected using ETW event
data are removed from the total number of techniques, the detection ratio is 36/36, where
all techniques that were expected to produce alerts using the artifact produced alerts as
expected.
In order to fully conclude the success/failure of the artifact with respect to the atomic
techniques that were not expected to be detected, each must be assessed for the un-
derlying cause on why the techniques were not, or could not be detected. Table 4.14
illustrates all of the 12 atomic techniques that were not detected within the larger body
of 12 parent techniques and the underlying cause that was identified for their no detection
result. As shown in the table, the underlying reasons why these atomic techniques were
not detected were that internal process code operations/executions are not logged by any
ETW provider, or a specific Windows API function that was called is not logged by an
ETW provider. These are not limitations with the developed artifact, but rather, limi-
tations with using only ETW event data as the telemetry source. This fact corroborates
why current-generation endpoint solutions use user-mode hooking as a method to obtain
telemetry on internal process function calls, and also corroborates why the recommended
operation of the developed artifact is to operate it in addition to a commercial endpoint
202
solution.
Table 4.14: Techniques Not Detected During Experiments

Experiment # - Parent Sub-Technique Cause for No Detection
Technique
1 - Software Exploit Socket writes to memory bu↵er Internal process code/operation executed is not
logged by an ETW provider
1 - Software Exploit EIP overwrite, execution flow deviates Internal process code/operation executed is not
1 - Software Exploit Shellcode begins execution Internal process code/operation executed is not
1 - Software Exploit MessageBoxA() API function is called Internal process code/operation executed is not
2 - Macro Word Document Macro executes and iterates through Macro code/operation executed is not logged by
paragraphs an ETW provider
2 - Macro Word Document Macro executes EXE file written with Windows API function called is not logged by an
ShellExec() API function call ETW provider
4 - Simulated Malware Exe- VirtualAlloc() is called to allocate Windows API function called is not logged by an
cution memory inside process ETW provider
4 - Simulated Malware Exe- Memcpy is called to copy shellcode into Windows API function called is not logged by an
cution memory ETW provider
4 - Simulated Malware Exe- Call to start of shellcode by function Internal process code/operation executed is not
cution pointer logged by an ETW provider
4 - Simulated Malware Exe- NOP+RET shellcode executes Internal process code/operation executed is not
cution logged by an ETW provider
11 & 12 - HTTP(S) C2 Shellcode calls InternetOpenA() API Windows API function called is not logged by an
function ETW provider
11 & 12 - HTTP(S) C2 Shellcode calls HttpOpenRequestA() Windows API function called is not logged by an
API function ETW provider
Many of the academic research methodologies that include measurements of perfor-

mance for software experiments focus on pass/fail or positive/negative results and the
interpretation of those results. For detecting cyber threats, these kinds of performance
metrics and ratios of detection/no detection can be misleading and not actually be rep-
resentative of the real response that would occur in real environments under attack. For
example, while Experiment 1 (software exploit) only resulted in 1/5 atomic techniques
detected, this does not indicate that the developed software artifact is not e↵ective at
detecting a software exploitation scenario. First, by specifying in a construct in the User-
Defined Context file that only specific IP addresses should be communicating with the
legacy software, the alert on inbound network connections from an untrusted IP imme-
diately elevates the severity of this alert to defenders. One of the primary goals of the
203
software artifact is to allow users to set boundaries at the process-level of what behavior
is normal and abnormal on a system-by-system basis. While an alert on inbound con-
nections to legacy software does not indicate that the legacy software has been exploited,
it gives defenders a chance to investigate in the event that no other detections occur.
The author’s professional experience using commercial endpoint solution software con-
firms that many industry-leading commercial solutions do not alert on seemingly minor
behavioral actions such as inbound network connections. The ability to provide context
with the User-Defined Context file elevates a defender’s perception of the system and
situation.
Additionally, the experiments are executed individually and subsequently analyzed
individually in experimentation research methodology. This does not account for the
totality of the circumstances in a real-world incident as cyber-attacks will likely always be
a body of techniques in order to achieve operational goals. The likelihood of a cyber-attack
including a single technique such as a software exploit and no other post-exploitation
techniques is low. The recommended way to use the developed software artifact and
User-Defined Context file in a real environment is to build behaviors that overlap with
each other, so that, in the event a failure occurs in the design of a single behavior for a
specific category of the MITRE ATT&CK framework, or a telemetry source cannot log
an atomic technique, other behaviors can still detect other subsequent techniques within
an attack chain. For example, many of the atomic techniques executed in the Execution
(PowerShell), Credential Access, and C2 experiments can be configured to be executed
as part of the shellcode within the software exploit experiment. The developed behaviors
in the various validation experiments illustrate that using the User-Defined Context file,
users can configure many behaviors from di↵erent telemetry sources in the same context
file to have a greater likelihood of detecting techniques in a chain.
204
Chapter 5
Conclusions
5.1 Research Summary
The primary goals of this research were to develop a software artifact that detects attack
behaviors/techniques executed by individual processes running on a system, but do so
under specific criteria. The criteria were: the artifact must utilize a user-defined config-
uration file defining the specific behaviors to detect, the artifact must utilize only Event
Tracing for Windows (ETW) event data as the core telemetry source to detect behaviors,
and the artifact must be validated against a real-world attack technique chain. In order
to accomplish this, three core problems needed to be solved before development of an
artifact could commence. For example, the first major problem identified was that, in
order to detect attack techniques using strictly ETW event data, initial background re-
search was needed to identify any gaps in ETW event data for specific attack techniques or
technique categories across the MITRE ATT&CK framework. This inherently introduced
additional problems that needed to be solved, such as, the need for a standardized body
of attack techniques that represent major categories of the MITRE ATT&CK framework,
the need to execute these techniques in an automated way, and the need to iteratively
monitor all available ETW event providers in a Windows operating system while attack
techniques are being executed. This initial background research proved that all developed
attack techniques could be executed iteratively in an automated way, and all attack tech-
niques contained artifacts within ETW event data that could be used for detection of the
techniques.
205
The second overarching problem that needed to be solved was the development of
the user-defined configuration file, named the ”User-Defined Context” file. This context
file needed to have a syntax that solved the readability/usability problems identified in
prior research with Sysmon XML configuration files, but also needed to provide the arti-
fact with all of the necessary inputs to successfully subscribe to defined ETW telemetry
sources, and detect behaviors defined by users. In the end, a custom syntax was devel-
oped for the User-Defined Context file that solved all of the identified problems in prior
research, provides the user a way to granularly define behavioral boundaries for specific
processes on a system, and maintains the highest level of readability. The third overar-
ching problem that needed to be solved in this research was identifying how to subscribe
to all user-defined ETW telemetry sources, detect all user-defined behaviors, and do so
in a dynamic and performant way. The identified method was to generate a single C#
class that included all of the necessary ETW subscription functionality, behavior checking
functionality, and alerting functionality, but make this class file a template that is changed
based on the specific configurations that the user inputs into the User-Defined Context
file. This allowed for the artifact to ingest all configurations in the context file, generate
a fully working C# class as a string of code, compile this class file, and have all ETW
event subscription and behavior checking self-contained in separate execution thread.
During the final validation of the software artifact, the developed User-Defined Context
file syntax and the developed methods for class generation enabled the software artifact
to detect 36/48 atomic attack techniques executed on real-world Windows 10 system
configurations. The remaining techniques in this ratio were expected to not be detected as
they are not detectable within any ETW event providers, therefore, the artifact was able to
detect 36/36 of the known-detectable techniques. Either ratio is a significant achievement
because the detection of these techniques was accomplished without any significant or
sustained spikes in CPU or RAM utilization. Based on the issues with current-generation
endpoint defense solutions that are illustrated throughout this document, these results
206
prove that the artifact solves specific problems within the overarching research problem
context, and the artifact is able to be used in operational environments as-is with no
modifications.
5.2 Answering the Research Questions
At the beginning of this research, four research questions were identified that supported
the overall goal of developing a software artifact that detected attack techniques from a
set of user-defined behaviors using ETW event data. These primary research questions
were:
1. Can real-time process behavior events from multiple Event Tracing for Windows
(ETW) event providers be simultaneously consumed and analyzed programmatically
without user intervention?
2. Can a data schema of ”system behaviors” be created in such a way that A) the syntax
of the schema maps directly to incoming event data from ETW event providers, and
B) the schema syntax is easily configurable by operators?
3. Can the user-defined schema data be consumed by the software in such a way that
the software can quickly identify matching behaviors within incoming events for
alerting?
4. Are there any gaps in ETW event provider data that do not allow user-defined
schema data to sufficiently identify the activity?
At the end of the research, all research questions were answered and confirmed to be
correct. The developed artifact is able to programmatically subscribe to, consume, and
analyze ETW event data from any number of ETW providers defined by a user without
requiring any user interaction on the consumption or analysis of the events. As part of
the artifact development, a User-Defined Context file with a custom syntax was created
207
to enable users the ability to configure telemetry sources to subscribe to, configure con-
structs representing facts and circumstances about the system to consider during behavior
checking, as well as the ability for users to configure behaviors and behavioral boundaries
for individual processes, groups of processes, individual users, or groups of users. The
final component of the artifact development was the development of the ability for the
artifact to dynamically create behavior checking criteria that is fast and results in low
levels of CPU utilization while detecting common attack technique artifacts. The final
question regarding gaps in ETW event coverage resulted in confirmation that gaps do
exist, however, Chapter 4 illustrates that each overarching validation experiment includes
many individual atomic attack techniques. These techniques when alerted on in the con-
text of other behavior alerts still help defenders identify that abnormalities are occurring
within the system and that a defensive response should still happen.
It should be noted that the research questions established at the beginning of this
research did not focus on the variety of problems current generation endpoint defense
solution have that are discussed in Chapter 2. The established research questions fo-
cused strictly on whether it was possible to achieve the core capabilities of developing
a configurable method to detect attack techniques using ETW event data. However,
the final state of the developed artifact includes many features that solve some of the
critical issues identified in research projects detailed in Chapter 2. For example, one of
the core resiliency issues current-generation endpoint defense solutions have is the lack
of fault-tolerance and recovery in the event a core telemetry source or pipeline has been
intentionally defeated. This issue was assessed within the context of the developed ar-
tifact and mitigated using custom code that checks whether the core function used for
ETW telemetry events sent from user-mode applications has been tampered with. This
example capability is not meant to be exhaustive and cover all possible ETW evasion
techniques, however, it is illustrative on how to build in resiliency checking functionality
into an artifact that is meant to provide defensive capabilities. Another identified prob-
208
lem that current-generation endpoint defense solutions have is the inability to change the
semantics and display of alert messages for specific technique detections. This problem
was solved in the software artifact by allowing users the ability to not only define how
a specific behavior should be detected, but how an alert should be displayed in the con-
sole by specifying custom alert strings/meanings and alert message text color. In future
versions of the software artifact, additional capabilities such as these will be added to
increase defender capability in production environments.
5.3 Conclusions on the Hypothesis
The overarching hypothesis for this research was developed from a combination of both
the established research questions, author professional experience in reverse engineering
detection problems in commercial endpoint defense solutions, as well as the signature/de-
tection gaps identified in related open-source research. The hypothesis established for this
research was: A software artifact that is able to successfully analyze low-level kernel/user
events against a highly detailed schema of data defining granular boundaries of behavior
from these events will result in a more complete level of detection of attack techniques in an
attack chain than traditional endpoint defense systems. The conclusions on whether this
hypothesis proved true at the end of this research were reached from two primary sources
of information. First, the attack technique detection issues identified by the author in on-
going resiliency research coupled with professional experience on alert breadth served as
the first piece of supporting information. Second, the overall detection statistics collected
after the completion of the validation experiments in Chapter 4 compared with these out-
side observations served as the second piece of information. It should be noted that exact
comparisons of attack technique detection with the developed artifact and commercial
endpoint defense solutions cannot be made with a high-degree of exactness as commer-
cial solutions restrict publishing of performance metrics of their products. However, one
209
overarching conclusion that can be made is a general lack of breadth that commercial end-
point solutions appear to have with regard to alerting on supporting/atomic techniques
for individual parent techniques within the MITRE ATT&CK framework.
For each of the developed validation experiments, additional ”atomic” attack tech-
niques were documented as part of the overarching technique. This was done because
many of the surrounding or supporting techniques executed for an overarching technique
can aid defenders in defense by providing critical contextual information that they would
research as part of their defensive operations workflow. As noted multiple times through-
out this document, many commercial endpoint defense solutions alert on only a single
atomic technique and do not alert on all of the atomic techniques for an overarching tech-
nique. The combination of all the facts and the overall detection ratio of 36/48 atomic
techniques detected in the validation experiments support the conclusion that the de-
veloped artifact detects a more complete set of behaviors for a given attack chain than
traditional endpoint defense solutions.
5.4 Contributions
As is standard for Design Science Research, the topic of real-world applicability and
contributions to the field are a primary consideration throughout the process of artifact
development and validation. The overarching contribution that this research provides to
the field is the artifact itself. As shown in Chapters 1 & 2, there are currently no open-
source/free ETW monitoring tools that provide the level of configurability and attack
technique detection performance that the artifact in this research provides. Once the
artifact is available on the GTRI Github page, security researchers can immediately use
it to detect the techniques that were used during the validation experiments on their
own systems. Additionally, the configurability of the artifact allows users to use the
artifact in ways that were not focused on during the primary research. For example,
210
the artifact allows users to subscribe to any ETW provider and search for any amount
of strings in any amount of fields within the data emanating from that provider. Using
creative defense logic, security researchers can use the developed artifact to experiment
with logging di↵erent types of behaviors at the user level, individual process level, or the
entire system level.
A useful exercise could be to configure the artifact to alert on any process that com-
municates with IP address ranges that are not in the RFC 1918 non-routable address
ranges (192.168.0.0/16, 172.16.0.0/12, 10.0.0.0/8). Configuring the artifact to alert on
this activity would immediately reveal processes that communicate with Internet IP ad-
dresses that were not previously known. The User-Defined Context file allows users the
ability to specify negations in the field names they are specifying to search within. For
example, users can specify to alert on fields that do not contain RFC 1918 IP addresses
by supplying a ”!” character before the value supplied (example using a construct: [UDC-
line-preamble].field.DestIP = !global:RFC1918Addresses). This definition will tell the
parser to negate all addresses in the ranges defined in the construct when generating the
IF statement blocks for behavior checking during the dynamic class generation process.
This searching logic can be applied to any field that exists within an ETW provider
and extends to many more use-cases. Using combinations of fields to search in with
negation/non-negation matching allows security researchers to quickly identify what be-
haviors from processes are normal across the system under inspection to then be able
to make conclusions on what activity should be considered abnormal. Researchers could
then apply this logic to many other ETW event providers and very quickly establish a
broad set of normal behaviors across many additional configurations of Windows systems
in operational environments.
Some secondary contributions from the artifact itself are the Automated Attack Tech-
nique Execution and the Automated ETW Event Generation scripts that enabled the
ability to iterate through roughly 1,100 ETW event providers and execute a standard-
211
ized body of attack stimuli for each provider. While there are many open-source projects
dedicated to executing attack techniques based on the MITRE ATT&CK framework, the
two scripts developed for this research were developed to work together for the common
goal of identifying evidence of attack techniques within ETW event data. There are no
known existing open or academic projects that focus on the goal of identifying evidence
of attack techniques across many di↵erent MITRE ATT&CK framework categories and
across the entire body of ETW event providers available in Windows. The Automated
Attack Technique Execution and Automated ETW Event Generation scripts are closed-
source at the time of this writing pending an export-control review. However, once these
scripts are released on the GTRI Github page, security researchers will immediately be
able to use these scripts to identify evidence of more attack techniques within unexplored
ETW event data sources.
Some tertiary contributions to the field are the validation experiment design, the
reverse engineering of the atomic techniques for each validation experiment, and the
results of the experimentation with the developed artifact. Each of these outputs from
the research provide their own contributions to the field. There has not been any known
research published in open or academic sources on identifying atomic techniques for parent
techniques across many categories of the MITRE ATT&CK framework and mapping those
atomic techniques to detection coverage within ETW event data. The MITRE ATT&CK
framework is a very useful reference for identifying high-level techniques across typical
attacker campaign progression, however, as shown in this research, there are a significant
amount of supporting techniques/behaviors that could occur before and after the parent
technique that are useful for defensive reasoning. The atomic techniques identified for
each of the validation experiments and the results of detection will immediately increase
the knowledge of the industry that there are additional behaviors that can be used to aid
in detection of parent attack techniques.
212
5.5 Limitations & Future Work
5.5.1 Limitations
While the developed software artifact performs very well detecting impactful attack tech-
niques across the MITRE ATT&CK framework, there are some inherent limitations of
the artifact that are worth noting. First, the developed artifact is a user-mode application
by design and runs in user-mode with administrative privileges. Applications that run
in user-mode are inherently at risk of tampering and disablement by attackers that are
at the same or higher privilege level. Attackers running at the same or higher privilege
level will be able to stop ETW sessions, which will cause events to stop being processed
by the artifact, or stop the application entirely. Other than the prototype anti-evasion
capability illustrated in Section 3.3.4.5, no other anti-evasion or resiliency features are
included with the current artifact software version. The current artifact version is de-
signed to be a security research tool and should not be used on production systems for
the purposes of detecting real attack techniques without trained personnel monitoring the
state of the artifact while in operation. As noted in the following section (Section 5.5.2)
future versions of the artifact will address anti-evasion and resiliency features.
Due to the fact that the developed artifact runs in user-mode, there are special cir-
cumstances that should be noted with respect to attackers who have achieved kernel-level
code execution. Any attacker that has already established kernel-level code execution
on a system could be able to execute attack techniques and ETW evasion techniques in
kernel-mode without users being alerted to this activity occurring. System configuration
policies and driver load detection can alert users to this activity materializing, however,
kernel-level driver exploitation that results in immediate kernel code execution/access can
subvert these e↵orts. This is not a fault of the developed artifact, but rather, a funda-
mental fact of the threat landscape and privilege separations in Windows systems. In
213
future versions, the artifact will include a health pipeline mechanism that continuously
tests the health of the logging pipeline within the system in order to alert on any devia-
tions of expectations that could a↵ect the artifact’s ability to alert the end user. Having
such a health checking feature in place would allow for the conclusion that intentional
evasions could be occurring at the kernel-level that are inherently invisible to user-mode
applications.
The second limitation that the developed artifact has as of this writing relates to
application performance under specific conditions. As noted in Chapter 3, one of the
limitations identified during development of the artifact behavior detection code was that
large lists of construct items would result in exponentially increasing computational com-
plexity to check each user defined behavior against each ETW event. The author and
Bhaskar Rimal (DSU) are designing and executing research concurrent to the research
presented in this document to identify ways to reduce behavior checking computational
complexity to O(1) for large construct lists such as known binaries on a system and IP
address ranges. Initial tests using built-in .NET data structures, such as hash sets and
dictionaries, have proven to be a promising performance improvement. The intent of this
research is to identify the most efficient method to process large lists of items within the
developed artifact and incorporate the changes into the EDAC code base.
5.5.2 Future Work
The developed artifact has a primary purpose to detect attack techniques executed by
attackers. As shown in Chapters 1 & 2, many of the problems current-generation end-
point solutions have relate to evasion/tampering techniques executed by attackers and
the resultant failure to alert users on resiliency problems with the software. The next
software version planned for the developed artifact will include features to alert users on
the resiliency of the ETW pipeline as well as any other component that the artifact relies
on to generate alerts to end users. On top of this, future versions of the artifact will
214
also include capabilities to run the artifact as a Windows service in coordination with
a Windows driver to allow Windows process protection features to be enabled. Process
protection features in Windows are meant for security-related processes and will prevent
any user-mode process from stopping the process that is protected. This will require at-
tackers to achieve kernel-level code execution in order to tamper with the artifact or its
process run state.
Other features that are currently on the road-map for future versions of the artifact
include the ability to execute the artifact without the need for a user interface and stream
logs to an external log aggregator, such as Splunk and Elasticsearch. One of the visions
for the use of the developed artifact is for the artifact to run on every endpoint in an enter-
prise. Having the developed artifact run on every endpoint in an enterprise introduces the
capability to monitor process-level behaviors at a very large scale if all endpoints reported
their process behaviors to a centralized location. The standard for security monitoring of
computer systems in many industries is to establish a Security Operations Center (SOC)
where defenders operate o↵ of centrally collected data on the health/security of all sys-
tems on the network. These operation centers typically ingest and visualize security data
from computer systems using a log aggregator. Since the artifact generates log messages
that are displayed to the console in the current version, the next logical step is to generate
log messages in a standardized logging format, such as JSON, for consumption into log
aggregation platforms. This would allow organizations to ingest process-level telemetry
into a SOC and visualize all of the abnormal behaviors that are occurring for every process
on every system in the enterprise.
The overall intent for this research is for it to continue beyond this document as
there are many other additional use-cases and attack techniques to research using the
EDAC tool. The EDAC tool is planned to become open-source under the Georgia Tech
Research Institute (GTRI) Github account [28] after this writing. Releasing the code of
the developed software artifact will ensure that the open-source community can contribute
215
to the overarching goals of the research to develop tools that augment commercial endpoint
defense solutions for greater defensive capability across the world.
216
References
[1] L. Columbus, 5 key insights from absolute’s 2019 endpoint security trends report,
Electronic Article, 2019. [Online]. Available: https://www.forbes.com/sites/
louiscolumbus/2019/09/08/5-key-insights-from-absolutes-2019-endpoint-
security-trends-report/?sh=2e64da553b7c.
[2] T. R. Group, Forecast revenue from endpoint security market worldwide from 2020
to 2024 (in million u.s. dollars), Chart or Table, 2021. [Online]. Available: https:
//www.statista.com/statistics/497965/endpoint-security-market/.
[3] MITRE, Att&ck matrix for enterprise, Web Page, 2022. [Online]. Available: https:
//attack.mitre.org/.
[4] Crowdstrike, Mitre att&ck framework, Web Page, 2022. [Online]. Available: https:
//www.crowdstrike.com/cybersecurity-101/mitre-attack-framework/.
[5] Kaspersky, What is heuristic analysis? Web Page. [Online]. Available: https://
usa.kaspersky.com/resource-center/definitions/heuristic-analysis.
[6] N. Moran, M. Scott, M. Oppenheim, and J. Homan, Operation double tap, Electronic
Article, 2014. [Online]. Available: https://www.mandiant.com/resources/blog/
operation-doubletap.
[7] A. Arfeen, A. Saad, K. M. Asim, and J. S. F. Ali, Endpoint detection & response: A
malware identification solution, Conference Paper, 2021. doi: 10.1109/ICCWS53234.
2021.9703010.
[8] Z. Stein, Blinding edr on windows, Blog, 2020. [Online]. Available: https : / /
synzack.github.io/Blinding-EDR-On-Windows/.
[9] 0x00sec, Defeating userland hooks (ft. bitdefender), Blog, 2019. [Online]. Available:
https://0x00sec.org/t/defeating-userland-hooks-ft-bitdefender/12496.
[10] Microsoft, Detours, Web Page, 2002. [Online]. Available: https://www.microsoft.

com/en-us/research/project/detours/.
[11] B. Delpy, Mimikatz, Web Page, 2022. [Online]. Available: https://github.com/

gentilkiwi/mimikatz.
[12] PowerShellMafia, Invoke-mimikatz.ps1, Web Page, 2016. [Online]. Available: https:

//github.com/PowerShellMafia/PowerSploit/blob/master/Exfiltration/
Invoke-Mimikatz.ps1.
217
[13] M. R. Garnier and Thomas, Sysmon v14.14, Web Page, 2023. [Online]. Available:
https://learn.microsoft.com/en-us/sysinternals/downloads/sysmon.
[14] T. M. Lewis, “Process behavior analysis and classification for threat detection,”
Unpublished Work, Georgia Tech Research Institute (GTRI), 2022.
[15] W. Matsuda, F. Mariko, and M. Takuho, Real-time detection system against ma-
licious tools by monitoring dll on client computers, Conference Paper, 2019. doi:
10.1109/AINS47559.2019.8968697.
[16] E. Co, Elastic stack, Web Page, 2023. [Online]. Available: https://www.elastic.
co/elastic-stack/.
[17] T. M. Lewis, “At-scale process monitoring for cyber threat identification,” Unpub-
lished Work, Georgia Tech Research Institute (GTRI), 2021.
[18] Splunk, Splunk enterprise, Web Page, 2022. [Online]. Available: https : / / www .
splunk.com/en_us/products/splunk-enterprise.html.
[19] Splunk, Splunk universal forwarder 9.0.3, Web Page, 2023. [Online]. Available:
https://www.splunk.com/en_us/download/universal-forwarder.html.
[20] SwiftOnSecurity, Sysmon-config — a sysmon configuration file for everybody to fork,

Web Page, 2021. [Online]. Available: https : / / github . com / SwiftOnSecurity /
sysmon-config.
[21] W. U. Hassan, B. Adam, and M. Daniel, Tactical provenance analysis for endpoint
detection and response systems, Conference Paper, 2020. doi: 10.1109/SP40000.
2020.00096.
[22] R. J. Wieringa, Design Science Methodology for Information Systems and Software
Engineering. Berlin Heidelberg: Springer-Verlag, 2014, isbn: 978-3-662-43839-8.
[23] Microsoft, Event tracing for windows simplified, Web Page, 2021. [Online]. Avail-
able: https://learn.microsoft.com/en-us/troubleshoot/windows-server/
system-management-components/event-tracing-for-windows-simplified.
[24] Microsoft, About event tracing, Web Page, 2021. [Online]. Available: https : / /
learn.microsoft.com/en-us/windows/win32/etw/about-event-tracing.
[25] R. Boonen, Silketw: Because free telemetry is . . . free! Blog, 2019. [Online]. Avail-
able: https://www.mandiant.com/resources/silketw-because-free-telemetry-
is-free.
218
[26] FuzzySecurity, Silketw, Computer Program, 2021. [Online]. Available: https : / /
github.com/mandiant/SilkETW.
[27] A. Project, Ansible 7.0 glossary, Web Page, 2022. [Online]. Available: https://
docs.ansible.com/ansible/latest/reference_appendices/glossary.html.
[28] G. T. R. Institute, Repositories, Web Page, 2023. [Online]. Available: https : / /

github.com/orgs/gtri/repositories.
[29] MITRE, Caldera, Web Page, 2022. [Online]. Available: https : / / github . com /
mitre/caldera.
[30] Endgame, Red team automation (rta), Web Page, 2018. [Online]. Available: https:
//github.com/endgameinc/RTA.
[31] R. C. Co, Atomic-red-team, Web Page, 2022. [Online]. Available: https://github.

com/redcanaryco/atomic-red-team.
[32] Uber, Metta, Web Page, 2018. [Online]. Available: https://github.com/uber-

common/metta.
[33] Fugawi, Mitre att&ck technique emulation (mate), Web Page, 2019. [Online]. Avail-
able: https://github.com/fugawi/mate.
[34] Praetorian, Purple-team-attack-automation, Web Page, 2020. [Online]. Available:

https://github.com/praetorian-inc/purple-team-attack-automation.
[35] S. Bradshaw, Vulnserver, Web Page, 2020. [Online]. Available: https://github.

com/stephenbradshaw/vulnserver.
[36] sgtatham, Chapter 7: Using the command-line connection tool plink, Web Page.
[Online]. Available: https://the.earth.li/ ~sgtatham/putty/0.58/htmldoc/
Chapter7.html.
[37] Rapid7, Metasploit framework, Web Page, 2022. [Online]. Available: https://www.
metasploit.com/.
[38] Rapid7, Resource scripts, Web Page. [Online]. Available: https://docs.rapid7.

com/metasploit/resource-scripts/.
[39] E. Lin, Trends in the recent emotet maldoc outbreak, Blog, 2022. [Online]. Available:
https://www.fortinet.com/blog/threat-research/Trends-in-the-recent-
emotet-maldoc-outbreak.
219
[40] M. Dunwoody, Dissecting one of apt29’s fileless wmi and powershell backdoors (posh-
spy), Blog, 2017. [Online]. Available: https://www.mandiant.com/resources/
blog/dissecting-one-ofap.
[41] O. Security, Msfvenom, Web Page. [Online]. Available: https://www.offensive-

security.com/metasploit-unleashed/msfvenom/.
[42] C. (CISA) and I. S. Agency, Fastcash 2.0: North korea’s beagleboyz robbing banks,
Press Release, 2020. [Online]. Available: https://www.cisa.gov/uscert/ncas/
alerts/aa20-239a.
[43] Microsoft, Windows registry information for advanced users, Web Page, 2022. [On-
line]. Available: https://learn.microsoft.com/en-us/troubleshoot/windows-
server/performance/windows-registry-advanced-users.
[44] Microsoft, Run and runonce registry keys, Web Page, 2022. [Online]. Available:
https://learn.microsoft.com/en- us/windows/win32/setupapi/run- and-
runonce-registry-keys.
[45] Mandiant, “Apt1 exposing one of china’s cyber espionage units,” Report, 2021.
[Online]. Available: https://www.mandiant.com/sites/default/files/2021-
09/mandiant-apt1-report.pdf.
[46] O. Damari, Etw provider docs, Web Page, 2020. [Online]. Available: https : / /
github . com / repnz / etw - providers - docs / blob / master / Manifests - Win10 -
17134/Microsoft-Windows-Kernel-Audit-API-Calls.xml.
[47] Microsoft, Process security and access rights, Web Page, 2022. [Online]. Available:
https://learn.microsoft.com/en-us/windows/win32/procthread/process-
security-and-access-rights.
[48] P. J. Robinson, M. Simos, R. Grimes, A. Margosis, and Laura, Mitigating pass-the-

hash (pth) attacks and other credential theft techniques, Electronic Article, 2012.
[Online]. Available: http : / / download . microsoft . com / download / 7 / 7 / A /
77ABC5BD - 8320 - 41AF - 863C - 6ECFB10CB4B9 / Mitigating \ %20Pass - the - Hash \
%20(PtH)\%20Attacks\%20and\%20Other\%20Credential\%20Theft\%20Techniques_
English.pdf.
[49] G. d. tijl, Retrieving ntlm hashes and what changed in windows 10, Blog, 2018. [On-
line]. Available: https://www.insecurity.be/blog/2018/01/21/retrieving-
ntlm - hashes - and - what - changed - technical - writeup / #Step _ By _ Step _
Examplenew_NTLM_Hash_Retrieval_AES_Cipher.
[50] M. Russinovich, Psexec v2.40, Web Page, 2022. [Online]. Available: https://learn.
microsoft.com/en-us/sysinternals/downloads/psexec.
220
[51] C. (CISA) and I. S. Agency, Alert (ta17-181a), Press Release, 2018. [Online]. Avail-
able: https://www.cisa.gov/uscert/ncas/alerts/TA17-181A.
[52] m. EdKaim, juliakm, and v-thepet and, What is infrastructure as code (iac), Web
Page, 2022. [Online]. Available: https://learn.microsoft.com/en-us/devops/
deliver/what-is-infrastructure-as-code.
[53] A. Project, How to build your inventory, Web Page, 2022. [Online]. Available:
https : / / docs . ansible . com / ansible / latest / inventory _ guide / intro _
inventory.html#adding-variables-to-inventory.
[54] YAML, Yaml 1.2, Web Page, 2021. [Online]. Available: https://yaml.org/.
[55] R. Ivanescu, What is an ini file? here’s how to create, open and edit an ini file, Web
Page, 2022. [Online]. Available: https : / / www . advancedinstaller . com / what -
is- ini- file.html#: ~ :text=The\%20INI\%20file\%20format\%20was, rely\
%20extensively\%20on\%20INI\%20files..
[56] Microsoft, List¡t¿ class, Web Page. [Online]. Available: https://learn.microsoft.

com/en-us/dotnet/api/system.collections.generic.list-1?view=net-7.0.
[57] Microsoft, Dictionary¡tkey,tvalue¿ class, Web Page. [Online]. Available: https://

learn . microsoft . com / en - us / dotnet / api / system . collections . generic .
dictionary-2?view=net-7.0.
[58] Microsoft, Reflection in .net, Web Page, 2021. [Online]. Available: https://learn.
microsoft.com/en-us/dotnet/framework/reflection-and-codedom/reflection.
[59] Veil-Framework, Veil, Web Page, 2022. [Online]. Available: https://github.com/

Veil-Framework/Veil.
[60] HarmJ0y, Arya, Web Page, 2017. [Online]. Available: https : / / github . com /
HarmJ0y/Arya.
[61] R. Strahl, Compilingcsharpcode.cs, Web Page, 2019. [Online]. Available: https :

//gist.github.com/RickStrahl/f65727881668488b0a562df4c21ab560.
[62] M. a. Contributors, 31k ernelandclrmonitor.cs, Web Page, 2018. [Online]. Available:

https://github.com/microsoft/perfview/blob/6b4771eecba5ed00555a34d70e19a8742af4ee
src/TraceEvent/Samples/31_KernelAndClrMonitor.cs.
[63] M. a. Contributors, 10s impleeventsourcemonitor.cs, Web Page, 2020. [Online]. Avail-

able: https://github.com/microsoft/perfview/blob/6b4771eecba5ed00555a34d70e19a8742a
src/TraceEvent/Samples/10_SimpleEventSourceMonitor.cs.
221
[64] FuzzySecurity, Etwcollector.cs, Web Page, 2019. [Online]. Available: https : / /
github.com/mandiant/SilkETW/blob/master/SilkETW/ETWCollector.cs.
[65] NewtonSoft, Json.net, Web Page, 2023. [Online]. Available: https://www.newtonsoft.

com/json.
[66] MDSec, Hiding your .net - etw, Blog, 2020. [Online]. Available: https : / / www .
mdsec.co.uk/2020/03/hiding-your-net-etw/.
[67] StatCounter, Desktop windows version market share worldwide, Web Page, 2022.
[Online]. Available: https://gs.statcounter.com/os-version-market-share/
windows/desktop/worldwide.
[68] Microsoft, Active directory domain services overview, Web Page, 2022. [Online].
Available: https://learn.microsoft.com/en-us/windows-server/identity/
ad - ds / get - started / virtual - dc / active - directory - domain - services -
overview.
[69] RedHat, Edge computing for the department of defense, Web Page, 2021. [Online].
Available: https://www.redhat.com/en/resources/edge-for-dod-brief.
222

Endpoint Defense As Code (Edac) Configurable

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Endpoint Defense As Code (Edac) Configurable

Uploaded by

Copyright:

Available Formats

Dakota State University

Masters Theses & Doctoral Dissertations

ENDPOINT DEFENSE AS CODE (EDAC): CONFIGURABLE

Follow this and additional works at: https://scholar.dsu.edu/theses

A dissertation submitted to Dakota State University in partial fulfillment of the

Dissertation Title: _______________________________________

Dissertation Chair/Co-Chair: Date: _____

Dissertation Chair/Co-Chair: Date: _____

Committee member: ______ Date: _____

Committee member: ______ Date: _____

Committee member: ______ Date: _____

Committee member: ______ Date: _____

Original to Office of Graduate Studies and Research

Dissertation Approval Form ii

Table of Contents vii

List of Figures xiii

1.1 Background of the Problem . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2.1 Attack Technique Detection Research . . . . . . . . . . . . . . . . . . . . . 18

3.1 Overall Research Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

4.1 Validation Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

5.1 Research Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205

Table 3.1 Attack Techniques Chosen for Validation in Automation Script 45

Table 4.1 Windows 10 Standard Endpoint - System Parameters . . . . 165

Figure 1.1 Modern EDR Hooking/Detection Architecture . . . . . . . . . . . . 7

Figure 2.1 SwiftOnSecurity Sysmon Configuration File [20] . . . . . . . . . . . 21

Figure 3.1 Windows 10 21H1 ETW Providers . . . . . . . . . . . . . . . . . . . 36

Figure 4.1 EDAC Validation Architecture . . . . . . . . . . . . . . . . . . . . . 162

1.1 Background of the Problem

1.1.1 Endpoint Defense Software Architecture

Figure 1.1: Modern EDR Hooking/Detection Architecture

1.1.2 User-Mode Hook Dependency Problem

1.1.3 Endpoint Solution Resiliency Problem

1.1.4 Technique/Signature Detection Problem

1.1.5 Alert Semantics/Endpoint Configurability Problem

1.2 Special Considerations for Air-Gapped, Discon-

nected, & DDIL Systems

Much of the endpoint defense solution industry is moving to a cloud-based connectivity

1. Create a software artifact that analyzes process/system events in real-time against

Literature Review/Related Work

2.1.1 Technique Detection Using Sysmon

2.2 Process Behavior Monitoring Research

Figure 2.1: SwiftOnSecurity Sysmon Configuration File [20]

2.3 Process Behavior Classification Research

2.4 Endpoint Defense Solution Research

Figure 2.4: Debugger Illustrating an EDR User-Mode hook

2.4.2 Endpoint Solution Resiliency Research

2.4.3 Endpoint Solution Research

Research Methodology & Proposed

3.1 Overall Research Design

3.1.2 Hypothesis and Research Questions

The primary research questions focus on answering whether it is possible to A) con-

3.1.3 Background/Supporting Research Phase

3.1.4 Artifact Development Phase

3.1.5 Artifact Validation Phase

3.2 Background & Supporting Research Phase

3.2.1 Event Tracing for Windows (ETW) Introduction

3.2.2 ETW Event Data Completeness/Attack Technique Cov-

3.2.3 Automated ETW Event Generation

1. Delete old JSON log files in the collected logs directory.

3. Capture a list of all available ETW providers on the executing system.

5. Execute a separate attack chain script with all chosen techniques.

6. Stop all SilkETW processes.

3.2.4 Automated Attack Technique Generation/Execution

Committee member: __ Date: _

Committee member: __ Date: _

Committee member: __ Date: _

Committee member: __ Date: _