Professional Documents
Culture Documents
net/publication/336846762
CITATION READS
1 943
2 authors:
Some of the authors of this publication are also working on these related projects:
All content following this page was uploaded by Sergey Gordeychik on 28 October 2019.
Abstract
This paper provides an overview of the SD-WAN New Hop Project - a community-driven
research project devoted to security analysis of Software-Defined Wide Area Network
(SD-WAN) products and solutions. The research team held several Internet-wide scans
to survey the distribution of SD-WAN products and to detect known software
vulnerabilities in its components. Manual and tool-aided vulnerability assessment of
multiple SD-WAN products were performed. This paper describes an overview SD-WAN
Threat Landscape, methodology and implementation of the Internet-wide scans,
provides security checklists and vulnerabilities found during the project.
Keywords
SDN, SD-WAN, NFV, security, cybersecurity, threat intelligence, security assessment,
threat modelling
1 Introduction
Nowadays SDx technologies are very popular. SD-WAN is used for Cloud platforms and
Enterprise networks for branch connectivity. Vendors are developing different
subcomponents such as SD-LAN, SD-CORE, SD-VPN, SD-Access and SD-DC
products. At the time of writing, Metro Ethernet Forum (MEF) has unveiled its “SD-WAN
Service Attributes and Services Standard”1.
1
MEF 70 - SD-WAN Service Attributes and Services,
https://www.mef.net/resources/technical-specifications/download?id=122&fileid=file1
The SD-WAN New Hop Project targets the security of SD-WAN. It was started in
December 2017, as a side project of SD-WAN solution security design and Security
SDL planing. During the course of the project multiple critical vulnerabilities, such as
RCE, XXE, SQLi, unpatched software, outdated packages, vulnerable multi-tenant
access control mechanisms were found. It was decided to investigate the security of
well-known and famous SD-WAN products. The assessment covered the following
products.
Table 1. Explored SD-WAN vendors and products
Vendor Products
Riverbed SteelConnect
SteelHead
3 SD-WAN Threats
This section briefly considers the main and the most widespread security threats related
to the SD-WAN functional blocks and components according to the developed SD-WAN
threat landscape4.
3.1. Outdated Software. Most of SD-WAN systems use GNU/Linux-based operating
systems such as Ubuntu, Debian, CentOS, etc. Very often other open source tools are
used in different network functions. For example, IPsec overlay can be based on
“strongSwan”, while control plane runs API on top of Apache Tomcat, Karaf, WildFly,
Node.js.
The following typical issues were found in many SD-WAN systems:
● Using outdated software
● Insecure software update mechanisms
● Using insecure cryptographic primitives (e.g., MD5, SHA1)
3.2. Insecure Web Interfaces. All analysed SD-WAN products uses Web components
to implement user-machine or machine-to-machine interfaces. At the same time, it was
found that the existence of “low-hanging fruit” vulnerabilities in those Web interfaces is
widespread and systematic.
We identified multiple vulnerabilities to the following well-known attacks:
● HTTP Slow DoS-attacks
2
Verizon. SDN-NFV Reference Architecture. URL:
http://innovation.verizon.com/content/dam/vic/PDF/Verizon_SDN-NFV_Reference_Architecture.pdf
3
ETSI GS NFV-SWA 001 V1.1.1. Network Functions Virtualisation (NFV);Virtual Network Functions
Architecture. URL:
https://www.etsi.org/deliver/etsi_gs/NFV-SWA/001_099/001/01.01.01_60/gs_NFV-SWA001v010101p.pdf
4
S. Gordeychik. D. Kolegov. SD-WAN Threat Landscape. URL: https://arxiv.org/abs/1811.04583
● Password brute-forcing
● XSS (Reflected, stored)
● Command injection and RCE
● XXE and SSRF
● CSRF
● IDOR
For instance, the security bulletin5 describes multiple vulnerabilities found on the
Web UI of the Citrix SD-WAN appliances. The vulnerabilities could allow an
unauthenticated attacker having network access to the management interface to
compromise a corresponding SD-WAN network.
3.3. Multi-tenancy. An SD-WAN system is called multi-tenant if it has an ability to
provide logical isolation of shared resources within the NFV concept. In general, this
suggests the existence of a common Web UI used by different tenants to access
isolated SD-WAN instances. It turned out that those Web UI also have security
vulnerabilities specific to multi-tenant applications such as missing or improper access
control.
In this case, an adversary having legitimate access to the Web interface can
implement different attacks against other tenants (clients or providers). For example,
standard or weak passwords for different tenants or weak password recovery
mechanism can lead to horizontal privilege escalation. In this regard, the adversary can
analyse the implementation and its possible weaknesses using an own isolated session
and then exploit vulnerabilities against other clients.
During the research, the following main threats regarding multi-tenancy were identified:
● Unauthorized access to provider data
● Unauthorized access to tenant data
● Unauthorized access to tenant VNF
● Unauthorized access to the tenant stored flow data (e.g., NetFlow, IPFIX)
● Denial of Service
3.4. Cryptography. Cryptographic mechanisms are employed on all SD-WAN planes:
data, control, management and orchestration (application). Cryptographic mechanisms
are the essence of SDN/SD-WAN.
At the present vendors have to invent cryptography protocols to implement SD-WAN
specific flows because there are no production-ready safe cryptography protocols
developed within SD-WAN conception and taken into account all peculiarities. Most
vendors approach to this problem by adopting IPsec for SD-WAN6.
5
Citrix SD-WAN Multiple Security Updates. URL https://support.citrix.com/article/CTX236992
6
R. Marin-Lopez, et. al. Internet draft Software-Defined Networking (SDN)-based IPsec Flow Protection.
URL: https://datatracker.ietf.org/doc/draft-ietf-i2nsf-sdn-ipsec-flow-protection/
We also found that most SD-WAN products have the following flaws related to
cryptographic provisioning:
● Use of hardcoded public-key cryptography key pairs and corresponding
certificates that are the same for all customers and can not be replaced
● Use of self-signed certificates for generated public-key cryptography key pairs
issued by the SD-WAN product
● Manual installation of self-signed certificates on SD-WAN nodes without
certification revocation features
3.5. Zero-touch Provisioning. Zero Touch Provisioning (ZTP) is a mechanism that
allows nodes to be provisioned and configured automatically. All the main SD-WAN
products support ZTP. At the same time, by design, ZTP server must accept requests
from unidentified and unauthenticated devices on the Internet. This fact increases the
attack surface and attacker capabilities. The following threats can be encountered
regarding ZTP:
● ZTP server or/and client spoofing
● Unauthorized access to ZTP service and data in cloud
● Exhaustive Denial of Service on ZTP service
● Privilege escalation on ZTP server
● Eavesdropping
● Insufficient access control on multi-tenant ZTD service
7
S. Gordeychik, D.Kolegov, A, Nikolaev. SD-WAN Internet Census. URL:
https://arxiv.org/abs/1808.09027
4.2. The Grinder Framework
The Grinder framework8 is a security research intelligence framework that have been
developing since 2018. It was created to automatically enumerate, fingerprint and scan
hosts on the Internet using different specialised back-end systems: search engines
(e.g., Shodan or Censys) for discovering, scanners (e.g., NMAP) for active fingerprinting
and scanning, vulnerability databases (e.g., Vulners) for getting information related to
vulnerabilities in the discovered software. The Grinder framework can be used in many
different areas of security research, as a connected Python module to your project or as
an independent ready to use from the box tool.
8
The Grinder Framework. URL:https://github.com/sdnewhop/grinder
● Collecting Threat Intelligence Information, such as security bulletins, public
exploits etc. for detected vulnerabilities and software.
● Confidence filtering.
● Building an interactive map with information about the hosts found.
● Creating plots and tables based on the collected results.
Practical evaluation of this approach during massive Internet census projects has
shown the benefits of the combination of active and passive host identification methods.
Having said this, the main purpose of Grinder is to unify different sophisticated security
tools, aggregate gathered information and to help understanding collected information
and exposed statistics related to the systems in the research scope. Grinder
incrementally saves all scans, results and statistics to its database to compare results
over time and track the statistics changes.
To visualize gathered data, Grinder provides an interactive world map containing all
results. Grinder map backend is written in Flask and supports additional REST API
methods to get more information about all scanned hosts or some particular host from
the map. Also, it is possible to show some additional information about host interactively
from the map. For example, currently open host will be automatically checked for
availability with “ping” from the backend, also for every host many additional features
are available: current host can be directly opened in Shodan, Censys and ZoomEye
search engine web interfaces, host can be shown on Google Maps with all available
information about geolocation, also it is possible to make an IP lookup, or open raw
information in JSON directly in browser or from your application with provided API
methods.
9
R Merget. Scalable Scanning and Automatic Classification of TLS Padding Oracle Vulnerabilities. URL:
https://github.com/RUB-NDS/TLS-Padding-Oracles
the paper considers and encounters only vulnerabilities to CBC padding attack. We
thought that it is also interesting to scan SD-WAN nodes related to all main TLS
vulnerabilities.
Within our research, we considered only SD-WAN nodes (controllers, Web UI, edge
routers, etc.) we had already collected and stored via enumeration and fingerprinting10.
We used the TLS-Attacker11 framework as a TLS core scanning engine and the Grinder
framework as an orchestrator. In this case, the TLS-Attacker was used as a back-end
within the Grinder framework. To get information about TLS configuration, bugs, and
possible attacks, Grinder provides a wrapper module for TLS-Scanner and
TLS-Attacker tools, that is used to handle all scanning options and analyze the results.
TLS-Attacker is a Java-based framework for analyzing TLS libraries. It is able to send
arbitrary protocol messages in an arbitrary order to the TLS peer, and define their
modifications using a provided interface. Grinder uses related TLS-Scanner module to
scan TLS configuration with TLS-Attacker.
Gathered data by TLS-Scanner and TLS-Attacker helps Grinder to count unique
attacks and bugs, also it is possible to count different unique types of entities for every
vendor, product, protocol, and many other types and categories. To get more accurate
results, Grinder firstly checks availability of every host that was found in Shodan or
Censys, after that Grinder tries to detect proper port and other options to successfully
start scan with TLS-Scanner, and finally Grinder saves every result from every host in
separate files that later can be parsed more accurately with additional parser and
analyzer.
The employed methodology can be defined as follows:
1. Run TLS scanning engines (e.g., TLS-Attacker, SSL Labs Server Scan, etc.) on
the appropriate hosts and interfaces from the SD-WAN Knowledge Database.
2. If vulnerabilities are found, rescan the node two times to minimize false positives.
3. If the vulnerabilities are still present, check them again using PoC scripts in
Python.
4. Save the confirmed results to the database.
We scanned about 7500 SD-WAN nodes and found the following:
● 1873 nodes were vulnerable to Sweet 32 attack
● 121 nodes were vulnerable to CBC Padding Oracle attack
● 30 nodes were vulnerable to CRIME attack
● 29 nodes were vulnerable to Logjam attack
● 28 nodes were vulnerable to CVE-2016-2107
● 14 nodes were vulnerable to DROWN attack
10
SD-WAN Internet Census Knowledge Base. URL:
https://github.com/sdnewhop/sdwannewhope/blob/master/docs/census.md
11
TLS-Attacker Tool. URL: https://github.com/RUB-NDS/TLS-Attacker
● 6 nodes were vulnerable to ROBOT attack
● 1 node was vulnerable to Heartbleed
Vulnerabilities to the ROBOT attack were found in Viprinet product only. We found
one Riverbed SteelHead node vulnerable to Heartbleed attack. The software was
released in 2013.
Riverbed SteelHead nodes released prior 2014 and Viprinet VPN Virtual Hub nodes
were vulnerable to CBC Padding Oracle attack.
5 Vulnerabilities
The following security vulnerabilities and issues were identified13:
1. Versa Networks
○ Arbitrary command execution on FlexVNF appliances connected to Versa
Director by unauthenticated remote user
○ Vulnerability to XPath and XXE attack allows an attacker to read arbitrary
files or enable SSO authentication and create SSO user to obtain root
privileges
○ Vulnerability to OS command injections allows any authenticated user to
execute arbitrary system commands with root privileges
○ Multiple memory management flaws can lead to DoS
○ Weak permissions and hardcoded passwords allow local and remote
attackers to get access to sensitive data.
12
SD-WAN Harvester. URL: https://github.com/sdnewhop/sdwan-harvester
13
SD-WAN New Hop vulnerability reports:
https://github.com/sdnewhop/sdwannewhope/tree/master/reports
○ It is possible to directly access the database via crafted HTTP request in
different scripts
○ Some Suricata IDPS Emerging Threat rules use regular expressions that
are vulnerable to ReDoS-attack
○ An attacker can craft a malicious Apache Solr configuration file to execute
arbitrary OS command
○ Apache Solr is vulnerable to XXE attack
○ Cryptographically unprotected control plane protocols
○ Versa Analytics Driver REST API Hardcoded credentials
○ Weak CSRF protection can be bypassed via Flash
2. Silver Peak Unity EdgeConnect
○ Lack of protection against brute-forcing password
○ Web UI leaks software versions
○ Lack of protection against CSRF for REST API
○ Denial of service of Web UI via Slow HTTP attacks
○ REST API leaks software versions
○ Use of default SNMP community strings
○ Access to operating system interface via administrative CLI backdoor
○ Multiple vulnerabilities to reflected XSS
○ Arbitrary file reading via path traversal
3. Riverbed SteelConnect
○ Stored XSS via user name field
○ Denial of service of a gateway via slow HTTP attacks
○ Password reset link spoofing via HTTP Host header
○ TLS server is vulnerable to CBC Padding Oracle Attack
4. Cisco (Viptela) SD-WAN
○ OpenSSH leaks system version via a warning message
○ Incorrect protection against CSRF for REST API and Web UI
5. Viprinet Virtual VPN Hub
○ Stored XSS in CLI via item names
○ TLS server is vulnerable to ROBOT attack
○ TLS server is vulnerable to CBC Padding Oracle Attack
6. Citrix NetScaler SD-WAN
○ Denial of Service on Web UI via Slow HTTP attacks
○ Multiple stored and reflected XSS
○ Lack of protection against CSRF for REST API and Web UI
○ Absence of function level access control mechanism
○ Multiple command injections
○ Multiple SQL injections
○ Arbitrary file reading via path traversal
○ Unauthorized access to Munin web UI
○ MitM using a hard-coded cryptography identity
7. Fortinet SD-WAN
○ No Rate Limiting on Web UI Authentication
○ Denial of Service via Slow HTTP DoS Attacks
○ Disclosure of sensitive configuration information for unauthenticated users
○ Highly insecure TLS configuration for Web UI
○ Cross-Site WebSocket Hijacking
○ Insecure password storage
○ Sandboxed shell escaping on FortiGate
○ Sandboxed shell escaping on FortiManager
8. Brain4Net SD-WAN
○ Docker Hub credentials leakage
○ Cryptographically unprotected control plane protocols
○ Multiple stored and reflected XSS
○ Cross-site Websocket hijacking
All these issues were reported to corresponding vendors. Some issues were
disclosed via “Full Disclosure Mailing List”.
6 Vulnerability Management
According to OpenSAMM14, Operation Enablement is one of the crucial elements of
Secure Software Development Lifecycle (SSDL). In general, a vendor implements that
process by creating a Product CERT team which coordinates the vulnerability
remediation process together with the software development team using best practices
and international security standards like ISO/IEC 29147:2018 – Vulnerability Disclosure
in Information Technology.
We believe interactions with external security researchers is an indirect sign of
efficient internal processes related to SSDL. We identify the following indicators
showing security maturity from a security researcher perspective:
● Security contact: The official web site of the product contains web links or emails
related to a product security team.
● Communications: vendor complies the main business rules and norms within
communications (polite interaction, acknowledgement, email encryption, etc.).
● Bug confirmations and patches: vendor notifies security researchers when the
state of vulnerabilities are changed (approved, fixed, tested, done, etc.).
14
“OWASP's Software Assurance Maturity Model”. URL: https://www.opensamm.org/
● CVE, Credits: Vendor requests and assigns CVE numbers to the reported
vulnerabilities, mentions security researchers found that vulnerabilities.
● Researcher friendly: Vendor made a good general impression on the security
researchers within the interactions on the vulnerability management process.
It is important to note that vendors having bad reputation and negative feedback
from the security community perspective will not receive responsibly disclosed
vulnerabilities. This increases the risk of appearance of publicly available zero-day
exploits and the overall risk of compromising.
The summarized impressions of our interactions with SD-WAN vendors within the
security research are as follows.
Riverbed NO NO NO NO NO
Versa NO NO NO NO NO
Networks
Brain4Net NO NO NO NO NO
Viprinet NO NO NO NO NO
Acknowledgements
The authors would like to thank the following members of the “SD-WAN New Hop” team
for finding security issues: Maxim Gorbunov, Oleg Broslavsky, Anton Nikolaev, Nikita
Oleksov, Nikolay Tkachenko and Christoph Jaggi.