You are on page 1of 185

Using Declarative Information in Web Application Frameworks to Improve Security

Master Thesis

Advisor: Prof. Dr. Marc Rennhard Kevin Denver Winterthur, 14. April 2011

Abstract
This thesis is concerned with improving security in web applications by using declarative information provided in web application frameworks. The security testing tool that was researched and developed in this thesis is especially useful for detecting privilege escalation vulnerabilities and misconfigurations in access control checks in web applications with at least two distinguishable user roles in an automated fashion whereas one can be anonymous. Depending on the overall design of the web application, adding access control checks to each accessible resource can be a very tedious task and it is very likely that some checks might have gone forgotten, rendering the application vulnerable to privilege escalation attacks. By using declarative information provided in source- or configuration files of a developed web application it becomes possible to detect privilege escalation vulnerabilities and misconfigurations in access control checks in an automated fashion allowing a developer to make the web application more secure. Finding access control misconfigurations and thus possible privilege escalation vulnerabilities is being done in three phases: (1) The source code and configuration files of a targeted web application are being parsed by a Java command line Framework Parser specifically tailored to support a particular used web application framework. The Framework Parser’s output is a general Meta Layer XML file containing all the found data such as a (non) complete list of available URLs for seeding the web crawling component used in phase 2, a (non) complete list of available users and user roles, a (non) complete list of existing access constraints, etc. (2) The second phase consists of a web crawling component which takes the Meta Layer XML file as an input. The crawling of the targeted web application is done for each configured user respectively role in the Meta Layer XML file to find all possible transitions and enforced access restrictions. (3) The third and final phase consists of a Java Swing Visualisation Application that analyses the collected data from the crawling component and visualises it as a directed graph whereas nodes correspond to found HTML resources and edges correspond to transitions between HTML resources. The analysis is being done in an automated fashion based on the information available in the Meta Layer XML file. In addition the Visualisation Application allows a user to interact and inspect the collected data and enables him to find eventual security threats visually. The test cases showed that the developed security testing tool is indeed capable of detecting privilege escalation vulnerabilities in an automated fashion in real life applications. Whereas false positives might be generated if the crawling component is being prematurely terminated. The number of false positives can be reduced by allowing the crawling component to examine the deployed web application more thoroughly.

i

Abstract
Diese Thesis beschreibt, wie Web Applikationen mittels deklarativen Information aus Konfigurationsdateien oder Applikations-Quellcode in ihrer Sicherheit verbessert werden können. Die in dieser Thesis entwickelte Security Testing Applikation ist besonders für das Auffinden von Privilege Escalation Schwachstellen und für die Detektierung von Konfigurationsfehlern in Access Control Beschränkungen geeignet, falls die zu testende Web Applikation mindestens zwei unterscheidbare Benutzerrollen besitzt, wobei eine der Rollen auch ein anonymer Benutzer sein kann. Je nach Design der Web Applikation kann es sehr mühsam für einen Entwickler sein, jede erreichbare Ressource mit Access Control Beschränkungen auszustatten und es ist sehr wahrscheinlich, dass gewisse Beschränkungen vergessen gehen und die Web Applikation somit verwundbar für Privilege Escalation Schwachstellen wird. Durch die Verwendung von deklarativen Informationen, die aus Konfigurationsdateien oder aus dem Quellcode gewonnen werden können, wird es möglich Privilege Escalation Schwachstellen oder Konfigurationsfehler in Access Control Beschränkungen automatisiert aufzudecken, was es einem Entwickler erlaubt seine Web Applikation sicherer zu machen. Die Detektierung von Privilege Escalation Schwachstellen oder Konfigurationsfehlern in Access Control Beschränkungen findet in drei Phasen statt: (1) Der Quellcode und die Konfigurationsdateien der zu testenden Web Applikation werden durch eine Java Kommandozeilen-Applikation - Framework Parser - analysiert, welche eigens für das verwendete Web Applikations-Framework zugeschnitten ist. Das Resultat des Framework Parser ist eine generische Meta Layer XML Datei, welche all die deklarativ gefundenen Informationen enthält wie: eine (nicht) komplette Liste aller erreichbaren URLs der Applikation, die später vom Web Crawler als Seeds verwendet werden können, eine (nicht) komplette Liste aller bekannten Benutzer und Benutzerrollen, eine (nicht) komplette Liste bereits konfigurierter Access Control Beschränkungen etc. (2) Die zweite Phase besteht aus einem Web Crawler, der die Meta Layer XML Datei als Input verwendet. Die zu testende Applikation wird dann vom Crawler mehrmals mit jedem konfigurierten Benutzer respektive mit jeder Benutzerrolle aus der Meta Layer XML gespidert. Dabei werden alle Übergänge zwischen den Ressourcen und die vorhandenen Access Control Beschränkungen aufgezeichnet. (3) Die dritte und letzte Phase beinhaltet eine Java Swing Visualisation Application, welche die aufgezeichneten Daten analysiert und als gerichteter Graph darstellt, wobei Knoten den gefundenen URLs entsprechen und Kanten den Übergängen zwischen URLs. Die Analyse der Daten findet anhand der vorhandenen Informationen aus der Meta Layer XML Datei automatisiert statt. Zusätzlich erlaubt die Visualisation Application dem Benutzer mögliche Privilege Escalation Schwachstellen oder Konfigurationsfehler in Access Control Beschränkungen visuell zu finden. Eine Testreihe mit verschiedenen Web Applikationen hat gezeigt, dass der entwickelte Ansatz in dieser Thesis durchaus funktioniert und es möglich ist Privilege Escalation Schwachstellen oder Konfigurationsfehler in Access Control Beschränkungen automatisiert aufzudecken. Jedoch können False Positives generiert werden, wenn der Web Crawler frühzeitig beendet wird. False Positives können reduziert werden, wenn dem Web Crawler mehr Zeit eingeräumt wird um die Web Applikation zu testen.

ii

Contact
Zürcher Hochschule für Angewandte Wissenschaften (ZHAW) c/o Institut für angewandte Informationstechnologie InIT Steinberggasse 13 Postfach 805 CH-8401 Winterthur Tel.: +41 (0) 58 934 75 87 Fax: +41 (0) 58 935 75 87 EMail: info.init@zhaw.ch

http://www.zhaw.ch http://www.init.zhaw.ch
Name Kevin Denver Prof. Dr. Marc Rennhard Function Student Project Leader & Advisor E-Mail kevin.denver@gmail.com marc.rennhard@zhaw.ch

iii

Contents
1 Introduction 1

1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2 Java Web Application Frameworks

1 1
2

2.1 Java Technologies for Web Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.1 Java Servlet API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.2 JavaServer Pages Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.3 JavaServer Pages Standard Tag Library . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.4 JavaServer Faces Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Securing Java Web Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.1 Working with Realms, Users, Groups and Roles . . . . . . . . . . . . . . . . . . . 2.2.2 Specifying Security Roles Using Deployment Descriptor Elements . . . . . 2.2.3 Specifying Security Roles Using Annotations . . . . . . . . . . . . . . . . . . . . . 2.2.4 Specifying Security Constraints Using Deployment Descriptor Elements 2.2.5 Specifying Security Constraints Using Annotations . . . . . . . . . . . . . . . . 2.3 Securing Java EE Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.1 Specifying Method Permissions Using Annotations . . . . . . . . . . . . . . . . 2.3.2 Specifying Method Permissions Using Deployment Descriptors . . . . . . . 2.4 Java Security Frameworks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.1 Java Authentication and Authorization Service (JAAS) . . . . . . . . . . . . . . 2.4.2 Spring Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.3 PicketBox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.4 OWASP Enterprise Security API (ESAPI) . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.5 Apache Shiro . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.6 jGuard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.7 Apache Commons Validator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5 Popular Web Application Frameworks for the Java Programming Language . . . . 2.6 Spring Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6.1 Security Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6.2 Extendibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.7 Apache Struts 1.x . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.7.1 Security Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.7.2 Extendibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.8 Apache Struts 2.x . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.8.1 Security Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.8.2 Extendibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.9 Apache MyFaces Tomahawk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.9.1 Security Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.9.2 Extendibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.10 Wicket . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.10.1 Security Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2 3 6 7 7 8 9 11 12 13 14 15 15 16 17 17 18 20 21 22 23 25 28 28 30 30 31 32 34 36 37 38 38 39 39 41 41

iv

Contents 2.10.2 Extendibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 2.11 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3 Related Work 46

3.1 Declarative Security Policies and Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 3.2 Aspect-Oriented Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 3.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
4 Finding Web Application Vulnerabilities by using Declarative Information 60

4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8 4.9 4.10
5

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Injection Flaws . . . . . . . . . . . . . . . . . . . . . . . . . . Broken Authentication and Session Management . Insecure Direct Object References . . . . . . . . . . . . Security Misconfiguration . . . . . . . . . . . . . . . . . . Insecure Cryptographic Storage . . . . . . . . . . . . . . Failure to Restrict URL Access . . . . . . . . . . . . . . . Insufficient Transport Layer Protection . . . . . . . . Unvalidated Redirects and Forwards . . . . . . . . . . Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

60 61 66 66 68 68 68 71 73 73
76

Finding Privilege Escalation Vulnerabilities by using Declarative Information

5.1 Differences between RBAC and ABAC . . . . . . . . . . . 5.2 Detecting Privilege Escalation Vulnerabilities (RBAC) 5.2.1 Extractable Declarative Information . . . . . . . 5.2.2 Crawling the Web Application . . . . . . . . . . . . 5.2.3 Evaluation of the collected Data . . . . . . . . . . 5.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6 Design

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

76 77 77 79 81 82
83

6.1 Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 Meta Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2.1 Meta Layer XML Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3 Framework Specific Parsers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.1 Struts 1.x Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4 Web Crawler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4.1 HtmlUnit Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4.2 Submitting a HTML Form with HtmlUnit . . . . . . . . . . . . . . . . . . 6.4.3 HtmlUnit JavaScript Performance . . . . . . . . . . . . . . . . . . . . . . . . 6.4.4 Design of the Web Spider Component . . . . . . . . . . . . . . . . . . . . . 6.4.5 Smartly filling out HTML Forms . . . . . . . . . . . . . . . . . . . . . . . . . 6.4.6 Spidering with different User Roles . . . . . . . . . . . . . . . . . . . . . . . 6.4.7 Saving the Crawled Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.5 Working with Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.5.1 Graph Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.5.2 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.6 Saving the Crawled Data in Neo4j . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.7 Analysis Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.7.1 Comparing crawled Data with the <access-restrictions> Tag 6.7.2 Graph Differences - Clustering by User / User Roles . . . . . . . . . . . 6.7.3 Edge Analysis - Clustering by URL Paths . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

83 84 85 91 91 95 96 97 99 100 103 105 106 106 108 109 109 111 111 112 113

v

Contents 6.7.4 Clustering by URL Paths and Re-Colouring 6.7.5 Showing Differences by Re-Colouring . . . . 6.7.6 Software Testing Lifecycle . . . . . . . . . . . . . 6.8 Presentation Layer . . . . . . . . . . . . . . . . . . . . . . . 6.8.1 Java Swing Application . . . . . . . . . . . . . . . 6.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7 Testing

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

115 115 116 117 118 119
120

7.1 Web Crawler - WIVET Results . . . . . . . . . . . . . . . . . . . . . 7.2 Test Environment & Application . . . . . . . . . . . . . . . . . . . 7.3 Struts Test Application . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3.1 Phase 1: Parsing Struts Source-/Configuration Files 7.3.2 Phase 2: Crawling the deployed Web Application . . 7.3.3 Phase 3: Analysing the Results . . . . . . . . . . . . . . . 7.4 WordPress 3.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4.1 Phase 1: Preparing the Meta Layer XML File . . . . . 7.4.2 Phase 2: Crawling the deployed Web Application . . 7.4.3 Phase 3: Analysing the Results . . . . . . . . . . . . . . . 7.5 PrivaSphere Secure Messaging Service . . . . . . . . . . . . . . . 7.5.1 Phase 1: Preparing the Meta Layer XML File . . . . . 7.5.2 Phase 2: Crawling the deployed Web Application . . 7.5.3 Phase 3: Analysing the Results . . . . . . . . . . . . . . . 7.6 False Positives in the Analysis Layer . . . . . . . . . . . . . . . . . 7.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8 Summary

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

120 121 123 123 127 127 129 130 132 133 134 134 136 138 138 140
141

8.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 8.2 Further Steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
9 User Manual & Development Guide 143

9.1 User Manual . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.1.1 Hard- & Software Requirements . . . . . . . 9.1.2 Installation Instructions . . . . . . . . . . . . . 9.1.3 Running the Meta Layer Extraction Tool . 9.1.4 Running the RBAC Spider . . . . . . . . . . . . 9.1.5 Running the RBAC Visualiser . . . . . . . . . 9.1.6 Video Tutorials . . . . . . . . . . . . . . . . . . . . 9.2 Development Guide . . . . . . . . . . . . . . . . . . . . . 9.2.1 Prerequisites . . . . . . . . . . . . . . . . . . . . . 9.2.2 Installing the Development Environment 9.2.3 Building with Maven . . . . . . . . . . . . . . . 9.2.4 Free/Third-Party Libraries Used . . . . . . . 9.2.5 Use logback for debugging purposes . . . 9.3 RBACVisualiser Screenshots . . . . . . . . . . . . . . .
Appendix

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

143 143 143 144 145 146 148 148 148 149 149 149 150 150
160

List of figures List of tables . List of listings Bibliography

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

161 162 164 167

vi

Contents
A B C D E F Original Assignment Meta Layer XML Structure DTD Web Spider UML Class Diagram URL Extractors UML Class Diagram HtmlUnit JavaScript Performance (Runtime Numbers) HtmlUnit JavaScript Performance (Code) 168 171 172 173 174 175

vii

Contents
• Chapter 1: The first Chapter gives a brief overview over the main aspects of this thesis. This includes the motivation behind the thesis and how combining white- and black box testing techniques improve the detection of vulnerabilities in web applications. • Chapter 2: Investigates the current state of the art practices for developing web applications with the Java programming language. Furthermore, the available security features such as access controls, input validation, etc. shall be analysed and compared between the available web application frameworks. This should give a broad overview over the current practices in developing large scale web applications with complex flows, user management features and security restrictions and how developers are able to add declarative information to their web applications which is an important topic for this thesis. Only open source frameworks are being considered because a part of this thesis might be the enhancement of such a framework, which is obviously difficult without the actual source code. • Chapter 3: Presents the current state of the research literature related to adding declarative information to web application source code to improve security. The papers summarised have been found through the Google Scholar search engine. The available research on enhancing web application security declaratively can be roughly divided into three categories: added security through declarative security policies, adding an aspect oriented approach to web application security and static code analysis with the aid of metadata. • Chapter 4: This Chapter tries to identify possible ways to enhance the detection of security vulnerabilities in web applications with an automated security testing tool by incorporating declarative security information. The OWASP Top Ten project has been used to identify the most common web application vulnerabilities and for each one an appropriate solution has been proposed to enhance the detection of such vulnerabilities by using a white box testing approach. Each vulnerability from the OWASP Top Ten project is being shortly described and for each popular web application framework such as Struts 1.x, Struts 2.x and Wicket the framework specific features are shown which help a developer to prevent a particular vulnerability. • Chapter 5: The following Chapter finalises the main idea of this thesis, namely detecting privilege escalation vulnerabilities and misconfigurations in role based access control checks (RBAC) in web applications by using declarative information provided by web application framework specific configuration files. After analysing the current research in the field of enhancing web application security through the use of declarative information, as outlined in Chapter 3, did not reveal any suitable idea worth continuing for this thesis and evaluating possibilities of finding common web application vulnerabilities through the use of declarative information in Chapter 4, the idea of doing role based access control policy checks seems to be the most valuable for a developer or testing agent. • Chapter 6: This Chapter finalises the design for detecting privilege escalation vulnerabilities by considering declarative information in web application frameworks. The Chapter contains the design for the underlying storage solution as well as UML diagrams for the Java classes which have to be developed. • Chapter 7: The Testing Chapter is concerned with functionality testing to verify whether the security testing tool meets the intended specifications and functional requirements laid out in the development and design specifications. The goal is to verify that it is indeed possible and feasable to detect privilege escalation vulnerabilities and misconfigurations in real life web applications by using the developed security testing tool. • Chapter 8: This Chapter recapitulates critically the findings and limitations of this thesis and proposes a list of further steps for the project. • Chapter 9: Contains the user manual and the development guide for the developed security testing tools. The user manual leads new users through the installation process of the tools and helps to get familiar with the various command line arguments. The development guide is for advanced users who want to extend the functionality of the testing tools and build them from source.

viii

1 Introduction
This first Chapter gives a brief overview over the main aspects of this thesis. This includes the motivation behind this thesis and how combining white- and black box testing techniques improve the detection of security related vulnerabilities in web applications.

1.1 Motivation
The Automated Security-Testing Framework (ASTF) which has been developed at the Institute of Applied Information Technology (InIT) as part of a KTI-Project, combines different security and nonsecurity testing tools. The main focus of ASTF lies in doing continuous and reproducible software tests during the lifetime of an application. ASTF relies on third party tools which perform the actual tests. Tools like nmap or Nessus are used to scan for open ports or known security holes. The detection of typical web application vulnerabilities like Cross Site Scripting and SQL Injection are covered by w3af. Yet all of these tools take an external perspective of the test object to derive test cases. This method of testing is also called black box testing. While this method can uncover flaws in the application, one cannot be sure that all existent attack vectors are being tested. White box testing on the other hand takes more of an internal perspective on the test object and uses as much additional information as possible to derive test cases. Such additional information can consist of the actual source code of the application or metadata about the internal structure of the application. One of the future goals of ASTF is to provide a mechanism to conduct white box security tests in an automated way on different kinds of web applications. The objective of this thesis is to build a working web application security testing tool by combining white and black box testing techniques as described in [13] and [12].

1.2 Limitations
The approach described throughout this thesis generates the most benefit in regard to detecting privilege escalation vulnerabilities and misconfigurations in access control checks in web applications that use non-container managed access controls and thus enforce these programmatically in the source code. However, the designed approach is generic enough to be used with web applications that use container managed access controls and is capable of visualising the enforced access controls. The main difference between web applications that use container managed access controls and those who do not is that it is far more likely to find security threats and misconfigurations in web applications that enforce their access controls programmatically because it can be assumed that the container is properly implemented and tested to enforce the access controls as they have been configured by a developer. Using the here described approach would be a bit exaggerated for only visualising the configured and properly enforced access controls in the deployment descriptor of the container but nonetheless possible. The approach is tailored to suit common role based access controls (RBAC) policies and does not work with attribute based access controls (ABAC) policies (more on the differences between RBAC and ABAC can be found in Chapter 5).

Chapter 1. Introduction

| 1

2 Java Web Application Frameworks
The following Chapter investigates the current state of the art practices for developing web applications with the Java programming language. Furthermore, the available security features such as access controls, input validation, etc. shall be analysed and compared between the available web application frameworks. This should give a broad overview over the current practices in developing large scale web applications with complex flows, user management features and security restrictions and how developers are able to add declarative information to their web applications which is an important topic for this thesis. Only open source frameworks are being considered because a part of this thesis might be the enhancement of such a framework, which is obviously difficult without the actual source code.

2.1 Java Technologies for Web Applications
This Section provides an overview of the Java technologies involved in creating Java web applications. Most of the information provided here has been taken from the Sun Developer Network (SDN)1 or the Oracle Java EE 5 Tutorial2 . The Java technologies used to create web applications are a part of the Java EE platform, in addition to many of the Java Platform, Standard Edition (Java SE) classes and packages. Java EE web components are either Servlets or pages created using JSP technology (JSP pages) and/or JavaServer Faces technology. Servlets are Java programming language classes that dynamically process requests and construct responses. JSP pages are text-based documents that execute as Servlets but allow a more natural approach to creating static content. JavaServer Faces technology builds on Servlets and JSP technology and provides a user interface component framework for web applications. Static HTML pages and applets are bundled with web components during application assembly but are not considered web components by the Java EE specification. Server-side utility classes can also be bundled with web components and like HTML pages, are not considered web components. As shown in Figure 2.1, the web tier, like the client tier, might include a JavaBeans component to manage the user input and send that input to enterprise beans running in the business tier for processing. Containers are the interface between a component and the low-level platform-specific functionality that supports the component. Before a web, enterprise bean, or application client component can be executed, it must be assembled into a Java EE module and deployed into its container. The assembly process involves specifying container settings for each component in the Java EE application and for the Java EE application itself. Container settings customise the underlying support provided by the Java EE server, including services such as security, transaction management, Java Naming and Directory Interface (JNDI) lookups, and remote connectivity. One security related feature is that the Java EE security model lets one configure a web component or enterprise bean so
http://java.sun.com/developer/technicalArticles/tools/webapps_1/ [10/11/10] http://download.oracle.com/javaee/5/tutorial/ [10/11/10]

1 2

Chapter 2. Java Web Application Frameworks

| 2

Figure 2.1: Web Tier and Java EE Applications

that system resources are accessed only by authorised users. The deployment process installs Java EE application components in the Java EE containers as illustrated in Figure 2.2. The following types of containers exist: • Java EE server: The runtime portion of a Java EE product. A Java EE server provides EJB and web containers. • Enterprise JavaBeans (EJB) container: Manages the execution of enterprise beans for Java EE applications. Enterprise beans and their container run on the Java EE server. • Web container: Manages the execution of JSP page and Servlet components for Java EE applications. Web components and their container run on the Java EE server. • Application client container: Manages the execution of application client components. Application clients and their container run on the client. • Applet container: Manages the execution of applets. Consists of a web browser and Java plugin running on the client together. Figure 2.3 illustrates the availability of the Java EE 5 platform APIs in each Java EE container type. The following Sections give a brief summary of some of the technologies required by the Java EE platform, and the APIs used in Java EE applications.

2.1.1 Java Servlet API
The Java Servlet API allows a developer to define HTTP-specific classes. A Servlet class extends the capabilities of servers that host applications that are accessed by way of a request-response programming model. Although Servlets can respond to any type of request, they are commonly used to extend the applications hosted by web servers. Thus, a software developer may use a Servlet to add dynamic content to a web server using the Java platform. The generated content is commonly

Chapter 2. Java Web Application Frameworks

| 3

Figure 2.2: Java EE Server and Containers

Figure 2.3: Java EE Platform APIs

Chapter 2. Java Web Application Frameworks

| 4

HTML, but may be other data such as XML. Servlets are the Java counterpart to non-Java dynamic web content technologies such as CGI and ASP.NET. Servlets can maintain state in session variables across many server transactions by using HTTP cookies or URL rewriting. The Servlet API, contained in the Java package hierarchy javax.servlet, defines the expected interactions of a web container and a Servlet. A web container is essentially the component of a web server that interacts with the Servlets. The web container is responsible for managing the lifecycle of Servlets, mapping a URL to a particular Servlet and ensuring that the URL requester has the correct access rights. Listing 2.1 shows a simple Java HTTP Java Servlet which returns a valid HTML document containing the sentence Hello World! and Listing 2.2 shows the necessary deployment descriptor to deploy the Servlet. The deployment descriptor defines that the HelloWorldServlet is implemented by the com.samples.HelloWorld class and that the Servlet can be accessed through the /HelloWorld URL.
1 2 3 4 5 6 7 8 9 10 11 12 13

public final class HelloWorld extends HttpServlet { @Override public void doGet ( HttpServletRequest req , HttpServletResponse resp ) throws ServletException , IOException { PrintWriter out = resp . getWriter () ; out . println ( " < html >\ n " + " <head > < title > Hello World </ title > </ head >\ n " + " <body >\ n " + " <h1 > Hello World ! </ h1 >\ n " + " </ body >\ n " + " </ html > " ) ; } }

Listing 2.1: Simple Java HelloWorld Servlet (Specification 2.5)
1 2 3 4 5 6 7 8 9 10 11 12

... <web - app > < servlet > < servlet - name > HelloWorldServlet </ servlet - name > < servlet - class > com . samples . HelloWorld </ servlet - class > </ servlet > < servlet - mapping > < servlet - name > HelloWorldServlet </ servlet - name > <url - pattern >/ HelloWorld </ url - pattern > </ servlet - mapping > </ web - app >

Listing 2.2: Deployment Descriptor for the HelloWorld Servlet The new Java Servlet 3.0 specification which has been approved in the Java Specification Request (JSR) 315 and can be downloaded from http://jcp.org/en/jsr/detail?id=315 noticeably simplifies the development of Servlets by introducing a set of new Java annotations. • @HandlesTypes - This annotation is used to declare the class types that a ServletContainerInitializer can handle. • @HttpConstraint - This annotation is used within the ServletSecurity annotation to represent the security constraints to be applied to all HTTP protocol methods for which a corresponding HttpMethodConstraint element does NOT occur within the ServletSecurity annotation (see Section 2.2).

Chapter 2. Java Web Application Frameworks

| 5

• @HttpMethodConstraint - This annotation is used within the ServletSecurity annotation to represent security constraints on specific HTTP protocol messages (see Section 2.2). • @MultipartConfig - Annotation that may be specified on a Servlet class, indicating that instances of the Servlet expect requests that conform to the multipart/form-data MIME type. • @ServletSecurity - This annotation is used on a Servlet implementation class to specify security constraints to be enforced by a Servlet container on HTTP protocol messages (see Section 2.2). • @WebFilter - Annotation used to declare a Servlet Filter. • @WebInitParam - This annotation is used on a Servlet or Filter implementation class to specify an initialisation parameter. • @WebServlet - Annotation used to declare a Servlet. As the HelloWorld Servlet from Listing 2.1 is annotated using the @WebServlet annotation as seen in Listing 2.3, it gets initialised during the startup of the web container. Note that the deployment descriptor (as illustrated before in Listing 2.2) is optional in this case. Basically all the configuration elements which could be set through a deployment descriptor file can now be set through the newly provided Java annotations, making the XML deployment descriptor obsolete. This includes initialisation parameters and security constraints.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

@WebServlet ( name = " HelloWorldServlet " , urlPatterns ={ " / HelloWorld " , " / Hello " } , initParams = { @WebInitParam ( name = " param1 " , value = " Hello " ) , @WebInitParam ( name = " param2 " , value = " World ! " ) }) public final class HelloWorld extends HttpServlet { @Override public void doGet ( HttpServletRequest req , HttpServletResponse resp ) throws ServletException , IOException { PrintWriter out = resp . getWriter () ; out . println ( " < html >\ n " + " < head > < title > Hello World </ title > </ head >\ n " + " < body >\ n " + " <h1 > " + getInitParameter ( " param1 " ) + getInitParameter ( " param2 " ) + " </ h1 >\ n " + " </ body >\ n " + " </ html > " ) ; } }

Listing 2.3: Simple Java Servlet (Specification 3.0)

2.1.2 JavaServer Pages Technology
JavaServer Pages (JSP) technology provides a simplified, fast way to create dynamic web content. JSP technology enables rapid development of web-based applications that are server- and platformindependent. JSP technology lets the developer add snippets of Servlet code (see Section 2.1.1) directly into a text-based document. Typically, a JSP page is a text-based document that contains two types of text: • Static data, which can be expressed in any text-based format, such as HTML or XML.

Chapter 2. Java Web Application Frameworks

| 6

• JSP technology elements, which determine how the page constructs dynamic content. The packages involved in creating JSP pages are javax.el, javax.servlet.jsp, javax.servlet. jsp.el and javax.servlet.jsp.tagext. JSPs are compiled into Servlets by a JSP compiler. The compiler either generates a Servlet in Java code that is then compiled by the Java compiler, or it may compile the Servlet to byte code which is directly executable. JSPs can also be interpreted on-the-fly, reducing the time taken to reload changes. Listing 2.4 shows a simple JSP page which returns a valid HTML document containing the sentence Hello World!.
1 2 3 4 5 6 7

<% @ page contentType = " text / html ; charset = UTF -8 " % > < html > < head > < title > Hello World </ title > </ head > < body > <h1 > <% out . println ( " Hello World ! " ) ; % > </ h1 > </ body > </ html >

Listing 2.4: Simple JSP Page

2.1.3 JavaServer Pages Standard Tag Library
The JavaServer Pages Standard Tag Library (JSTL) is a component of the Java EE web application development platform. It extends the JSP specification (see Section 2.1.2) by adding a tag library of JSP tags for common tasks, such as XML data processing, conditional execution, loops and internationalization. JSTL provides an effective way to embed logic within a JSP page without using embedded Java code directly. The use of a standardised tag set, rather than breaking in and out of Java code leads to more maintainable code and enables separation of concerns between the development of the application code and user interface (UI). Listing 2.5 shows how JSTL can be used to dynamically print ten times Hello World! into a valid HTML document.
1 2 3 4 5 6 7 8 9 10

<% @ taglib uri = " http :// java . sun . com / jstl / core " prefix = " c " % > < html > < head > < title > Hello World </ title > </ head > < body > <c : forEach var = " i " begin = " 1 " end = " 10 " > <c: out value = " Hello World ${ i }! " / > < br / > </c : forEach > </ body > </ html >

Listing 2.5: Simple JSTL usage Example

2.1.4 JavaServer Faces Technology
JavaServer Faces (JSF) technology is a UI framework for building web applications. The main components of JavaServer Faces technology involve a GUI component framework, a flexible model for rendering components in various markup languages and technologies and a standard RenderKit for generating HTML markup. JSF is a request-driven Model-View-Control (MVC) web framework based on component driven UI design model, using XML files called view templates or Facelets views. Requests are processed by the FacesServlet, which loads the appropriate view template, builds a component tree, processes events and renders the response (typically HTML) to the client. The state of UI components (and some other objects) is saved at the end of each request (called stateSaving)

Chapter 2. Java Web Application Frameworks

| 7

and restored upon next creation of that view. Several types of state-saving are available, including client-side and server-side state saving. Out of the box, JSF 1.x uses JavaServer Pages (JSP) for its display technology, but can also accommodate other technologies (such as XUL and Facelets). JSF 2 uses Facelets by default for this purpose. Facelets is a more efficient, simple and yet more powerful view description language. Listing 2.6 shows the Java class used by the JSF view in Listing 2.7 to display Hello World! to the client.
1 2 3 4 5 6

public final class HelloWorldBean { private String hello = " Hello World ! " ; public String getHello () { return this . hello ; } }

Listing 2.6: Hello World Java Bean (POJO)
1 2 3 4 5 6 7 8 9 10

<% @ taglib uri = " http :// java . sun . com / jsf / core " prefix = " f " % > <% @ taglib uri = " http :// java . sun . com / jsf / html " prefix = " h " % > < html > < head > < title > Hello World </ title > </ head > < body > <f : view > <h1 > < h : outputText value = " #{ helloWorldBean . hello } " / > </ h1 > </ f : view > </ body > </ html >

Listing 2.7: Hello World JSF Page Projects like MyFaces3 or ICEFaces4 create and maintain an open source JavaServer Faces implementation, along with several component libraries containing UI widgets for building web applications with JSF.

2.2 Securing Java Web Applications
In Java EE, the component containers are responsible for providing application security. A container provides two types of security: declarative and programmatic. The following Sections discuss these concepts in more detail. Using Deployment Descriptors for Declarative Security: Declarative security expresses an application component’s security requirements using deployment descriptors. A deployment descriptor is a XML document that describes the deployment settings of an application, a module, or a component. Because deployment descriptor information is declarative, it can be changed without the need to modify the source code of the web application. At runtime, the Java EE server reads the deployment descriptor and acts upon the application, module, or component accordingly. Deployment descriptors must provide certain structural information for each component if this information has not been provided in annotations or is not to be defaulted. Different types of components use different formats, or schema, for their deployment descriptors. The security elements of deployment descriptors include the following:

3 4

http://myfaces.apache.org/ [10/11/10] http://www.icefaces.org/main/home/ [10/11/10]

Chapter 2. Java Web Application Frameworks

| 8

• Enterprise JavaBeans components use an EJB deployment descriptor that must be named META-INF/ejb-jar.xml and must be contained in the EJB JAR file. The schema for enterprise bean deployment descriptors is provided in the EJB 3.0 Specification (JSR-220), which can be downloaded from http://jcp.org/en/jsr/detail?id=220. • Web Services components use a jaxrpc-mapping-info deployment descriptor defined in JSR 109. This deployment descriptor provides deployment-time mapping functionality between Java and WSDL. In conjunction with JSR 181, JAX-WS 2.0 complements this mapping functionality with development-time Java annotations that control mapping between Java and WSDL. The schema for web services deployment descriptors is provided in Web Services for Java EE (JSR-109), which can be downloaded from http://jcp.org/en/jsr/detail?id=109. • Web components use a web application deployment descriptor named web.xml. The schema for web component deployment descriptors is provided in the Java Servlet 2.5 Specification (JSR-154), Section SRV.13, which can be downloaded from http://jcp.org/en/jsr/ detail?id=154. Using Annotations: Annotations enable a declarative style of programming and so encompass both the declarative and programmatic security concepts. Users can specify information about security within a Java class file using annotations. When the application is deployed, this information is used by the Application Server. Not all security information can be specified using annotations, however. Some information must be specified in the application deployment descriptors. Annotations eliminate the need for maintaining side files that must be kept up to date with changes in source files. Instead the information can be maintained in the source code. Using Programmatic Security: Programmatic security is embedded in an application and is used to make security decisions. Programmatic security is useful when declarative security alone is not sufficient to express the security model of an application. The API for programmatic security consists of two methods of the javax.ejb.EJBContext interface and two methods of the Servlet javax.servlet.http.HttpServletRequest interface. These methods allow components to make business logic decisions based on the security role of the caller or remote user.

2.2.1 Working with Realms, Users, Groups and Roles
Applications often need to protect resources to ensure that only authorised users have access. Authorisation provides controlled access to protected resources. Authorisation is based on identification and authentication. Identification is a process that enables recognition of an entity by a system and authentication is a process that verifies the identity of a user, device, or other entity in a computer system, usually as a prerequisite to allowing access to resources in a system. A realm is defined on a web or application server. It contains a collection of users, which may or may not be assigned to a group, that are controlled by the same authentication policy. An application will often prompt a user for their username and password before allowing access to a protected resource. After the user has entered their username and password, that information is passed to the server, which either authenticates the user and sends the protected resource or does not authenticate the user, in which case access to the protected resource is denied.

Chapter 2. Java Web Application Frameworks

| 9

In some applications, authorised users are assigned to roles. In this situation, the role assigned to the user in the application must be mapped to a group defined on the application server. For a web application, a realm is a complete database of users and groups that identify valid users of a web application (or a set of web applications) and are controlled by the same authentication policy. The Java EE server authentication service can govern users in multiple realms. In case of the Apache Tomcat Application Server5 , principals can be configured in the tomcatusers.xml configuration file (or in the deployment descriptor web.xml). The following snippet shows an excerpt from the tomcat-users.xml configuration file which defines a new role called Employee and a user john.smith with a password secret:

<tomcat-users> ... <role rolename="Employee"/> ... <user username="john.smith" password="secret" roles="Employee"/> </tomcat-users>
The deployment descriptor web.xml of the web application needs to define how users will authenticate themselves to the application. This example uses the FORM authentication method, other means are BASIC, DIGEST and CLIENT-CERT as specified in Version 2.5 of the Java Servlet Specification. The FORM based authentication requires the specification of a login page (that is automatically shown to unauthenticated users when trying to access secured resources), as well as a login error page (shown in case of a bad username/password combination). Finally, in what appears to be a redundant task, the security roles that the application uses. The following excerpt shows the necessary declarations in the web.xml configuration file for a FORM based authentication (more detailed in Section 2.2.2):

... <security-role> <role-name>Employee</role-name> </security-role> <login-config> <auth-method>FORM</auth-method> <realm-name>Secured Application Area</realm-name> <form-login-config> <form-login-page>/login.jsp</form-login-page> <form-error-page>/login.jsp?error</form-error-page> </form-login-config> </login-config> ...
The login form login.jsp is just a simple HTML form that is submitted to a special container managed handler called j_security_check (in case of Tomcat, this is automatically handled by a valve that is implicitly added to the application context). This handler checks for two special request parameters: j_username and j_password. The following snippet shows the login form and the necessary HTML input elements:
5

http://tomcat.apache.org/ [07/04/11]

Chapter 2. Java Web Application Frameworks

| 10

... <form method="post" action="j_security_check"> Username: <input type="text" name="j_username"/> Password: <input type="password" name="j_password"/> <input type="submit" value="Log In"/> </form> ...
2.2.2 Specifying Security Roles Using Deployment Descriptor Elements
Application developers communicate how the security is to be set up for the deployed application declaratively by use of the deployment descriptor mechanism. A deployment descriptor enables an application’s security structure, including roles, access control and authentication requirements to be expressed in a form external to the application. A web application is defined using a standard Java EE web.xml deployment descriptor. A deployment descriptor is a XML schema document that conveys elements and configuration information for web applications. The deployment descriptor must indicate which version of the web application schema (2.4 or 2.5) it is using and the elements specified within the deployment descriptor must comply with the rules for processing that version of the deployment descriptor. Version 2.5 of the Java Servlet Specification, which can be downloaded at http://jcp.org/en/jsr/detail?id=154, contains more information regarding the structure of deployment descriptors. Listing 2.8 is an example of the elements in a deployment descriptor that apply specifically to declaring security for web applications or for resources within web applications.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

<web - app ... > < display - name > A Secure Application </ display - name > < servlet > < servlet - name > catalog </ servlet - name > < servlet - class > com . webapp . CatalogServlet </ servlet - class > < security - role - ref > < role - name > EMP </ role - name > < role - link > Employee </ role - link > </ security - role - ref > </ servlet > < security - role > < role - name > Employee </ role - name > </ security - role > < servlet - mapping > < servlet - name > catalog </ servlet - name > <url - pattern >/ catalog /* </ url - pattern > </ servlet - mapping > < security - constraint > <web - resource - collection > <web - resource - name > CartInfo </ web - resource - name > <url - pattern >/ catalog / cart /* </ url - pattern > < http - method > GET </ http - method > < http - method > POST </ http - method > </ web - resource - collection > < auth - constraint > < role - name > Employee </ role - name >

Chapter 2. Java Web Application Frameworks

| 11

31 32 33 34 35 36 37 38 39 40

</ auth - constraint > < user - data - constraint > < transport - guarantee > CONFIDENTIAL </ transport - guarantee > </ user - data - constraint > </ security - constraint > < login - config > < auth - method > BASIC </ auth - method > </ login - config > </ web - app >

Listing 2.8: Specifying Security Roles using Deployment Descriptor elements • <security-role-ref> (Line 7) The security role reference element contains the declaration of a security role reference in the web application’s code. The declaration consists of an optional description, the security role name used in the code and an optional link to a security role. The security role name specified here is the security role name used in the code. The value of the role-name element must be the String used as the parameter to the HttpServletRequest.isUserInRole(String role) method. The container uses the mapping of security-role-ref to security-role when determining the return value of the call. The security role link specified here contains the value of the name of the security role that the user may be mapped into. The role-link element is used to link a security role reference to a defined security role. The role-link element must contain the name of one of the security roles defined in the security-role elements. • <security-role> (Line 13) A security role is an abstract name for the permission to access a particular set of resources in an application. The security-role element is used with the security-role-ref element to map roles defined in code to roles defined for the web application. • <login-config> (Line 37) The login configuration element is used to specify the user authentication method to be used for access to web content, the realm in which the user will be authenticated and in the case of form-based login, additional attributes. When specified, the user must be authenticated before access to any resource that is constrained by a security constraint is being granted. The types of user authentication methods that are supported include basic, form-based, digest and client certificate.

2.2.3 Specifying Security Roles Using Annotations
In Servlets, one can use the following annotations to secure a web application: @DeclareRoles and @RunAs. The @DeclareRoles annotation is used to define the security roles that comprise the security model of the application. This annotation is specified on a class and it typically would be used to define roles that could be tested (for example, by calling isUserInRole(String)) from within the methods of the annotated class. Listing 2.9 is an example of how this annotation would be used. In this example, Employee is the only security role specified, but the value of this parameter can include a list of security roles specified by the application.

Chapter 2. Java Web Application Frameworks

| 12

1 2 3 4 5 6 7 8 9

@DeclareRoles ( " Employee " ) public final class CalculatorServlet extends HttpServlet { @Override public void doGet ( HttpServletRequest req , HttpServletResponse resp ) { if ( req . isUserInRole ( " Employee " ) ) { // ... } } }

Listing 2.9: @DeclareRoles Annotation Example Specifying @DeclareRoles("Employee") is equivalent to defining the following in the web.xml deployment descriptor:

<security-role> <role-name>Employee</role-name> </security-role>
The syntax for declaring more than one role is shown in the following example:

@DeclareRoles({"Administrator", "Manager", "Employee"})
When a call is made to isUserInRole from the annotated Java class, the caller identity associated with the invocation of the class is tested for membership in the role with the same name as the argument to isUserInRole. If a security-role-ref has been defined for the argument role-name, the caller is tested for membership in the role mapped to the role-name. The @RunAs annotation defines the role of the application during execution in a Java EE container. It can be specified on a class, allowing developers to execute an application under a particular role. The role must map to the user/group information in the container’s security realm. The value element in the annotation is the name of a security role of the application during execution in a Java EE container. Listing 2.10 is an example that uses the @RunAs annotation. The @RunAs annotation is equivalent to the run-as element in the deployment descriptor.
1 2 3 4 5 6 7

@RunAs ( " Administrator " ) public final class CalculatorServlet extends HttpServlet { @Override public void doGet ( HttpServletRequest req , HttpServletResponse resp ) { // ... } }

Listing 2.10: @RunAs Annotation Example

2.2.4 Specifying Security Constraints Using Deployment Descriptor Elements
Security constraints are a declarative way to define the protection of web content. A security constraint is used to define access privileges to a collection of resources using their URL mapping. Security constraints are defined in a deployment descriptor by using the following XML elements (see Listing 2.8): • <web-resource-collection> (Line 23) A web resource collection is a list of URL patterns (the part of a URL after the host name and

Chapter 2. Java Web Application Frameworks

| 13

port which should get constrained) and HTTP operations (the methods within the files that match the URL pattern which should get constrained (for example, POST, GET) that describe a set of resources to be protected. • <auth-constraint> (Line 29) An authorisation constraint establishes a requirement for authentication and names the roles authorised to access the URL patterns and HTTP methods declared by this security constraint. If there is no authorisation constraint, the container must accept the request without requiring user authentication. If there is an authorisation constraint but no roles are specified within it, the container will not allow access to constrained requests under any circumstances. The wildcard character * can be used to specify all role names defined in the deployment descriptor. • <user-data-constraint> (Line 32) A user data constraint establishes a requirement that the constrained requests be received over a protected transport layer connection. This guarantees how the data will be transported between client and server. The choices for type of transport guarantee include NONE, INTEGRAL and CONFIDENTIAL. If no user data constraint applies to a request, the container must accept the request when received over any connection, including an unprotected one. Security constraints only work on the original request URI and not on calls made through a RequestDispatcher (which include <jsp:include> and <jsp:forward>). Inside the application, it is assumed that the application itself has complete access to all resources and would not forward a user request unless it had decided that the requesting user also had access. The Java Servlet Specification defines the request URI as the part of a URL after the host name and port. For example, let’s consider an e-commerce site with a browsable catalog which is accessible for anyone and a shopping cart area that’s only accessible for customers. A developer setts up the paths for the web application so that the pattern /cart/* is being protected but nothing else. Assuming that the application is installed at context path /myapp, the following is true: • http://localhost:8080/myapp/index.jsp is not protected. • http://localhost:8080/myapp/cart/index.jsp is protected.

2.2.5 Specifying Security Constraints Using Annotations
The @ServletSecurity annotation (since version 3.0 of the Servlet Specification) provides an alternative mechanism for defining access control constraints equivalent to those that could otherwise have been expressed declaratively via security-constraint elements in the deployment descriptor (see Listing 2.8). The Servlet container will enforce these constraints on the URL patterns mapped to the Servlets mapped to the annotated class. Listing 2.11 shows an example on how to use the @ServletSecurity annotation. The added constraints enforce that only a user with the customer role is allowed to access the CartServlet.
1 2 3 4 5 6 7 8

@WebServlet ( name = " cart " , urlPatterns ={ " / cart / " }) @ServletSecurity (( httpMethodConstraints = { @HttpMethodConstraint ( value = " GET " , rolesAllowed = " customer " ) , @HttpMethodConstraint ( value = " POST " , emptyRoleSemantic = EmptyRoleSemantic . DENY ) }) ) public final class CartServlet extends HttpServlet { @Override

Chapter 2. Java Web Application Frameworks

| 14

9 10 11 12 13 14 15 16 17 18 19

public void doGet ( HttpServletRequest req , HttpServletResponse resp ) throws ServletException , IOException { // ... } @Override public void doPost ( HttpServletRequest req , HttpServletResponse resp ) throws ServletException , IOException { // ... }

}

Listing 2.11: @ServletSecurity Annotation Example

2.3 Securing Java EE Applications
Enterprise beans are the Java EE components that implement Enterprise JavaBeans (EJB) technology. Enterprise beans run in the EJB container, a runtime environment within the Application Server. Besides the already discussed @DeclareRoles and @RunAs annotations (see Section 2.2.3), beans can be additionally annotated with method permission constraints as discussed in the following Sections.

2.3.1 Specifying Method Permissions Using Annotations
The method permissions for the methods of a bean class can be specified on the class, the business methods of the class or both. Method permissions can be specified on a method of the bean class to override the method permissions value specified on the entire bean class. The following annotations are used to specify method permissions: • @RolesAllowed(list-of-roles) The value of the @RolesAllowed annotation is a list of security role names to be mapped to the security roles that are permitted to execute the specified method or methods. Specifying this annotation on the bean class means that it applies to all applicable business methods of the class. • @PermitAll The @PermitAll annotation specifies that all security roles are permitted to execute the specified method or methods. Specifying this annotation on the bean class means that it applies to all applicable business methods of the class. • @DenyAll The @DenyAll annotation specifies that no security roles are permitted to execute the specified method or methods. Listing 2.12 illustrates the use of these annotations in an enterprise bean for updating a shopping cart in an e-commerce application. Only users belonging to the CartUser role are allowed to update their shopping carts through methods provided by the CartBean.
1 2 3 4 5 6

@Stateful () public class CartBean implements Cart { private String customerName ; private String customerId ; private List < String > contents ;

Chapter 2. Java Web Application Frameworks

| 15

7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

@RolesAllowed ( " CartUser " ) public void addBook ( final String title ) { // ... } @RolesAllowed ( " CartUser " ) public void removeBook ( final String title ) throws BookException { // ... } @DenyAll public void debug () { // ... }

}

Listing 2.12: @RolesAllowed Annotation Example

2.3.2 Specifying Method Permissions Using Deployment Descriptors
Method permissions can be defined by using the method-permission element in the deployment descriptor as discussed below: • Each method-permission element includes a list of one or more security roles and a list of one or more methods. All the listed security roles are allowed to invoke all the listed methods. Each security role in the list is identified by the role-name element. Each method (or set of methods) is identified by the method element. • The method permissions relation is defined as the union of all the method permissions defined in the individual method-permission elements. • A security role or a method can appear in multiple method-permission elements. There are three legal styles for composing the method element: • The first style is used for referring to all of the business interface, home interface, component interface and web service endpoints methods of a specified bean.

<method> <ejb-name>CartBean/ejb-name> <method-name>*</method-name> </method>
• The second style is used for referring to a specified method of the business interface, home interface, component interface or web service endpoints methods of the specified enterprise bean. If the enterprise bean contains multiple methods with the same overloaded name, the element of this style refers to all of the methods with the overloaded name.

<method> <ejb-name>CartBean</ejb-name> <method-name>addBook</method-name> </method>
• The third style is used for referring to a specified method within a set of methods with an overloaded name. The method must be defined in the business interface, home interface,

Chapter 2. Java Web Application Frameworks

| 16

component interface or web service endpoints methods of the specified enterprise bean. If there are multiple methods with the same overloaded name, however, this style refers to all of the overloaded methods. All of the parameters are the fully-qualified Java types, for example, java.lang.String.

<method-permission> <role-name>CartUser</role-name> <method> <ejb-name>CartBean</ejb-name> <method-name>addBook</method-name> <method-params> <method-param>String</method-param> </method-params> </method> </method-permission>

2.4 Java Security Frameworks
The following Sections look at various Java security frameworks which can be used to secure Java web or EE applications. Most of the described security frameworks offer programmatic security features with a few exceptions that can be integrated into an application container to enforce security constraints.

2.4.1 Java Authentication and Authorization Service (JAAS)
The Java Authentication and Authorization Service (JAAS) was introduced as an optional package (extension) to the Java 2 SDK, Standard Edition (J2SDK), v 1.3. JAAS was integrated into the J2SDK 1.4. JAAS can be used for two purposes: • for authentication of users, to reliably and securely determine who is currently executing Java code, regardless of whether the code is running as an application, an applet, a bean or a servlet; and • for authorisation of users to ensure they have the access control rights (permissions) required to do the actions performed. For an application developer, JAAS is a standard library that provides: a representation of identity (Principal) and a set of credentials (Subject), a login service that will invoke the application callbacks to ask the user things like username and password and a service that tests if a Subject was granted a permission by an administrator. For the security system integrator, JAAS provides the following interfaces: to provide an identity namespace to applications, to attach credentials to threads (Subject) and for developing login modules. These modules invoke callbacks to query the user, check their response and generate a Subject. JAAS authentication is performed in a pluggable fashion. This permits applications to remain independent from underlying authentication technologies. New or updated authentication technologies can be plugged under an application without requiring modifications to the application itself. Applications enable the authentication process by instantiating a LoginContext object, which in turn references a Configuration to determine the authentication technology(ies) or LoginModule(s),

Chapter 2. Java Web Application Frameworks

| 17

to be used in performing the authentication. A typical LoginModule may prompt for and verify a username and password. Others may read and verify a voice or fingerprint sample6 . To authenticate a subject (user or service), the following steps are performed: 1. An application instantiates a LoginContext. 2. The LoginContext consults a Configuration to load all of the LoginModules configured for that application. 3. The application invokes the LoginContext’s login method. 4. The login method invokes all of the loaded LoginModules. Each LoginModule attempts to authenticate the subject. Upon success, LoginModules associate relevant Principals and credentials with a Subject object that represents the subject being authenticated. 5. The LoginContext returns the authentication status to the application. 6. If authentication succeeded, the application retrieves the Subject from the LoginContext.

2.4.2 Spring Security
The Spring Security project started in 2003 (formerly known as “Acegi Security”) and turned into a sophisticated authentication and access control system and became widely adopted as the standard solution for securing Spring Framework based applications (see [26, p.5]). Spring Security can also be used with different MVC frameworks such as Struts. Spring Security exists to fill a gap in the universe of Java third-party libraries [...] Standards such as Java Authentication and Authorization Service (JAAS) or Java EE Security do offer some ways of performing some of the same authentication and authorization functions, but Spring Security is a winner because it packages up everything you need to implement a top-to-bottom application security solution in a concise and sensible way. Additionally, Spring Security appeals to many because it offers out of the box integration with many common enterprise authentication systems; so it’s adaptable to most situations with little effort (beyond configuration) on the part of the developer. [26, p.18]) Spring Security’s primary influence over an application is through a series of ServletRequest filters. The DelegatingFilterProxy is a Servlet filter that allows Spring Security to wrap all application requests and ensure that they are appropriately secured (added to the deployment descriptor web.xml). Listing 2.13 shows an example Spring Security XML configuration file called *-security.xml, whereas the wildcard stands for the name of the web application. This style of configuration, using a Spring Security-specific XML dialect is known as the security namespace style, named after the XML namespace (http://www.springframework.org/schema/security) associated with the XML configuration elements. Listing 2.13 adds various constraints to the web application based on URL patterns such as restricting users to access resources available through the following URL /secure/extreme/** if they do not belong to the ROLE_SUPERVISOR role.
1 2 3

<? xml version ="1.0" encoding =" UTF -8"? > < beans ... > < http use - expressions =" true " >
6

http://download.oracle.com/javase/6/docs/technotes/guides/security/jaas/JAASRefGuide.html
[11/11/10]

Chapter 2. Java Web Application Frameworks

| 18

4 5 6 7 8 9 10 11 12 13 14 15 16 17

< intercept - url pattern ="/ login . do " access =" permitAll "/ > < intercept - url pattern ="/*" access =" hasRole ( ’ ROLE_USER ’) "/ > < intercept - url pattern ="/ secure / extreme /**" access =" hasRole ( ’ ROLE_SUPERVISOR ’) "/ > < intercept - url pattern ="/ secure /**" access =" isAuthenticated () " requires - channel =" https "/ > <!- - Uncomment to limit the number of sessions a user can have --> < session - management invalid - session - url ="/ timeout . jsp " > < concurrency - control max - sessions ="1" error - if - maximum - exceeded =" true " / > </ session - management > </ http > </ beans : beans >

Listing 2.13: Sample Spring Security XML Configuration File Spring offers the @PreAuthorize annotation for ensuring that a particular Java method can only be executed when the user is in the specified user group. Listing 2.14 shows an example on how to use the @PreAuthorize annotation for securing a method which a user can use to change his password. The security constraint enforces that users must be logged in as a valid user of the system before they are allowed to change a password. This is all that is required to ensure that valid, authenticated users can access the change password function. Spring Security will use a runtime aspect oriented programming (AOP) pointcut to execute before advice on the method and throws an AccessDeniedException if the security constraints are not met.
1 2 3 4

public interface IUserService { @PreAuthorize ( " hasRole ( ’ ROLE_USER ’) " ) public void changePassword ( String username , String password ) ; }

Listing 2.14: @PreAuthorize Annotation Example The SpEL (The Spring Expression Language) binding used by the Spring Security method annotations supports more sophisticated expressions, including expressions that incorporate method parameters. For example, it might make sense to restrict the changePassword method from Listing 2.14 further, so that a user attempting to invoke the method must satisfy two constraints: • The user must be attempting to change their own password or • the user is an administrator (in which case it is valid for the user to change anyone’s password, possibly through an administrative interface) Listing 2.15 shows how the username of the principal is being checked against the username method parameter (#username - the method parameter name prefixed with a # symbol).
1 2

@PreAuthorize ( " # username == principal . username and hasRole ( ’ ROLE_USER ’) " ) public void changePassword ( String username , String password ) ;

Listing 2.15: @PreAuthorize Annotation Example with SpEL Directive Two final Spring Security-dependent annotations are @PreFilter and @PostFilter, which are used to apply security-based filtering rules to Collections or Arrays (with @PostFilter only). This type of functionality is referred to as security trimming or security pruning and involves using the principal’s security credentials at runtime to selectively remove members from a set of objects. This filtering is performed using SpEL expression notation within the annotation declaration.

Chapter 2. Java Web Application Frameworks

| 19

Listing 2.16 illustrates how a collection of categories are being filtered by the use of the @PostFilter annotation. The filterObject is re-bound to the SpEL context for each element in the Collection. The SpEL expression must return a Boolean value. If the expression evaluates to true, the object will remain in the Collection, while if the expression evaluates to false, the object will be removed.
1 2 3

@PostFilter ( " (! filterObject . customersOnly ) or ( filterObject . customersOnly and hasRole ( ’ ROLE_USER ’) ) " ) Collection < Category > getCategories () ;

Listing 2.16: @PostFilter Annotation Example The @PreFilter annotation can be applied to a method to filter Collection elements that are passed into a method based on the current security context. Functionally, once it has a reference to a Collection, this annotation behaves exactly the same as the @PostFilter annotation, with a couple of exceptions: • @PreFilter supports only Collection arguments and does not support Array arguments. • @PreFilter takes an additional, optional attribute filterTarget, which is used to specifically identify the method parameter to filter when the annotated method has more than one argument. Listing 2.17 shows how the @PreFilter annotation can be added to a method in order to filter the collection passed to it.
1 2 3 4

@PreFilter ( " (! filterObject . customersOnly ) or ( filterObject . customersOnly and hasRole ( ’ ROLE_USER ’) ) " ) public Collection < Category > filterCategories ( Collection < Category > categories ) ;

Listing 2.17: @PreFilter Annotation Example

2.4.3 PicketBox
PicketBox7 (formerly JBoss Security) is a security framework for Java applications. The features available are: Authentication, Authorization/Access Control, Auditing and Mapping (Principal/Roles/Attribute). PicketBox is a simple framework with a single configuration file. PicketBox and Spring Security are very similar but PicketBox has a slightly different philosophy behind it. While Spring Security does not use the features of container managed security, PicketBox on the other hand tries to integrate with containers such as JBoss Application Server seamlessly so that applications using PicketBox can have seamless security into Java EE components such as EJB3. PicketBox is best used together with the Seam 3 web development framework. Seam 3 will utilise PicketBox as its security foundation (not yet released). PicketBox basically enhances the container managed security features by providing additional authentication modules. Authentication of users can be done as seen in Section 2.2.1 with a form pointing to j_security_check. In order to use the features of PicketBox, a developer has to define the appropriate PicketBox login module in the application server context (context.xml).

7

http://www.jboss.org/picketbox [3/12/10]

Chapter 2. Java Web Application Frameworks

| 20

2.4.4 OWASP Enterprise Security API (ESAPI)
ESAPI (The OWASP Enterprise Security API)8 is a free, open source web application security control library that makes it easier for programmers to write lower-risk applications. The ESAPI libraries are designed to make it easier for programmers to retrofit security into existing applications. The ESAPI libraries also serve as a solid foundation for new development. The API contains several input validation helpers, a web application firewall (WAF), modules for preventing Clickjacking9 attacks, modules for form-based user authentication, methods for enforcing access control lists, etc. The configuration is done through text files which are placed inside the WEB-INF folder of a Java web application. The code snippet in Listing 2.18 shows how to use the input validation feature of the ESAPI library to prevent Code Injection Attacks.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

try { String cleanComment = ESAPI . validator () . getValidInput ( " comment " , // // // // A descriptive name of the parameter that you are validating . This value is used by any logging or error handling that is done with respect to the value passed in .

request . getParameter ( " comment " ) , // The actual user input // data to validate . " CommentRegex " , // // // // 300 , false , The regular expression name that maps to the actual regular expression from " ESAPI . properties ". e . g .: CommentRegex =^[\\ s \\ p { L }\\ p { N }.]{0 ,1024} $

// The maximum post - canonicalized String length allowed . // If allowNull is true then an input that is NULL or an // empty string will be legal . If allowNull is false then // NULL or an empty String will throw a ValidationException .

// If canonicalize is true then input will be // canonicalized before validation } catch ( ValidationException e ) { // do error handling here ... }

true ) ;

Listing 2.18: ESAPI Example on how to do Input Validation The ESAPI web application firewall (WAF) implemented as a Servlet filter is driven by a XML policy file that tells it what rules to enforce in the application. These rules can do a number of things, from simple virtual patching to complex authorisation enforcement with BeanShell scripts10 . The <authentication-rules> section allows the WAF to enforce typical J2EE authentication requirements. A typical J2EE application authentication pattern involves making sure a session variable exists inside some base action class. If the variable does not exist, the request is considered to be unauthenticated and is handled appropriately. Listing 2.19 shows an example which makes the whole web application only accessible for authenticated users through the path="/.*" directive on
http://www.owasp.org/index.php/Category:OWASP_Enterprise_Security_API [15/11/10] http://www.sectheory.com/clickjacking.htm [11/04/11] 10 http://jcp.org/en/jsr/detail?id=274 [11/04/11]
8 9

Chapter 2. Java Web Application Frameworks

| 21

line 1 with the exception of the login and index pages etc. by checking if the UserAuthKey exists in the user’s session.
1 2 3 4 5 6 7 8 9 10 11

... < authentication - rules path ="/.*" key =" UserAuthKey " > < path - exception >/ </ path - exception > < path - exception >/ index . html </ path - exception > < path - exception >/ login . jsp </ path - exception > < path - exception >/ index . jsp </ path - exception > < path - exception type =" regex " >/ images /.* </ path - exception > < path - exception type =" regex " >/ css /.* </ path - exception > < path - exception type =" regex " >/ help /.* </ path - exception > </ authentication - rules > ...

Listing 2.19: ESAPI WAF Configuration Example on how to enforce Authorisation Constraints

2.4.5 Apache Shiro
The Apache Shiro project has been formerly known as JSecurity and Apache KI. Apache Shiro is a powerful and easy-to-use Java security framework that performs authentication, authorization, cryptography and session management. With Shiro’s easyto-understand API, you can quickly and easily secure any application - from the smallest mobile applications to the largest web and enterprise applications11 . Shiro offers quite similar features as the OWASP Enterprise Security API (see Section 2.4.4). Especially for web applications Shiro offers a set of Servlet filters for securing Java web applications. These filters are being configured in the deployment descriptor and include the following: • AnonymousFilter (anon): A filter that allows access to a path immediately without performing security checks of any kind. This filter is useful primarily in exclusionary policies. • FormAuthenticationFilter (authc): Requires the requesting user to be authenticated for the request to continue, and if they are not, forces the user to login by redirecting them to the configured loginUrl. • BasicHttpAuthenticationFilter (authcBasic): Requires the requesting user to be authenticated for the request to continue, and if they are not, requires the user to login via the HTTP Basic protocol-specific challenge. Upon successful login, they are allowed to continue on to the requested resource or URL. This implementation is a “clean room” Java implementation of Basic HTTP Authentication specification per RFC 2617 [16]. • PermissionsAuthorizationFilter (perms): Filter that allows access if the current user has the permissions specified by the mapped value or denies access if the user does not have all of the permissions specified. • PortFilter (port): A Filter that requires the request to be on a specific port, and if not, redirects to the same URL on that port. • HttpMethodPermissionFilter (rest): A filter that translates a HTTP request’s method (e.g. GET, POST, etc.) into a corresponding action (verb) and uses that verb to construct a permission that will be checked to determine access. This filter is primarily provided to support REST
11

http://shiro.apache.org/ [19/11/10]

Chapter 2. Java Web Application Frameworks

| 22

environments where the type (method) of request translates to an action being performed on one or more resources. This paradigm works well with Shiro’s concepts of using permissions for access control and can be leveraged to easily perform permission checks. This filter functions as follows: – The incoming HTTP request’s method (GET, POST, PUT, DELETE, etc.) is discovered. – The method is translated into a more ’application friendly’ verb, such as create, edit, delete, etc. – The verb is appended to any configured permissions for the currently matching path. – If the current Subject isPermitted to perform the resolved action, the request is allowed to continue. • RolesAuthorizationFilter (roles): Filter that allows access if the current user has the roles specified by the mapped value or denies access if the user does not have all of the roles specified. • SslFilter (ssl): The port property defaults to 443 and also additionally guarantees that the request scheme is always HTTPS (except for port 80, which retains the HTTP scheme). • UserFilter (user): Filter that allows access to resources if the accessor is a known user, which is defined as having a known principal. This means that any user who is authenticated or remembered via a remember me feature will be allowed access from this filter. If the accessor is not a known user, then they will be redirected to the loginUrl. The main advantage Shiro offers as compared to other existing Java security frameworks is the ability to define ad-hoc filter chains for any matching URL path in the web application. This is being configured in the [url] section of the Shiro INI file. Listing 2.20 shows an excerpt from that configuration file.
1 2 3 4 5 6 7 8 9 10

... [ urls ] / index . html = anon / user / create = anon / user /** = authc / admin /** = authc , roles [ administrator ] / rest /** = authc , rest / remoting / rpc /** = authc , perms [" remote : invoke "] / account /** = ssl , authc

Listing 2.20: Apache Shiro Ad-Hoc Filter Chain Configuration Example For example, line 10 from Listing 2.20 states that “Any request to the application’s path of /account or any of it’s sub paths (/account/foo, /account/bar/baz, etc.) will trigger the ssl, authc filter chain”. Note that all path expressions are relative to the application’s context root.

2.4.6 jGuard
The following statement has been taken from the jGuard project site12 : jGuard is a library that provides EASY security (authentication and authorization) for Java web applications. It is built over the stable and mature JAAS framework, which is part
12

http://jguard.xwiki.com/xwiki/bin/view/Main/WebHome [23/11/10]

Chapter 2. Java Web Application Frameworks

| 23

of the JAVA J2SE API. jGuard is very flexible and allows several different ways to configure those mechanisms for authentication and authorization, i.e., in a relational database, XML files or LDAP service. The net.sf.jguard.filters.AccessFilter class is the main integration between the web application and jGuard. Access control checks are being done through the AccessFilter. The AccessFilter is a Servlet filter and has the following duties: Configure the access control of the web application, authenticate the users, deny unauthorised access to a URL, log off the user (the user needs to log in again). After the roles (principals) have been configured in the jGuardUsersPrincipals.xml file it is necessary to define the permissions related to these principals in jGuardPrincipalsPermissions.xml. These jGuard XML configuration files need to be placed inside the WEB-INF folder of the web application. Listing 2.21 shows the jGuard jGuardPrincipalsPermissions.xml configuration file which is needed in order to specify which user roles are allowed to access which resources of the web application. Frankly, the configuration seems to be tedious to maintain for larger web applications.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38

... < configuration > < permissions > < domain > <!- - this domain will hold all permissions that everybody will be able to access . --> < name > public </ name > < permission > < name > LoginForm </ name > < class > net . sf . jguard . core . authorization . permissions . URLPermission </ class > < actions > < action >/ LoginForm . do </ action > < action > ANY </ action > </ actions > </ permission > ... </ domain > < domain > <!- - this domain will hold all permissions that only authenticated people having a related role will be able to access --> < name > secured </ name > < permission > < name > Authenticated </ name > < class > net . sf . jguard . core . authorization . permissions . URLPermission </ class > < actions > < action >/ Authenticated . do </ action > < action > ANY </ action > < action > you have been authenticated </ action > </ actions > </ permission > </ domain > </ permissions >

Chapter 2. Java Web Application Frameworks

| 24

39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59

< principals > < principal > < name > guest </ name > < class > net . sf . jguard . core . principals . RolePrincipal </ class > < permissionsRef > < permissionRef name =" LoginForm "/ > ... </ permissionsRef > </ principal > < principal > < name > admin </ name > < class > net . sf . jguard . core . principals . RolePrincipal </ class > < permissionsRef > < domainRef name =" secured "/ > </ permissionsRef > </ principal > </ principals > </ configuration >

Listing 2.21: jGuard User Permissions Configuration Example

2.4.7 Apache Commons Validator
Commons Validator13 is an input validation component devised (originally) by David Winterfeldt as an add-on for the Struts framework. Its functionality has since been split so that Validator can serve as the basis for an independent component and is now part of Jakarta Commons. Commons Validator provides two distinct sets of functionality: A configurable (typically XML) validation engine and reusable “primitive” validation methods. The Commons Validator is a basic validation framework that lets a developer define validation rules for a JavaBean in a XML file. Validators as well as the validation definition can also be defined in the XML file. An example of a validator would be defining what method and class will be called to perform the validation for a required field. Validation rules can be grouped together based on locale and a JavaBean/Form that the rules are associated with. The framework has basic support for user defined constants which can be used in some field attributes. The configuration of the validation rules are being done in a XML file called validation.xml which has to be placed inside the WEB-INF folder of a Java web application. The following example is based on the Struts 1.x Framework. The following snippet shows an excerpt from a validation.xml configuration file which validates two password input fields. This might be necessary for a user registration process where the user has to choose a password for his account and to make sure that the user did not make a typing error, he has to repeat his password in a second field. The validation routine checks if the two password fields meet the minimum length requirements and if the entered passwords are identical:

... <formset> <form name="userRegistrationForm"> ... <field property="password" depends="required,minlength,twofields"> <arg key="prompt.password"/>
13

http://commons.apache.org/validator/ [3/12/10]

Chapter 2. Java Web Application Frameworks

| 25

<arg name="minlength" key="${var:minlength}"/> <arg name="twofields" key="prompt.password2"/> <var> <var-name>minlength</var-name> <var-value>8</var-value> </var> <var> <var-name>secondProperty</var-name> <var-value>password2</var-value> </var> </field> </form> </formset>
Whereas the <form name="userRegistrationForm"> corresponds to the form bean definition in the struts-config.xml configuration file as seen below:

<struts-config> <form-beans> <form-bean name="userRegistrationForm" type="com.webapp.UserRegistrationActionForm"/> </form-beans> ... </struts-config>
Validator ships with a set of pre-defined validators, as follows: • required - mandatory field validation. • validwhen - validator for checking one field against another. • minlength - validate input data is not less than a specified minimum length. Requires a minlength variable. • maxlength - validate input data does not exceed a specified maximum length. Requires a maxlength variable. • mask - validate format according to a regular expression. Requires a mask variable to specify the regular expression. • byte - validates that a field can be converted to a Byte. • short - validates that a field can be converted to a Short. • integer - validates that a field can be converted to an Integer. • long - validates that a field can be converted to a Long. • float - validates that a field can be converted to a Float. • double - validates that a field can be converted to a Double.

Chapter 2. Java Web Application Frameworks

| 26

• date - validates that a field can be converted to a Date. This validator uses java.text. SimpleDateFormat to parse the date and optionally either a datePattern or datePatternStrict variable can be used. If no pattern is specified the default short date format is assumed. The difference between using the datePatternStrict and datePattern variables is that datePatternStrict checks additionally that the input data is the same length as the pattern specified (so for example 1/1/2011 would fail with a pattern of MM/dd/yyyy). • intRange - validates that an integer field is within a specified range. Requires min and max variables to specify the range. This validator depends on the integer validator which must also be in the field’s depends attribute. • longRange - validates that a long field is within a specified range. Requires min and max variables to specify the range. This validator depends on the long validator which must also be in the field’s depends attribute. • floatRange - validates that a float field is within a specified range. Requires min and max variables to specify the range. This validator depends on the float validator which must also be in the field’s depends attribute. • doubleRange - validates that a double field is within a specified range. Requires min and max variables to specify the range. This validator depends on the double validator which must also be in the field’s depends attribute. • creditCard - validate credit card number format. • email - validate email address format. • url - validates URL format. Has four optional variables (allowallschemes, allow2slashes, nofragments and schemes) which can be used to configure this validator. – allowallschemes - specifies whether all schemes are allowed. Valid values are true or false (default is false). If this is set to true then the schemes variable is ignored. – allow2slashes - specifies whether double ’/’ characters are allowed. Valid values are true or false (default is false). – nofragments - specifies whether fragments are allowed. Valid values are true or false (default is false - i.e. fragments are allowed). – schemes - use to specify a comma separated list of valid schemes. If not specified then the defaults are used which are http , https and ftp.

<field property="custUrl" depends="url"> <arg position="0" key="customer.url"/> <var> <var-name>nofragments</var-name> <var-value>true</var-value> </var> <var> <var-name>schemes</var-name> <var-value>http,https,telnet,file</var-value> </var> </field>

Chapter 2. Java Web Application Frameworks

| 27

2.5 Popular Web Application Frameworks for the Java Programming Language
Deciding which web application framework is the most popular or widely used among application developers is a very difficult task and can not be answered scientifically. Using services like Google Trends14 , the Google AdWords Keyword15 or the Google Insights16 tool it is however possible to give a rough estimate about how popular a particular web framework is by comparing the number of times people actually typed the name of the framework into the Google search engine. Another estimate whether a web framework is more or less popular than another can be found by comparing the number of available books (for instance from Amazon.com) for a particular framework. Table 2.2 contains a subset of frameworks from the "Open Source Web Frameworks in Java"17 list at Java-Source.net, the roughly estimates of Google searches per month and the roughly number of available books from Amazon.com. Many of the frameworks listed on Java-Source.net do not have a working project page any more and thus considered dead. Furthermore, frameworks which have not been updated in the last 6 years are also considered dead. Analysing the gathered data in Table 2.2 four frameworks clearly stand out from the rest: Spring Framework, Apache Struts 1.x or 2.x, the Google Web Toolkit18 , the JavaServer Faces technology (respectively the popular Apache MyFaces Tomahawk implementation will be covered in the following security analysis of the frameworks) and the Apache Wicket framework. Each following web application framework Section will consist of an broad overview over the framework, the available security features and the features regarding extendibility.

2.6 Spring Framework
The following statement has been taken from the Spring Framework project site19 : Spring is a layered Java/J2EE application platform, based on code published in [18]. Spring includes: a lightweight container [. . . ], a common abstraction layer for transaction management [. . . ], a JDBC abstraction layer [. . . ] and a flexible MVC web application framework [. . . ] The Spring Framework features its own MVC framework. The Spring developers decided to write their own web framework as a reaction to what they perceived as the poor design of the popular Apache Struts Web Application Framework20 , as well as deficiencies in other available frameworks. In particular, they felt there was insufficient separation between the presentation and request handling layers and between the request handling layer and the model (see [18, Chap. 12]). Like Struts, Spring MVC is a request-based framework. The framework defines strategy interfaces for all of the responsibilities which must be handled by a modern request-based framework. The goal of each interface is to be simple and clear so that it is easy for Spring MVC users to write their own implementations if they choose so. MVC paves the way for cleaner front end code. All interfaces are tightly coupled to the Java Servlet API. This coupling makes sure that the features of the Java Servlet API remain available to developers while offering a high abstraction framework to ease working
http://www.google.com/trends [09/11/2010] https://adwords.google.com/select/KeywordToolExternal [09/11/2010] 16 http://www.google.com/insights/search/ [10/11/10] 17 http://java-source.net/open-source/web-frameworks [09/11/10] 18 http://code.google.com/webtoolkit/ [10/11/10] 19 http://www.springsource.org/about [10/11/10] 20 http://www.theserverside.com/news/1364527/Introduction-to-the-Spring-Framework [11/11/10]
14 15

Chapter 2. Java Web Application Frameworks

| 28

Popularity
(# of Google searches / month) (Last release date) (# of books in Amazon.com)

Community 15/11/10 18/10/10 15/05/08 06/08/09 28/10/10 30/09/09 01/08/10 28/06/09 01/08/10 01/04/10 16/06/09 28/07/10 12/01/09 07/11/10 20/10/10 16/12/09 16/08/10 13/06/07 01/11/10 12/06/08 23/12/09 25/09/10 21/07/07 17/09/10 09/04/08 02/11/10 ~1 ~0 ~4 ~0 ~14 ~14 ~0 ~22+ ~7 ~0 ~0 ~1 ~0 ~1 ~17 ~1 ~20 ~0 ~1 ~0 ~0 ~6 ~1 ~3 ~0 ~2

Learning

AppFuse AribaWeb Cocoon Echo Web Framework Google Web Toolkit ICEfaces Induction JavaServer Faces JBoss Seam JPublish jZeno OpenXava Pustefix Restlet Spring Framework Stripes Apache Struts Strecks Tapestry Turbine Makumba Apache MyFaces Tomahawk Webwork Wicket WingS Zkoss

~14’800 ~720 ~260 ~260 ~141’000 ~90’500 ~210 ~550’000 ~40,500 ~110 ~22 ~2’400 ~4’400 ~18’100 ~372’500 ~1’900 ~134’880 ~5,400 ~13’900 ~720 ~880 ~43’800 ~49’500 ~24’420 ~0 ~9’900

Chapter 2. Java Web Application Frameworks

Table 2.2: Popularity of Open Source Java Web Application Frameworks (November 2010)

| 29

with said API. Spring also offers the functionality to exchange the underlying MVC framework which allows the developer to use Spring as a library and frameworks such as Struts or Webworks as the underlying MVC framework21 . Spring 2.5 introduced an annotation-based programming model for MVC controllers that uses annotations such as @RequestMapping, @RequestParam, @ModelAttribute and so on. Controllers implemented in this style do not have to extend specific base classes or implement specific interfaces. Controllers map relative URLs to particular Java classes implementing the desired functionality by the developer through the use of Java annotations. As shown in Listing 2.22, the @Controller and @RequestMapping annotations allow flexible method names and signatures. @Controller and @RequestMapping form the basis for the Spring MVC implementation.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

@Controller @RequestMapping ( value = " / account " ) public final class AccountController { private Map < Long , Account > accounts = new ConcurrentHashMap < Long , Account >() ; @RequestMapping ( method = RequestMethod . GET ) public String getCreateForm ( Model model ) { // ... return " account / createForm " ; } @RequestMapping ( method = RequestMethod . POST ) public String create ( Account account , BindingResult result ) { // ... return " redirect :/ account / " + account . getId () ; } @RequestMapping ( value =" { id } " , method = RequestMethod . GET ) public String getView ( @PathVariable Long id , Model model ) { // ... return " account / view " ; }

}

Listing 2.22: Spring MVC Controller Example Spring provides a couple of out of the box solutions for JSP and JSTL views. Using JSP or JSTL is done using a normal view resolver defined in the WebApplicationContext. See Section 2.1.3 for a JSTL example.

2.6.1 Security Features
The Spring Framework does not provide any kind of security features, however this is realised by the sub project called Spring Security (see Section 2.4.2).

2.6.2 Extendibility
Spring’s handler mapping mechanism has a notion of handler interceptors, which can be extremely useful when specific functionality is needed for certain requests, for example, checking for a principal. Interceptors located in the handler mapping must implement HandlerInterceptor
21

http://static.springsource.org/spring/docs/3.0.x/spring-framework-reference/html/mvc.html
[11/11/10]

Chapter 2. Java Web Application Frameworks

| 30

interface from the org.springframework.web.portlet package. The HandlerInterceptor is basically a workflow interface that allows for customised handler execution chains. Applications can register any number of existing or custom interceptors for certain groups of handlers, to add common pre-processing behaviour without needing to modify each handler implementation. A HandlerInterceptor is basically similar to a Servlet filter, but in contrast to the latter it allows custom pre-processing with the option to prohibit the execution of the handler itself and custom post-processing. Servlet filters are more powerful; for example they allow for exchanging the request and response objects that are handed down the chain. Note that a Servlet filter gets configured in web.xml, a HandlerInterceptor in the application context. Just like the Servlet version, the HandlerInterceptor interface defines three methods: one that will be called before the actual handler will be executed (preHandle), one that will be called after the handler is executed (postHandle) and one that is called after the complete request has finished (afterCompletion). These three methods should provide enough flexibility to do all kinds of preand post- processing. The preHandle method returns a Boolean value. A developer can use this method to break or continue the processing of the execution chain. When this method returns true, the handler execution chain will continue. When it returns false, the DispatcherPortlet assumes the interceptor itself has taken care of requests (and, for example, rendered an appropriate view) and does not continue executing the other interceptors and the actual handler in the execution chain. The postHandle method is only called on a RenderRequest. The preHandle and afterCompletion methods are called on both an ActionRequest and a RenderRequest. If business logic is needed in these methods for just one type of request, the developer has to check what kind of request it is before processing it. Listing 2.23 shows a simple customised HandlerInterceptor which breaks the workflow chain if the request has been received on port 80 on the application server.
1 2 3 4 5 6 7 8 9 10 11

public final class MyInterceptor implements HandlerInterceptor { @Override public boolean preHandleAction ( ActionRequest request , ActionResponse response , Object handler ) { if ( request . getServerPort () == 80) { return false ; } return true ; }

}

Listing 2.23: Spring HandlerInterceptor Example

2.7 Apache Struts 1.x
Apache Struts is an open source web application framework for developing Java EE web applications. It uses and extends the Java Servlet API to encourage developers to adopt a MVC architecture. Struts provides the controller (a Servlet known as ActionServlet) and facilitates the writing of templates for the view or presentation layer (typically in JSP, but XML/XSLT and Velocity are also supported). The web application programmer is responsible for writing the model code and for creating a central

Chapter 2. Java Web Application Frameworks

| 31

configuration file struts-config.xml that binds together model, view and controller. Requests from the client are sent to the controller in the form of Actions defined in the configuration file; if the controller receives such a request it calls the corresponding Action class that interacts with the application-specific model code. The model code returns an ActionForward, a string telling the controller what output page to send to the client. Information is passed between model and view in the form of special JavaBeans. A powerful custom tag library allows it to read and write the content of these beans from the presentation layer without the need for any embedded Java code. Struts is categorised as a request-based web application framework. Listing 2.24 shows the struts-config.xml file for configuring a form bean and the Action mappings. If a user enters the following URL into his web browser /submitCustomerForm.do then the com.webapp.CustomerAction Java class will be handling his requests and the CustomerForm.jsp view will be displayed. Listing 2.25 illustrates a common Struts Action which contains the business logic.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

... < struts - config > < form - beans > < form - bean name =" CustomerForm " type =" com . webapp . CustomerForm "/ > </ form - beans > < action - mappings > < action path ="/ submitCustomerForm " type =" com . webapp . CustomerAction " name =" CustomerForm " scope =" request " validate =" true " input ="/ WEB - INF / jsp / CustomerForm . jsp " > < forward name =" SUCCESS " path ="/ WEB - INF / jsp / confirm . jsp "/ > < forward name =" ERROR " path ="/ WEB - INF / jsp / error . jsp "/ > </ action > </ action - mappings > </ struts - config >

Listing 2.24: Struts 1.x struts-config.xml Example
1 2 3 4 5 6 7 8 9 10

public class CustomerAction extends Action { @Override public ActionForward execute ( ActionMapping mapping , ActionForm form , HttpServletRequest req , HttpServletResponse resp ) throws Exception { // ... ActionForward nextPage = mapping . findForward ( " SUCCESS " ) ; }

Listing 2.25: Struts 1.x Action example

2.7.1 Security Features
Apache Struts offers application managed declarative as well as programmatic access control features. Container managed security is also available and explained in Section 2.2 in more detail.

Chapter 2. Java Web Application Frameworks

| 32

Declarative Access Control

The Struts Framework has a role based access control mechanism for securing Actions declaratively. In struts-config.xml, each Action has a roles attribute, which accepts a comma-separated list of available roles as seen in Listing 2.26 on line 6.
1 2 3 4 5 6 7 8 9 10

... < struts - config > < action - mappings > < action path ="/ AdminLogin " forward ="/ admin_login . jsp " roles =" administrator "/ > ... </ action - mappings > ... </ struts - config >

Listing 2.26: Apache Struts 1.x Declarative Access Control Example This allows for a more natural approach to access control based on the named Action classes rather than URL patterns. One important thing to note is that this does not secure JSPs, HTML or other non-Action resources. If all JSPs are only accessible through the controller (which is a Struts/MVC best practice), a recommended countermeasure is to put all JSPs in the WEB-INF folder so they are not directly available to users. However, this requires that the container stipulates that files in the WEB-INF folder are not accessible to web users. Otherwise, JSPs can be protected by standard security constraints in the configuration file.
Programmatic Access Control

The Struts Framework offers several different extension points where programmatic access controls can be implemented. Specifically, the request processor has a processRoles() method that can be extended to do custom programmatic access control checks such as calling isUserInRole() for all requests to the application as seen in Listing 2.27 on line 5.
1 2 3 4 5 6 7 8

... protected boolean processRoles ( HttpServletRequest req , HttpServletResponse resp , ActionMapping mapping ) throws IOException , ServletException { if ( req . isUserInRole ( " administrator " ) ) { // ... } ...

Listing 2.27: Apache Struts Programmatic Access Control Example Additionally, Struts has tag libraries to support programmatic "content-level" access controls in JSP pages. Using the following tag, content in JSP pages can be secured according to roles:

<logic:present role="administrator"/> <a href="admin_page.jsp">Administrator Home</a> </logic:present>
The RequestProcessor class is where most of the per-request processing takes place. There are many different methods that can be overridden here, but some have very direct security relevance: • processRoles(): This method should be overridden for any Action-level access control that must be programmatic (for instance, if the application does not use JAAS).

Chapter 2. Java Web Application Frameworks

| 33

• processNoCache(): This method should be overridden to tag the response as "no-cache" to avoid security vulnerabilities associated with locally caching content. • processException(): This method should do any special exception handling that can not be done declaratively. • processValidate(): This method should do any special validation that can not be done declaratively. Obviously, more security concerns can be addressed by extending the RequestProcessor instead of using declarative access control mechanisms in Struts 1.x. However, it is important to note that any request that does not go through the Action/Controller Servlet (e.g., for a static HTML page, JSP, image/multimedia file, etc.) is not intercepted by the RequestProcessor and won’t be subject to these controls (can be fixed by writing a Servlet filter).

2.7.2 Extendibility
Struts can be extended in three ways: 1. Plugin: Plugins can be used to execute business logic at startup or shutdown of the web application. 2. RequestProcessor: A RequestProcessor can be used to execute business logic at a particular point during the request processing phase. For example, to check a principal or other security constraints. 3. ActionServlet: An ActionServlet combines the RequestProcessor and the Plugin and should only be used in cases where neither the Plugin nor the RequestProcessor is able to fulfil the desired functionality.
Plugin

According to the Struts documentation “A plugin is a configuration wrapper for a module-specific resource or service that needs to be notified about application startup and shutdown events”. This for example might be useful to clean up and close any open database connections etc. To enable such a plugin it needs to be added to the struts-config.xml file with a <plug-in> tag. Listing 2.28 illustrates a simple plugin with a init() and destroy() method22 .
1 2 3 4 5 6 7 8 9 10 11 12 13

public final class HibernatePlugIn implements PlugIn { // This method will be called at application startup time public void init ( ActionServlet actionServlet , ModuleConfig config ) // ... } // This method will be called at application shutdown time public void destroy () { // ... } ...

}

Listing 2.28: Apache Struts Plugin Example
22

http://onjava.com/pub/a/onjava/2004/11/10/ExtendingStruts.html [07/04/11]

Chapter 2. Java Web Application Frameworks

| 34

RequestProcessor

ActionServlet is the only Servlet in the Struts framework and is responsible for handling all of the
requests. Whenever it receives a request, it first tries to find a sub-application for the current request. Once a sub-application is found, it creates a RequestProcessor object for that sub-application and calls its process() method by passing it HttpServletRequest and HttpServletResponse objects. The RequestProcessor.process() is where most of the request processing takes place. The process() method is implemented using the Template Method design pattern, in which there is a separate method for performing each step of request processing and all of those methods are called in sequence from the process() method. For example, there are separate methods for finding the ActionForm class associated with the current request, and checking if the current user has one of the required roles to execute action mapping. The RequestProcessor class in the Struts distribution provides a default implementation for each of the request-processing steps. That means a developer can override only the methods that interest him and use default implementations for the rest of the methods. For example, by default Struts calls request.isUserInRole() to find out if the user has one of the roles required to execute the current ActionMapping, but if a database should be queried for this, then all one has to do is override the processRoles() method and return true or false, based whether the user has the required role or not. Listing 2.29 shows a simple example of a customised RequestProcessor with the following business logic added: • The ContactImageAction should generate images instead of regular HTML pages • Before processing every request, the RequestProcessor should check whether the user is logged in by searching for the userName attribute stored in the session. If that attribute is not found, the user will be redirected to the login page.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

public final class CustomRequestProcessor extends RequestProcessor { protected boolean processPreprocess ( HttpServletRequest request , HttpServletResponse response ) { HttpSession session = request . getSession ( false ) ; if ( request . getServletPath () . equals ( " / loginInput . do " ) || request . getServletPath () . equals ( " / login . do " ) ) { return true ; } if ( session != null && session . getAttribute ( " userName " ) != null ) { return true ; } else { try { request . getRequestDispatcher ( " / Login . jsp " ) . forward ( request , response ) ; } catch ( Exception ex ) { System . out . println ( e . getMessage () ) ; } } return false ;

Chapter 2. Java Web Application Frameworks

| 35

24 25 26 27 28 29 30 31 32 33 34

} protected void processContent ( HttpServletRequest request , HttpServletResponse response ) { if ( request . getServletPath () . equals ( " / contactimage . do " ) ) { response . setContentType ( " image / gif " ) ; return ; } super . processContent ( request , response ) ; }

}

Listing 2.29: Apache Struts RequestProcessor Example

ActionServlet

ActionServlet extensions are being configured in the deployment descriptor (web.xml) as seen in Listing 2.30. That means the custom ActionServlet is responsible for handling all of the requests to Struts. It might be wise to extend the default ActionServlet of the Struts framework. Listing 2.30 shows how an extended ActionServlet called CustomActionServlet is being properly
configured in the deployment descriptor.
1 2 3 4 5 6 7 8 9 10

<web - app > < servlet > < servlet - name > action </ servlet - name > < servlet - class > com . webapp . CustomActionServlet </ servlet - class > </ servlet > < servlet - mapping > < servlet - name > action </ servlet - name > <url - pattern >*. do </ url - pattern > </ servlet - mapping > </ web - app >

Listing 2.30: Apache Struts ActionServlet Configuration Example

2.8 Apache Struts 2.x
The following statement has been taken from the Apache Struts project site23 : Apache Struts 2 is an elegant, extensible framework for creating enterprise-ready Java web applications. The framework is designed to streamline the full development cycle, from building, to deploying, to maintaining applications over time. Apache Struts 2 was originally known as WebWork 2. After working independently for several years, the WebWork and Struts communities joined forces to create Struts2. This new version of Struts is simpler to use and closer to how Struts was always meant to be. Struts 2 offers a new feature which allows the developer to declaratively configure the usual Struts Action results through Java annotations. As this is a new feature, and still being field-tested, the Struts documentation is a bit sketchy. By using these annotations no additional XML configuration is needed. Listing 2.31 shows an example Struts Action, annotated with the following annotations: @Namespace and @Result. When the execute() method returns "SUCCESS" then the user will be redirected to the listPeople.action page.
23

http://struts.apache.org/2.2.1/index.html [10/11/10]

Chapter 2. Java Web Application Frameworks

| 36

1 2 3 4 5 6 7 8 9 10 11 12 13 14

@Namespace ( " / User " ) @Results ({ @Result ( name = " SUCCESS " , value = " / listPeople . action " , type = ServletRedirectResult . class ) , @Result ( name = " ERROR " , value = " / userError . action " , type = ServletRedirectResult . class ) }) public class EditUserAction extends ActionSupport { @Override public String execute () { // ... return " SUCCESS " ; } }

Listing 2.31: Struts 2 Annotation Example As of now, not all settings which can be set and configured through the standard Struts XML configuration files are implemented and supported through annotations. Listing 2.32 shows the struts.xml configuration file needed to set up the Action mappings and Listing 2.33 shows the actual code of the Action which contains the same configuration as seen in Listing 2.31 but without using Java annotations.
1 2 3 4 5 6 7 8 9 10 11 12 13

... < struts > < package name =" basic " extends =" struts - default " > < action name =" index " > < result >/ index . jsp </ result > </ action > < action name =" User " class =" com . webapp . EditUserAction " method =" execute " > < result name =" SUCCESS " >/ listPeople . action </ result > < result name =" ERROR " >/ userError . action </ result > </ action > </ package > </ struts >

Listing 2.32: Struts 2.x struts.xml Example
1 2 3 4 5 6 7

public final class EditUserAction extends ActionSupport { @Override public String execute () throws Exception { // ... return " SUCCESS " ; } }

Listing 2.33: Struts 2.x Action Example

2.8.1 Security Features
The Struts 2 framework offers a Interceptor called RolesInterceptor which ensures that certain Actions will only be executed if the user has the correct role. Interceptors can execute code before and after an Action is invoked. Most of the framework’s core functionality is implemented as Interceptors. Features like double-submit guards, type conversion, object population, validation, file upload, page preparation, and more, are all implemented with the help of Interceptors. Each and every Interceptor is pluggable, so the developer can decide exactly which features an Action needs to support. Interceptors can be configured on a per-Action basis. Interceptors “set the stage” for the

Chapter 2. Java Web Application Frameworks

| 37

Action classes, doing much of the “heavy lifting” before the Action executes. The RolesInterceptor has two parameters: • allowedRoles a comma-separated list of roles to allow • disallowedRoles a comma-separated list of roles to disallow Configuration of the RolesInterceptor is done through the struts.xml file by adding a <interceptor-ref> element to the Action mapping as shown in Listing 2.34 on lines 3-6.
1 2 3 4 5 6 7 8 9

... < action name =" User " class =" com . webapp . EditUserAction " method =" execute " > < interceptor - ref name =" completeStack "/ > < interceptor - ref name =" roles " > < param name =" allowedRoles " > admin , member </ param > </ interceptor - ref > < result name =" SUCCESS " >/ listPeople . action </ result > < result name =" ERROR " >/ userError . action </ result > </ action >

Listing 2.34: Struts 2.x RolesInterceptor Configuration Example

2.8.2 Extendibility
Apache Struts 2 provides a simple plugin architecture so that developers can extend the framework just by adding a JAR to the application’s classpath (in addition to whatever JARs may be required to fulfil the dependencies of the plugin itself). At load time, the framework looks for JARs containing a struts-plugin.xml file at the root of the archive. To create a plugin, all a developer needs to do is build a JAR and put the expected struts-plugin.xml at the root. Later on these plugin classes can be instantiated (for example inside a Struts Action) using the @Inject annotation provided by the framework. Since a plugin contains the struts-plugin.xml file, it has the ability to: • Define new packages with results, interceptors and/or actions • Override framework constants • Introduce new extension point implementation classes Extension points allows a plugin to override a key class in the Struts framework with an alternate implementation. For example, a plugin could provide a new class to create Action classes or map requests to Actions. For a list of all the available extension points go to http://struts.apache. org/2.0.14/docs/plugins.html.

2.9 Apache MyFaces Tomahawk
The following statement has been taken from the Apache MyFaces project site24 : Apache MyFaces is a project of the Apache Software Foundation and hosts several subprojects relating to the JavaServer technology. The Apache MyFaces project provides: a JavaServer Faces implementation, several component libraries containing UI widgets for building web applications with JSF (e.g. MyFaces Tomahawk, MyFaces Trinidad, MyFaces Tobago), extension packages to JavaServer Faces (e.g. MyFaces Orchestra, MyFaces Extensions Validator)
24

http://myfaces.apache.org/ [10/11/10]

Chapter 2. Java Web Application Frameworks

| 38

A Java Server Faces example can be found in Section 2.1.4.

2.9.1 Security Features
Java Server Faces applications respectively MyFaces applications can be secured by using container managed security features as seen in Section 2.2. Additionally, application managed security can be used by using appropriate libraries as seen in Section 2.4 or by implementing Servlet filters which enforce certain security constraints. Such Servlet filters are not provided out of the box by MyFaces and have to be implemented by the developer of the web application. Java Server Faces classes have the ability to access the user context by using the following programmatic security approach:

FacesContext context = FacesContext.getCurrentInstance(); if (context.getExternalContext().isUserInRole(’’admin’’)) { // ... }
It might be worthwhile to mention the ExtVal25 project which is an extension to the MyFaces project. MyFaces Extensions Validator (ExtVal) is a JSF centric validation framework. It offers powerful and easy to use field based validation based on annotations. It has a pluggable architecture, so it is possible to add new validation modules or support for component libraries like Trinidad or other third party frameworks. ExtVal is compatible with JSF 1.x and JSF 2.x. After setting up ExtVal, the basic usage is very simple. Listing 2.35 shows how to use ExtVal in a concrete example. The @Column annotation maps the lastName property to the LAST_NAME column in the database. It also shows some information that is derived from the table definition in the database. nullable = false means the database will not accept an empty value in this field and length = 30 means that no more than 30 characters can be stored in the corresponding database column. This information could be used for validation in the view layer (see [19, p. 321]).
1 2 3 4

@Pattern ( value = "[ A - Za - z ]*") @Length ( minimum = 30) @Column ( name = " LAST_NAME " , nullable = false , length = 30) private String lastName ;

Listing 2.35: Struts ExtVal usage example

2.9.2 Extendibility
MyFaces processes all requests in 6 phases along a defined lifecycle. With a custom lifecycle class it is possible to extend the standard lifecycle and add additional checks and business logic. The MyFaces documentation does not mention any other extension points which can be used by a developer to add additional features to the framework in a pluggable fashion. This approach is similar to the extendibility feature of Struts 1.x described in Section 2.7. In Listing 2.36 the CustomLifecycleFactory class uses the delegating factory constructor to instantiate the factory and get a reference of the standard LifecyleFactory from the MyFaces runtime. In the factory constructor an instance of the default lifecycle is passed to the MyLifecycle class and an instance of the custom lifecycle is being registered within the standard LifecycleFactory.
25

http://myfaces.apache.org/extensions/validator/ [25/11/10]

Chapter 2. Java Web Application Frameworks

| 39

1 2 3 4 5 6 7 8 9 10 11 12 13 14

public final class CustomLifecycleFactory extends LifecycleFactory { public CustomLifecycleFactory ( LifecycleFactory defaultFactory ) { delegate = defaultFactory ; // 1. get default lifecycle to be wrapped by our lifecycle Lifecycle defaultLifecycle = delegate . getLifecycle ( LifecycleFactory . DEFAULT_LIFECYCLE ) ; // 2. register intercepting lifecycle implementation . addLifecycle ( " MyLifecycle " , new MyLifecycle ( defaultLifecycle ) ) ;

}

} ...

Listing 2.36: MyFaces Custom Lifecycle Factory Example of the CustomLifecycleFactory is done through an entry in the faces-config.xml file as seen in the following code snippet: Configuration

<factory> <lifecycle-factory>com.webapp.CustomLifecycleFactory</lifecycle-factory> </factory>
Listing 2.37 shows how it is possible to enhance the standard MyFaces lifecycle with additional business logic and checks by overriding the appropriate methods but in the same time it is possible to use the standard MyFaces lifecycle by using the private variable defaultLifecycle. A FacesContext object contains all of the per-request state information related to the processing of a single JavaServer Faces request and the rendering of the corresponding response. It is passed to, and potentially modified by, each phase of the request processing lifecycle.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

public final class MyLifecycle extends Lifecycle { private Lifecycle defaultLifecycle ; public MyLifecycle ( Lifecycle defaultLifecycle ) { this . defaultLifecycle = defaultLifecycle ; } @Override public void addPhaseListener ( PhaseListener listener ) { // ... } @Override public void execute ( FacesContext context ) throws FacesException { // ... } @Override public void render ( FacesContext context ) throws FacesException { // ... } ...

}

Listing 2.37: MyFaces Custom Lifecycle Example

Chapter 2. Java Web Application Frameworks

| 40

2.10 Wicket
Apache Wicket, commonly referred to as Wicket, is a lightweight component-based web application framework for the Java programming language conceptually similar to JavaServer Faces and Tapestry. Wicket is closely patterned after stateful GUI frameworks such as Swing (event driven). Wicket applications are trees of components, which use listener delegates to react to HTTP requests against links and forms in the same way that Swing components react to mouse and keystroke events. Wicket is categorised as a component-based framework and Wicket bridges the impedance mismatch between the stateless HTTP and stateful server-side programming in Java. To illustrate this new approach of an event driven web application the following example creates a simple Wicket web application with one page. This page contains a link to increment a counter and uses a label to display the number of clicks. Listing 2.38 shows the logic for the link counter which is achieved by overriding the onClick() method of the link object. Listing 2.39 shows the HTML layout which is being used to display the counter to the user. Note that this is pure HTML and no JSP etc. is needed, except that the wicket:id has to be provided for displaying dynamic content (see [9]).
1 2 3 4 5 6 7 8 9 10 11 12 13 14

public final class HomePage extends WebPage { public int counter = 0; public HomePage ( final PageParameters parameters ) { add ( new Link ( " link " ) { @Override public void onClick () { counter ++; } }) ; } add ( new Label ( " label " , new PropertyModel ( this , " counter " ) ) ) ;

}

Listing 2.38: Simple Wicket Page with link Counter
1 2 3 4 5 6

< html > < body > <a href ="#" wicket : id =" link " > This link </ a > has been clicked < span wicket : id =" label " >123 </ span > times . </ body > </ html >

Listing 2.39: Wicket Page Layout for link Counter

2.10.1 Security Features
Wicket has been developed with security in mind, furthermore Wicket-Security26 or the WicketAuth-Roles27 project can be used to add access controls constraints to a Wicket web application. Wicket URLs do not expose sensitive information and all component paths are session relative. Explicit steps must be taken to share information between sessions. This means an attacker would have to hijack the session and then guess the right page identifiers and version numbers, which would be relative to the session and the relevant component paths. Additionally, URL encryption can be enabled with four lines of code to support highly secure web sites. Of course container managed
26 27

http://wicketstuff.org/confluence/display/STUFFWIKI/Wicket-Security [22/11/10] http://wicket.apache.org/learn/projects/authroles.html [24/11/10]

Chapter 2. Java Web Application Frameworks

| 41

security can also be applied to Wicket web applications as seen in Section 2.2. Wicket-Security is an attempt to create an out of the box reusable authenticating and authorisation framework for Wicket28 . It contains several projects which can be used stand alone or in conjunction with each other. Wicket-Security uses the available security mechanisms from Wicket to build upon and expand the authentication and authorisation mechanisms. In order to maximize flexibility and customisation a new API, called Wicket Abstract Security Platform (WASP), was built on top of Wicket. WASP builds on the IAuthorisationStrategies provided by Wicket and allows individual components or models to do their own authentication and/or authorisation checks. Like Wicket it uses actions to grant permissions, however where Wicket uses string based actions (render and enable) WASP uses full blown Java objects. Besides an API to implement individual authentication and authorisation routines, Wicket-Security also contains a default implementation, called Swarm. Wicket is only aware of the render and enable actions. They are used for making components invisible (render), read only and clickable (enable). • access is used to check if a component is allowed to be created. • render is used to check if a component is allowed to be rendered. • enable is used to check if a component is allowed to be enabled, for links this means: can the user click on them, for components that take user input (like a textfield) this means: can the user write to them. • inherit is for container components and specifies that all the actions for this container are inherited by its children, meaning a developer does not need to specify permissions for each component individually. Swarm adds the access action for component instantiation and the inherit action for permission inheritance. Actions can imply other actions by default all actions imply the access action and the enable action implies the render action. This means if a developer specifies the enable action what one really get is access, render and enable action. This also means that if permissions without any explicit actions is granted, one will automatically get the access action. To make things more clear a sample standard.hive configuration file is shown in Listing 2.40.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

grant { // actually the login and HomePage are not SecureWebPage classes . permission $ { ComponentPermission } " com . mycompany . webapp . LoginPage " , " inherit , render , enable "; permission $ { ComponentPermission } " com . mycompany . webapp . HomePage " , " inherit , render , enable "; }; grant principal com . mycompany . webapp . SpringSecurePrincipal " ROLE_USER " { permission $ { ComponentPermission } " com . mycompany . webapp . FirstSecurePage " , " inherit , render , enable "; }; grant principal com . mycompany . webapp . SpringSecurePrincipal " ROLE_ADMIN " { permission $ { ComponentPermission } " com . mycompany . webapp . SecondSecurePage " , " inherit , render , enable ";
28

http://wicket.apache.org/ [22/11/10]

Chapter 2. Java Web Application Frameworks

| 42

17

18 19

};

permission $ { ComponentPermission } " com . mycompany . webapp . FirstSecurePage : ajaxLink " , " render , enable ";

Listing 2.40: Wicket-Sercurity Configuration Example Secure pages are pages requiring an authenticated user with certain privileges. In Swarm Secure pages are achieved by having the secure pages either extend SecureWebPage or implement ISecurePage. Using the configuration in Listing 2.40 would grant everyone the right to see (render) the HomePage, if there are secure components on the home page one would see them too (inherit). In addition links to the HomePage are granted the right to be clicked (enable). Another approach is to use Wicket-Auth-Roles for securing a Wicket web application. Wicket-AuthRoles is included in the standard Wicket release. In order to use Auth-Roles in a Wicket web application the initial base class needs to extend AuthenticatedWebApplication as seen in the following code snippet:

public class ExampleWicketApplication extends AuthenticatedWebApplication { // ... }
From now on, web pages can be annotated with the @AuthorizeInstantiation annotation as seen in the next code snippet:

// only users with role ROLE_ADMIN are allowed // to create instances of this page @AuthorizeInstantiation("ROLE_ADMIN") public class AdminAnnotationsBookmarkablePage extends WebPage { // ... }
When someone who does not have the role ROLE_ADMIN, Wicket will not allow the page to be fully constructed and throws an authorisation exception during the construction of the page. This will result in an access denied page for the user. Wicket-Auth-Roles might not be as sophisticated as Wicket-Security but it is much more intuitive to use.

2.10.2 Extendibility
The source code of the Wicket framework is freely available and thus extensible. Wicket is licensed under the Apache License, Version 2.0 which can be downloaded from http://www.apache.org/ licenses/LICENSE-2.0. Sadly the Wicket frameworks lacks documentation of its internal architecture. This circumstance makes it slightly harder to implement new features directly into the core of the framework before knowing its internal structure and workings but nonetheless possible.

2.11 Summary
This Chapter investigated the technologies involved in creating Java web applications, available security frameworks for securing Java web or EE applications and popular Java web application frameworks including their security features and extendibility. No matter which web application

Chapter 2. Java Web Application Frameworks

| 43

framework is being used to write a web application, container managed security as described in Section 2.2 is available and can be used to add security constraints to the URLs provided by the web application. These constraints can either be configured through the deployment descriptor of the web application called web.xml or through annotations if simple Java Servlets (Version 3.0) are being used. Usernames, passwords and roles have to be made available to the web container which enforces these constraints. Additionally to container managed security, all web application frameworks except for the Apache MyFaces project offer framework specific access controls. These can be either configured through XML configuration files or by adding framework specific Java annotations to the source code. Another possibility is to ignore framework specific access controls and use a security library as seen in Section 2.4 for enforcing access controls. Interesting to note is that basically all of the described web application frameworks do not offer any sophisticated input validation routines. Common validation such as checking for numeric values only, valid phone numbers etc. is mostly available, but none enforces checks against Code Injection Attacks although this can be solved by using security libraries such as ESAPI. All of the available security frameworks are implemented using Servlet filters which also have to be configured through the deployment descriptor of the web application on a per-URL basis. This makes them loosely coupled and the web application can be written with any technology or framework the developer desires. Interesting is the ESAPI project which offers a complete web application firewall in form of a Servlet filter. Also very astonishing is the ability to use a scripting language (The Spring Expression Language) inside of annotated security constraints offered by the Spring Security project. This allows the creation of sophisticated constraints which incorporate method parameters from the annotated classes. Wicket is the first web application framework which does not use a typical MVC paradigm, instead it behaves more like a Java stand alone application where actions respectively business logic is being triggered through events from the user. Especially nice is that views in Wicket consist of pure HTML and Cascading Style Sheets (CSS) without polluting it with programming semantics like JSP does. Table 2.4 compares the security features of the various Java web application frameworks discussed in this Chapter. Most frameworks offer the same functionality as their counterparts with the exception of MyFaces. Basically all support declarative- and programmatic security features. Annotation based security features are only available in Spring and Wicket. Noteworthy is the fact that it is possible to use Spring Security in conjunction with other frameworks, this means it is possible to use Struts together with Spring Security (see [30, p. 187]).

Chapter 2. Java Web Application Frameworks

| 44

Spring Framework X X X RBAC, Input Validation X X X RBAC, Input Validation

Apache Struts 1.x

Apache Struts 2.x

MyFaces X X RBAC, Input Validation

Wicket X X X X RBAC, URL encryption, Input Validation

(+ Spring Security)

Container Managed Security Declarative Security Security Annotations Programmatic Security Security Features

X X X X RBAC, Input Validation

Chapter 2. Java Web Application Frameworks

Table 2.4: Java Web Application Frameworks Security Features Comparison

| 45

3 Related Work
The following Chapter presents the current state of the research literature related to adding declarative information to web application source code to improve security. The papers summarised have been found through the Google Scholar search engine1 . The available research on enhancing web application security declaratively can be roughly divided into three categories: added security through declarative security policies, adding an aspect oriented approach to web application security and static code analysis with the aid of metadata.

3.1 Declarative Security Policies and Constraints
Some of the declarative security features Java Standard Edition/EE offers have already been discussed in Section 2.2 and 2.3. These include container managed security and standard Java EE features. Additional declarative security features are being provided by Java security frameworks such as Spring Security described in Section 2.4 and 2.4.2. David Scott et al. [31] introduced in their paper Abstracting Application-Level Web Security a security policy description language (SPDL) which is used to specify a set of validation constraints and transformation rules for web applications. The proposed system consists of the SDPL which is used to program an application level firewall (in the paper referred to as a security gateway). The security policies are written in SDPL and compiled for execution on the security gateway. The security gateway dynamically analyses and transforms HTTP request/responses to enforce the specified policies. The motivation behind this proposed system is the lack of language-level support in popular untyped scripting languages to prevent application-level security vulnerabilities. The programmer has to manually verify that all user input is appropriately HTML-encoded. An example of such a security policy in SPDL is shown in Listing 3.1. A policy element contains a series URL and cookie elements. For each URL element a number of parameters are declared. The attributes of a parameter element with name = p place constraints on data passed via p: • The maxlength and minlength attributes specify the maximum and minimum length of data passed via p. • Setting required to "Y" specifies that p must always contain a (non-zero length) value. • Setting MAC to "Y" specifies that the value of p must be accompanied by a Message Authentication Code (MAC) generated by the server. This prevents the user from changing the value of the parameter to arbitrary values. • The type attribute specifies the data-type of p (either int, float, bool or string). The method attribute determines whether the specified constraints apply to p passed as a GETparameter or a POST-parameter. Setting method to GETandPOST means that the constraints within the parameter element are applicable to both GET and POST parameters with the name = p.
http://scholar.google.com [14/04/11]

1

Chapter 3. Related Work

| 46

1 2 3 4 5 6 7 8 9 10 11 12

< policy > < URL prefix =" http :// example " > < parameter name =" p1 " maxlength ="4" type =" int " required =" Y " MAC =" Y "/ > < parameter name =" p2 " maxlength ="3" type =" string " method =" POST " > < validation > format ( this , "\ d \ d /\ d \ d ") and Int . fromString ( mid (s ,1 ,2) ) <= 12 and Int . fromString ( mid (s ,1 ,2) ) >= 0 </ validation > </ parameter > </ URL > </ policy >

Listing 3.1: Abstracting Application-Level Web Security with a Policy Description Language Example The example in Listing 3.1 specifies constraints on parameters passed to URLs with the prefix "http://example". The first parameter element defines constraints to be applied to a parameter named p1 (either GET or POST); the second parameter element defines constraints to be applied to a POST parameter named p2. In some circumstances a greater degree of control is required; this is provided by the validation element. The validation element allows complex constraints to be encoded in a general purpose validation language. A well-formed validation expression has type Boolean. If the validation expression of parameter p, evaluates to true then this signifies that p contains valid data. Within validation expressions, the value of the field specified in the enclosing parameter element is referred to as this. Functions are provided to cast between different types. For example, String.fromInt(i) returns the string representation of integer i. The main advantage of this approach is the separation of the security policy from the programmer and having a centralised security policy. The proposed approach is very similar to web application firewalls such as ModSecurity2 (open source web application firewall for the Apache web server), the difference is that the system of David Scott et al. pursues a “programmatic approach” of specifying security policies because of the SPDL compiler. Myers [27] describes in his paper a promising technique for protecting privacy and integrity of sensitive data by statically checking information flow within programs that manipulate the data. He argues that programming language extensions to allow this static checking are too restrictive for practical use and have not been implemented. He describes a new language called JFlow, which is an extension to the Java language that adds statically-checked information flow annotations. In JFlow, every Java value has a labelled type that consists of two parts: an ordinary Java type such as int and a label that describes the ways that the value can propagate. For example the expression int{customer:} val; represents an integer that principal customer owns and only customer can read (the owner of a policy is always implicitly a reader). A component expression of the form owner: reader1 , reader2 , . . . denotes a policy. A label expression is a series of component expressions, separated by semicolons, such as {o 1 : r1 , r2 ;o 2 : r2 , r3 }. The default label of an instance variable is the public label {}. JFlow offers a few optional annotations to manage information flow and authority delegation: • authority(p 1 , ..., p n ) - This clause lists principals that the method is authorised to act for. • caller(p 1 , ..., p n ) - Calling code may also dynamically grant authority to a method that has a caller constraint. A method with a caller clause may be called only if the calling code possesses the requisite static authority. • actsFor(p 1 , p 2 ) - An actsFor constraint may be used to prevent the method from being called unless the specified acts-for relationship (p 1 acts for p 2 ) holds at the call site.
2

http://www.modsecurity.org/ [1/12/10]

Chapter 3. Related Work

| 47

Listing 3.2 contains a JFlow implementation of a simple password file, in which the passwords are protected by information flow controls. Only the method for checking passwords is shown. This method, check, accepts a password, a username and returns a Boolean indicating whether the string is the right password for that user. The if statement is conditional on the elements of passwords and on the variables user and password, whose labels are implicit parameters. Therefore, the body of the if statement has policy = {user; password; root:} and the variable match also must have this label in order to allow the assignment match = true. This label prevents match from being returned directly as a result, since the label of the return value is the default label, {user; password}. Finally, the method declassifies match to this desired label, using its compiled-in authority to act for root. JFlow supports many language features that have not been previously integrated with static flow checking, including mutable objects (which subsume function values), subclassing, dynamic type tests, dynamic access control and exceptions. JFlow is not completely a superset of Java. Certain features have been omitted to make information flow control tractable. Also, JFlow does not eliminate all possible information leaks. Certain covert channels (particularly various kinds of timing channels) are difficult to eliminate.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

public class passwordFile authority ( root ) { private String [] names ; private String { root :}[] passwords ; public boolean check ( String user , String password ) { where authority ( root ) { boolean match = false ; try { for ( int i = 0; i < names . length ; i ++) { if ( names [ i ]. equals ( user ) && passwords [ i ]. equals ( password ) ) { match = break ; } } } catch ( Exception e ) { // ... } } }

}

Listing 3.2: JFlow Password File Example Another project with the goal of static information flow analysis is called Jif3 . Jif is a security-typed programming language that extends Java with support for information flow control and access control, enforced at both compile time and run time. Jif is written in Java and is built using the Polyglot extensible Java compiler framework. Static information flow control can protect the confidentiality and integrity of information manipulated by computing systems. The compiler tracks the correspondence between information the policies that restrict its use, enforcing security properties end-to-end within the system. After checking information flow within Jif programs, the Jif compiler translates them to Java programs and uses an ordinary Java compiler to produce secure executable programs. Jif extends Java by adding labels that express restrictions on how information may be used. For example, the following variable declaration declares not only that the variable x is an int, but also that the information in x is governed by a security policy:

3

http://www.cs.cornell.edu/jif/ [2/12/10]

Chapter 3. Related Work

| 48

int {Alice→Bob} x;
In this case, the security policy says that the information in x is controlled by the principal Alice, and that Alice permits this information to be seen by the principal Bob. The policy {Alice←Bob} means that information is owned by Alice and that Alice permits it to be affected by Bob. Based on label annotations like these, the Jif compiler analyses information flows within programs, to determines whether they enforce the confidentiality and integrity of information. Considering the following variable declaration and assignments:

int {Alice→Bob, Chuck} y; x = y; // OK: policy on x is stronger y = x; // BAD: policy on y is not as strong as x
The variable y is declared to be an int and Alice permits the information in y to be seen by both Bob and Chuck. The first assignment is secure because it does not make the information in y visible to any additional principals. The second assignment is insecure because now Chuck can see the information that was in x and the policy on x forbids this (unless Alice or Bob happen to trust Chuck). The Jif compiler makes these determinations at compile time, as part of type checking. If a Jif program type-checks, the compiler translates it into Java code that can be compiled with a standard Java compiler. The program can then be executed with a standard Java virtual machine. Although enforcement is mostly done at compile time, Jif does also allow some enforcement to take place at run time. Therefore, Jif programs in general require the Jif run-time library. A higher level of abstraction for declaring access control policies, regardless of the underlying programming language, is achieved by using the Extensible Access Control Markup Language (XACML) from the OASIS standards organisation4 . There are quite a few Java libraries/frameworks for parsing and enforcing XACML policy files such as: Sun’s SunXACML5 project and PicketBox XACML (formerly known as JBossXACML)6 . XACML is a declarative access control policy language implemented in XML and a processing model, describing how to interpret the policies. XACML is based on the Attribute Based Access Control (ABAC) model that is hailed as the next generation of access control models7 . Instead of only using a role as the determining factor whether to grant access or not, many attributes can be used. Roles can be used in ABAC as well - since ABAC can use multiple attributes to make access control decisions, so ABAC can emulate RBAC perfectly while adding many additional advantages. This means that it is possible to add context to the access control decisions and adds for a finer granularity, tighter controls and more flexibility for the business. With XACML, policies can be woven around a combination of subjects (users/user-agents etc), resources (on which the access control is desired) and environment (IP address, Date, Time etc). Now a developer is able to declaratively (via XML or construct policies) say things like “Allow this portion of the web site visible to 18 year olds when the time is between 9am and 5pm”, “You should update your own payroll information and can do it when you are employed and on Fridays only” etc. Enterprises have been doing this via ACLs and other proprietary mechanisms. Now they can use a standard way. A basic XACML architecture includes a Policy Enforcement Point (PEP) which intercepts any requests in a normal program flow, then asks a Policy Decision Point (PDP) to make an access decision based on the policies associated with the PDP The PDP evaluates the XACML request created by the PEP .
http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=xacml [2/12/10] http://sunxacml.sourceforge.net/ [3/12/10] 6 http://community.jboss.org/wiki/PicketBoxXACMLJBossXACML [3/12/10] 7 http://blogs.kuppingercole.com/gaehtgens/2009/07/31/finally-an-open-xacml-api/ [2/12/10]
4 5

Chapter 3. Related Work

| 49

and runs through the policies to make one of the four access decisions, namely: • PERMIT - approve request • DENY - access denied • INDETERMINATE - error at the PDP • NOTAPPLICABLE - some attribute missing in the request or no policy match The main job of a PEP is to intercept requests to any resources and then create an XACML request based on attributes about the Subject (actor accessing the resource), Action, Attributes about the resource and environment (Time, Date, IP address etc.) and pass it to a PDP for access decision. Based on the PDP decision, the PEP can either allow access to the resource or not.

3.2 Aspect-Oriented Security
Aspect-oriented programming (AOP) is programming paradigm which entails breaking down program logic into distinct parts (so called concerns, cohesive areas of functionality). AOP isolates secondary or supporting functions from the main program’s business logic. It aims to increase modularity by allowing the separation of cross-cutting concerns, forming a basis for aspect-oriented software development. Logging as well as security constraints exemplifies a cross-cutting concern because a logging strategy necessarily affects every single logged part of the system. Logging thereby cross-cuts all logged classes and methods. The following example code taken from [20] shall illustrate the possibilities of AOP programming. The example code has been tested using the AspectJ8 project which is an open source AOP implementation for the Java programming language. The MessageCommunicator class in Listing 3.3 has two methods: one to deliver a general message and the other to deliver a message to a specific person. Listing 3.4 shows a simple class to exercise the functionality of the MessageCommunicator class.
1 2 3 4 5 6 7 8 9

public class MessageCommunicator { public void deliver ( String message ) { System . out . println ( message ) ; } public void deliver ( String person , String message ) { System . out . println ( person + " , " + message ) ; }

}

Listing 3.3: Class encapsulating the Message-Delivery Functionality
1 2 3 4 5 6 7

public class Main { public static void main ( String [] args ) { MessageCommunicator messageCommunicator = new MessageCommunicator () ; messageCommunicator . deliver ( " Wanna learn AspectJ ? " ) ; messageCommunicator . deliver ( " Harry " , " having fun ? " ) ; } }

Listing 3.4: Class to exercise the Message-Delivery Functionality
8

http://www.eclipse.org/aspectj/[29/11/10]

Chapter 3. Related Work

| 50

Consider authentication functionality: before delivering a message, the class should check whether the user has been authenticated. Without using AOP, the code would probably look like the one in Listing 3.5.
1 2 3 4 5 6 7 8 9 10 11 12 13

public class MessageCommunicator { private Authenticator authenticator = new Authenticator () ; public void deliver ( String message ) { authenticator . authenticate () ; System . out . println ( message ) ; } public void deliver ( String person , String message ) { authenticator . authenticate () ; System . out . println ( person + " , " + message ) ; }

}

Listing 3.5: Class encapsulating the Message-Delivery Functionality (with Authentication) In order to verify if the user is properly authenticated, every method needs to call authenticate() which leads to code tangling. Similar code would have to be present in all classes that require the authentication functionality - leading to code scattering. By using an AOP approach the code from Listing 3.3 can be enhanced with the authentication functionality without changing a single line of code by adding an aspect to the system. The Authenticator class asks for credentials (username and password) when the authenticate() method is called for the first time in a thread. Upon successful authentication, it stores the user in a thread local so it does not ask for credentials in the same thread again. Upon failure, it throws a runtime exception. Not shown for brevity’s sake.
1 2 3 4 5 6 7 8 9 10

public aspect SecurityAspect { private Authenticator authenticator = new Authenticator () ; pointcut secureAccess () : execution (* MessageCommunicator . deliver (..) ) ; before () : secureAccess () { System . out . println ( " Checking and authenticating user " ) ; authenticator . authenticate () ; }

}

Listing 3.6: Authentication Aspect Compiling the classes along with the aspect is done by running the ajc command. The following snippet illustrates the usage and the output of the freshly compiled program:

# ajc -source 5 MessageCommunicator.java Main.java SecurityAspect.aj # java Main Checking and authenticating user Wanna learn AspectJ? Checking and authenticating user Harry, having fun?
The SecurityAspect.aj file declares the SecurityAspect aspect. An aspect is a unit of modularisation in AOP much like a class is a unit of modularisation in OOP. The declaration of an aspect ,

Chapter 3. Related Work

| 51

is similar to a class declaration. A pointcut selects interesting points of execution in a system, called join points. The aspect in Listing 3.6 defines a pointcut secureAccess() that selects execution of all the methods named deliver() in the MessageCommunicator class. The * indicates that the pointcut matches any return type and the .. inside parentheses after deliver specifies that it matches regardless of the number of arguments or their types. In this example, the pointcut selects execution of both overloaded versions of deliver() in the MessageCommunicator class. An advice defines the code to execute upon reaching join points selected by the associated pointcut. Listing 3.6 defines a piece of advice to execute before reaching the join points selected by the secureAccess() pointcut. The before() part indicates that the advice should run prior to the execution of the advised join point - in this case, prior to executing any MessageCommunicator. deliver() method. In the advice, the user gets authenticated. With the aspect now present in the system, each time MessageCommunicator.deliver() is executed, the advice code performs the authentication logic before the method. AspectJ offers the following pointcuts: • execution(void Point.setX(int)) - when a particular method body executes • call(void Point.setX(int)) - when a method is called • handler(ArrayOutOfBoundsException) - when an exception handler executes • this(SomeType) - when the object currently executing (i.e. this) is of type SomeType • target(SomeType) - when the target object is of type SomeType • within(MyClass) - when the executing code belongs to class MyClass • cflow(call(void Test.main())) - when the join point is in the control flow of a call to a Test’s no-argument main method Pointcuts compose through the operations or ("||") and ("&&") and not ("!"). It is possible to use wildcards: 1. execution(* *(..)) 2. call(* set(..)) (1) means the execution of any method regardless of return or parameter types and (2) the call to any method named set regardless of return or parameter types - in case of overloading there may be more than one such set method; this pointcut picks out calls to all of them. Additionally elements can be selected based on their types. For example: 1. execution(int *()) 2. call(* setY(long)) 3. call(* Point.setY(int)) 4. call(*.new(int, int))

Chapter 3. Related Work

| 52

(1) means the execution of any method with no parameters that returns an int, (2) the call to any setY method that takes a long as an argument, regardless of return type or declaring type, (3) the call to any of Point’s setY methods that take an int as an argument, regardless of return type and (4) the call to any classes constructor, so long as it takes exactly two ints as arguments. Even composition is possible. For example: 1. target(Point) && call(int *()) 2. call(* *(..))

&& (within(Line) || within(Point))

3. within(*) && execution(*.new(int)) 4. !this(Point) && call(int *(..)) (1) means any call to an int method with no arguments on an instance of Point, regardless of its name, (2) any call to any method where the call is made from the code in Point’s or Line’s type declaration, (3) the execution of any constructor taking exactly one int argument, regardless of where the call is made from, and (4) any method call to an int method when the executing object is any type except Point. Last but not least, methods and constructors can be selected based on their modifiers and on negations of modifiers. For example: 1. call(public * *(..)) 2. execution(!static * *(..)) 3. execution(public !static * *(..)) which means (1) any call to a public method, (2) any execution of a non-static method and (3) any execution of a public, non-static method. Bart De Win et al. [36] state that within the area of AOP ongoing research addresses a declarative description of security properties for application software. Such security specific language would be especially valuable to define the binding between application and security mechanisms. The core challenge is to create the right security abstractions. This is obviously not evident. The security abstractions will remain an evolving challenge for the foreseeable future. De Win et al. [36] have investigated how well AOP can deal with the separation of security concerns from an application. In order to construct a more generic solution, they have suggested to abstract relevant pointcuts out of the aspect implementation. Dima Alhadidi et al. [1] investigated how well AspectJ can be used for security hardening of Java programs. AspectJ offers three kinds of advices to describe where aspects are to be applied: Before, after or around. All of them are important from a security hardening point of view: • A before-advice can be used to inject some security code as authentication before a critical operation. • An after-advice is meant to inject some security code after some locations in the program. For instance, one can add some integrity checking code.

Chapter 3. Related Work

| 53

• An around-advice can be used to change a non-secure method by a more secure one at particular points in the program. They came to the conclusion that AspectJ is very well suited for security purposes but the AspectJ implementation and all other AOP frameworks lack some additional features respectively pointcuts which would increase the places where security checks could be done, for example a loop- and dataflow pointcut etc. An example of a multilevel security system which prevents unauthorised personnel from accessing information at higher classification than their authorisation and preventing personnel from declassifying information is proposed in the paper AspectJ for Multilevel Security of Roshan Ramachandran et al. [33]. In a case study involving a simple multi threaded payroll system where administrators can update and track specific information about employees, Ramachandran showed that it is possible to enforce complex security constraints through a security monitor implemented as an aspect. The information in the payroll system includes sensitive details like personal details, salary details and so forth. The working scenario is that different administrators of varying clearances access the payroll system to access different classes of information. For example, an administrator with a low clearance cannot see the salary details of a senior manager with a higher security classification. Also, an administrator with a higher clearance cannot leak the salary details of a higher ranking official such that a user with low clearance can access its details. Thus, to preserve the confidentiality of the information, an access control policy was implemented in the application. Amine Boukhtouta et al. go a step further in their paper called A Practical Framework for The Dataflow Pointcut in AspectJ [6]. The paper presents the design and the implementation of the dataflow pointcut in AspectJ compiler ajc 1.5.0 (the current version is 1.6.10). From the abstract: Some security concerns are sensitive to flow of information in a program execution. The dataflow pointcut has been proposed by Masuhara and Kawauchi in order to easily implement such security concerns in aspect-oriented programming languages. The pointcut identifies join points based on the origins of values. The dataflow pointcut can detect and fix a lot of vulnerabilities that result from not validating input effectively, e.g., web application vulnerabilities, process injection, log forging and path injection. AOP has not been initially engineered with security in mind which resulted in many shortcomings in this technology. There are some contributions that suggest new pointcuts for security purposes such as the one of Masuhara and Kawauchi [23]. They have presented the design of the dataflow pointcut with a web application example, and its prototype implementation. It has been implemented as an extension to Aspect SandBox (ASB) [24], which is a modelling framework for AOP mechanisms based on the pointcut-advice model. Listing 3.7 shows a tracing aspect written in AspectJ where the pointcut traced picks out any call to any method. AspectJ has a comprehensive and expressive pointcut specification language that allows to specify particular points in the control flow of the program where advices are to be applied.
1 2 3 4 5 6 7 8

public aspect ExtTracingAspect { private int callDepth ; pointcut traced () : ! within ( ExtTracingAspect ) ; before () : traced () { print ( thisJoinPoint ); callDepth ++; }

Chapter 3. Related Work

| 54

9 10 11 12 13 14 15 16 17 18 19 20 21

after () : traced () { callDepth - -; } private void print ( Object message ) { for ( int i = 0; i < callDepth ; i ++) { System . out . print ( " " ) ; } } System . out . println ( message ) ;

}

Listing 3.7: AspectJ Tracing Aspect Adding the trace aspect to a simple Struts 1.x web application would generate an output such as seen in Listing 3.8
1 2 3 4 5

6 7 8 9

staticinitialization ( com . webapp . HelloWorld . < clinit >) preinitialization ( com . webapp . HelloWorld () ) initialization ( com . webapp . HelloWorld () ) execution ( com . webapp . HelloWorld () ) execution ( ActionForward com . webapp . HelloWorld . execute ( ActionMapping , ActionForm , HttpServletRequest , HttpServletResponse ) ) get ( PrintStream java . lang . System . out ) call ( void java . io . PrintStream . println ( String ) ) call ( ActionForward org . apache . struts . action . ActionMapping . findForward ( String ) )

Listing 3.8: AspectJ Tracing output for a simple Struts 1.x Application Masuhara and Kawauchi [23] have defined a dataflow pointcut for security purposes but this pointcut has not been implemented yet. The pointcut identifies join points based on the origins of values. Cross-site scripting (XSS) problem in web applications is an example presented by them to clarify the need for such a pointcut. Local variables set and get pointcuts [2] have been introduced to protect the privacy and integrity of sensitive data. They allow to track the values of local variables inside a method. Many security hardening practices require the injection of code around a set of join points or possible execution paths. Examples of such cases would be the injection of security library initialisation/deinitialisation, privilege change and logging. A different pointcut has been proposed by Boukhtouta et al. [5] in their paper Capturing Returned Computable Values in AspectJ for Security Hardening. They have fully designed and implemented returns and returned pointcuts as extensions to ajc-1.5.0 compiler of AspectJ. These pair of pointcuts can capture the returned computable values of methods. Such pointcuts can be appropriate primitives for integrating data concerns in Java code. The returned pointcut is classified as a kindred pointcut whereas the returns pointcut is classified as a context matching pointcut. These newly implemented pointcuts are being used to define a data integrity service for security hardening of distributed programs. Data integrity refers to validity of data, which can be compromised through a malicious or an accidental altering. These pointcuts returns and returned together with the call pointcut compromise the main building block of the data integrity service. To clarify the idea, consider the following model. We consider a standard server which implements a set of remote methods that can be invoked at any time by clients. Once a client invokes a method implemented in the server, the server runs the method. Before returning the value to the client, it creates a hash for it, which is recorded in a hash repository implemented in the data integrity service. The

Chapter 3. Related Work

| 55

server then sends the value to the client. After receiving the value, the client creates a hash for the received value and sends the hash to the data integrity service, which acknowledges the hash. If the acknowledgement is positive, the client carries on the execution of its program. Otherwise it throws an exception. Martin et al. [21] uses static source code analysis to detect common web application vulnerabilities such as SQL Injection attacks and application errors by proposing a program query language (PQL). PQL allows programmers to write queries which express whether a program conforms to certain design rules. An important class of design rules deal with sequences of events associated with a set of related objects. A query looks like a code excerpt corresponding to the shortest amount of code that would violate a design rule. The programmer may also specify actions to perform when a match is found, such as recording relevant information or even correcting an erroneous execution on the fly. The system automatically generates from the query a pair of complementary checkers: a static checker that finds all potential matches in an application and a dynamic checker that traps all matches precisely as they occur and can initiate user specified logging or recovery actions upon a match. A simple PQL query for detecting SQL Injection vulnerabilities is shown in Listing 3.9.
1 2 3 4 5 6 7 8

query simpleSQLInjection () uses object HttpServletRequest r ; object Connection c ; object String p ; matches { p = r . getParameter ( _ ) ; } replaces c . execute ( p ) with Util . CheckedSQL (c , p ) ;

Listing 3.9: PQL Query for finding SQL Injection Vulnerabilities PQL attaches user specified actions to subquery matches; this capability puts PQL in the class of aspect-oriented programming languages. The PQL system may be considered as an aspect oriented system that defines its aspects with respect to the dynamic history of sets of objects. An extension of AspectJ to include dataflow pointcuts [23] has been proposed to represent a statement that receives a value from a specific source; PQL can represent these with a two-statement query and permits much more complex concepts of data flow. Walker and Veggers [35] introduce the concept of declarative event patterns, in which regular expressions of traditional pointcuts are used to specify when advice should run. Allan et al. [3] extend this further by permitting PQL-like free variables in the patterns. The primary focus in the AspectJ extensions is in permitting a developer to specify application development concerns very finely. As a result, they devote a great deal of work to ensuring properties such as guarantees that memory allocated by the matching machinery will eventually be available for collection. PQL, with its genesis focusing on detecting application errors, pays less attention to this. PQL differs from these systems in that its matching machinery can recognise nonregular languages and in exploiting advanced pointer analysis to prove points irrelevant to eventual matches. Kung et al. [7] designed access control aspects for web applications. In particular, they choose the MVC-based Struts framework as the architectural style of the target application. Two aspect suites in AspectJ have been developed to enforce fine-grained access controls in a modular and non-invasive manner. While both suites exhibit good software engineering properties, the first suite cannot guarantee the required conditional execution and the second suite achieves it at the cost of code duplication. They defined three abstract aspects that capture the common code structure for enforcing fine-grained access controls, namely Authentication, Precheck and Postfilter. The authentication aspect is responsible for verifying that the requesting user has passed the identity check or must be redirected to a login page. The Precheck aspects ensure that designated user

Chapter 3. Related Work

| 56

actions get executed only when a given constraint is satisfied. The Postfilter aspect will filter out unauthorised data records from a user query result, and, if necessary, mask out sensitive data fields. Listing 3.10 shows an aspect declaration which would be called before every execution of the Struts 1.x execute(...) method. This would allow a developer to do additional security checks based on the HTTP request. Every user request is dispatched to an Action class by the controller according to the Action mapping defined in the configuration file, struts-config.xml. These actions are responsible for serving user requests or passing them to the designated business tier components and for returning the correct view element that the controller should forward to after finishing the user request. This view forwarding is also based on the mapping information specified in the configuration file. In other words, the action classes play the role of gateway between the presentation tier and the business and the data tiers. Moreover, all action classes must inherit from the class Action and implement an execute(...) method. Thus makes it the right join point to weave in advice code for its arguments and exposes the right amount of information to do security checks.
1 2

public aspect RequestAspect { public pointcut getHTTPRequest ( ActionMapping mapping , ActionForm form , HttpServletRequest request , HttpServletResponse response ) : execution (* *. execute ( ActionMapping , ActionForm , HttpServletRequest , HttpServletResponse ) ) && args ( mapping , form , request , response ) ; before ( ActionMapping mapping , ActionForm form , HttpServletRequest request , HttpServletResponse response ) : getHTTPRequest ( mapping , form , request , response ) { System . out . println ( " Request received : " + request ) ; }

3 4

5 6 7

}

Listing 3.10: Aspect for catching all Struts 1.x execute Methods A similiar research paper to Kung’s et al. [7] is the paper from Andrew Prunicki et al. [28]. They described and designed Aclamate: An AOSD Security Framework for Access Control. Aclamate is a framework providing authorisation and access control in a non-invasive fashion. Aclamate is well suited to provide security for new or pre-existing applications. Upon successful authentication, a security context is created and attached to that session (e.g., login session). The framework uses filtering mechanisms to control access to secured resources (URLs or objects). Aclamate provides the following key features: • URL and object security: Both URLs and objects can be declared secure. • Returned object security: Objects that have been declared as completely secured will not be returned as values. • Role based access control: Role based access control is supported using the Aclamate security constraint mechanism. • Data-level access control: Data-level access control is supported using the Aclamate security constraint mechanism. • Pluggable architecture: Major features of the framework are customisable using plugins. • Major UI framework support: Support for Apache Struts is included. Other frameworks will be supported via pluggable architecture. • JSP taglib: JSP tags are included to allow secure portions of a page.

Chapter 3. Related Work

| 57

The framework is driven by security policy files containing descriptions of how to secure resources. Constraints for role based access control (RBAC) and expression based access control (i.e., for datalevel access control or other dynamic filtering) are included with the framework. New constraint types can be created using the plug-in support. An example of such resource constraints are shown in Listing 3.11.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

{

} OR role - constraint { role =" admin "; }

role - constraint { role =" instructor "; } AND expression - constraint { expression =" this . classNumber GT ’CS -5 ’"; } AND expression - constraint { expression =" this . school IN edu . iit . aclamate . sample . SchoolFactory . getInstance () . lookupSchoolsByType ( ’ ENG ’) "; }

Listing 3.11: Aclamate RBC Configuration Example execute Methods The example illustrates a security constraint that is satisfied when the security context possesses the instructor role and the object being secured is from an engineering school with a class number starting with CS-5 or the security context simply possesses the admin role. The Aclamate framework allows adding security to an application without requiring changes throughout an application to enforce security checks. While changes to the user interface are necessary in order to improve the user experience, these changes are less invasive than would be required using traditional methods. Aclamate was available on Sourceforge at http://sourceforge.net/projects/aclamate with an example application illustrating its usage. Sadly, the project does not exist any more on Sourceforge. AspectJ is not the only available aspect oriented project for the Java programming language. Alternatives include JBossAOP9 and SpringAOP10 . Kung et al. [7] although mentioned in their paper that they had difficulties porting the AspectJ implementation of their access control aspects to the JBossAOP platform. The Spring Framework offers its own AOP implementation which is a bit different than the common AOP projects like JBossAOP and AspectJ. The Spring AOP does not need to control the class loader hierarchy and is thus suitable for use in a Servlet container or application server and there is no need for a special compilation process. The aim of Spring AOP is not to provide the most complete AOP implementation (although Spring AOP is quite capable); it is rather to provide a close integration between AOP implementation and Spring IoC (Inversion of Control Container) to help solve common problems in enterprise applications. In Spring AOP, aspects are normal Java classes which are being annotated with the @Aspect and @Pointcut("execution(* com.xyz.someapp.service.*.*(..))") annotations.
9 10

http://www.jboss.org/jbossaop [30/11/10] http://static.springsource.org/spring/docs/3.0.x/spring-framework-reference/html/aop.html
[30/11/10]

Chapter 3. Related Work

| 58

3.3 Summary
This Chapter tried to give an overview over the current research in the field of declarative security for web applications. The current research has to be divided into three main categories: (1) static and dynamic checking of source code for detecting vulnerabilities and application design issues; (2) using aspect oriented programming to add additional security checks to existing web applications in a non invasive way; (3) annotating Java source code for data flow analysis and detection of tainted, non validated user input. Annotating Java source code for data flow analysis might be a sensible approach to detect subtle web application vulnerabilities, yet it is a rather difficult task when faced with the need to add these annotations to an already deployed and productive web application. Enhancements to the AspectJ project would significantly improve the ability to do more fine grained security checks and data flow analysis, sadly the enhancements described in several research papers have not yet been implemented by the AspectJ community. Nonetheless, the non invasiveness of AspectJ is a huge advantage when an existing web application needs to be enhanced with additional security checks or any other additional functionality. Some security frameworks have been proposed and been successfully implemented using an AOP approach, but most of these frameworks are either not available any more or have been abandoned years ago such as Aclamate. Bart De Win states [37] (2002) that the generalisation of aspects is currently still difficult. The limitations of the aspect language sometimes makes it very hard or even impossible to generalise certain cases. As a result, a completely reusable security framework can hardly be developed yet.

Chapter 3. Related Work

| 59

4 Finding Web Application Vulnerabilities by using Declarative Information
This Chapter tries to identify possible ways to enhance the detection of security vulnerabilities in web applications with an automated security testing tool by incorporating declarative security information. The OWASP Top Ten project1 has been used to identify the most common web application vulnerabilities and for each one an appropriate solution has been proposed to enhance the detection of such vulnerabilities by using a white box testing approach. Each vulnerability from the OWASP Top Ten project is being shortly described and for each popular web application framework such as Struts 1.x, Struts 2.x, Spring and Wicket the framework specific features are shown which help a developer to prevent a particular vulnerability.

4.1 Overview
The goal of this thesis is to write an automated security testing tool which is able to detect web application vulnerabilities more reliably by incorporating declarative security information than their sole black box testing counterparts. Commonly available security testing tools only take an external perspective from the target application, which is also called black box testing. White box testing on the other hand takes more of an internal perspective on the targeted application and uses as much additional information as possible to derive test cases. Such additional information can consist of the actual source code of the application or metadata about the internal structure of the application as seen in Chapter 2 and 3. Figure 4.1 illustrates the basic architecture of such a combined white- and black box security testing application. The automated testing tool has to be configured (1) by an developer or testing agent for the specific web application. It might be possible, depending on the specific test case scenario, that the testing agent has to provide additional metadata in form of a machine readable format such as a XML file to the testing tool. What the contents of such a XML file might be will be elaborated in the following Sections and Chapters. The automated security testing tool needs access to the actual source code of the web application (2) in order to extract further data which helps it to craft meaningful test cases. After all the information has been collected the actual process of finding misconfigurations and application vulnerabilities begins (3). A human readable report for the testing agent should be created. The following Sections are based on the OWASP Top Ten critical web application security risks which have just recently (2010) been updated to include the following risks:

1

http://www.owasp.org/index.php/Top_10_2010 [07/04/11]

Chapter 4. Finding Web Application Vulnerabilities by using Declarative Information

| 60

Figure 4.1: Architecture of a general combined White- and Black Box Security Testing Tool

A1 Injection A2 Cross-Site Scripting (XSS) A3 Broken Authentication and Session Management A4 Insecure Direct Object References A5 Cross-Site Request Forgery (CSRF)

A6 A7 A8 A9 A10

Security Misconfiguration Insecure Cryptographic Storage Failure to Restrict URL Access Insufficient Transport Layer Protection Unvalidated Redirects and Forwards

The points A2 and A5 have been incorporated into Section 4.2 - Injection Flaws.

4.2 Injection Flaws
Injection flaws occur when an application sends untrusted data to an interpreter. Injection flaws are very prevalent, particularly in legacy code, often found in SQL queries, LDAP queries, XPath queries, OS commands, program arguments, etc. Injection flaws are easy to discover when examining code, but more difficult to find via testing. Scanners and fuzzers can help attackers find them. Injection can result in data loss or corruption, lack of accountability or denial of access. Injection can sometimes lead to complete host takeover. Injection flaws can be averted by proper validating and filtering received user input. As seen in Section 2.4 there are quite a few security frameworks available which offer functions for validating user input. However, most of the Java web application frameworks offer additional means for validating user input but these offer a more high level approach to filtering. They are not particularly concerned about security, they rather provide means for checking if the received user input is a

Chapter 4. Finding Web Application Vulnerabilities by using Declarative Information

| 61

number, telephone number, credit card etc. Libraries such as Common Validator (see Section 2.4.7) and ESAPI (see Section 2.4.4) should be used to orderly protect a web application from injection flaws. The Web Application Attack and Audit Framework (w3af)2 , which is a sole black box security testing tool, has the ability to detect persistent and non-persistent XSS (Cross-Site Scripting) and blind and none-blind SQLi (SQL Injection) flaws (among others such as CSRF flaws etc.). w3af’s detection routine for detecting SQLi injection flaws is rather simple. Blind SQLi are being found by inserting SQL commands into GET and/or POST parameters such as BENCHMARK(5000000,ENCODE(’MSG’,’by 5 seconds’)) which will delay server responses if the expression can be injected. If the server response takes longer than usual, then this will be a strong indication that the web application might suffer from a SQL injection vulnerability. None-blind SQLi flaws are being detected by parsing server responses for databse error messages such as “You have an error in your SQL syntax;”. A different approach has been suggested and realised by Denver [12], his approach combines blackand white box testing techniques for enhancing the detection rate of SQLi and blind-SQLi flaws by parsing database query log files. The following enumeration shows framework specific features which help a developer to filter user provided input to prevent Injection Attacks. Depending on the used web application framework these validation rules can be configured declaratively and might help an automated security testing tool to create more sophisticated test cases for evaluating if a given GET or POST parameter might be vulnerable to Injection Attacks:

• Apache Struts 1.x Uses the Apache Commons Validator framework as described in Section 2.4.7. Validation rules are being configured through a XML file called validation.xml which has to be placed inside the WEB-INF folder of a deployed Java web application. • Apache Struts 2.x Validation in Struts 2.x can either be done through annotations or through a XML configuration file. Struts 2.x relies on a validation framework provided by XWork3 which enables the Java web application to execute input validation rules before actual Actions are being executed. This Section only provides the bare minimum to get started and focuses on the Struts 2.x extension of the XWork validators. XWork is a command-pattern framework that is used to power Struts 2 as well as other applications. XWork provides an Inversion of Control container (IoC), a powerful expression language, data type conversion, validation and pluggable configuration. Listing 4.1 shows how a Struts Action class can be annotated to enforce user input validation constraints. In this example the parameter bar is a field parameter from a standard HTML form and has to be between 6 and 10 characters long. The necessary annotations are @Validation and then additionally the specific annotations based on the kind of validation the framework should enforce upon the parameter such as @RequiredFieldValidator and @IntRangeFieldValidator.
1 2 3 4 2 3

@Validation public class SimpleAnnotationAction extends ActionSupport { private int bar ;

http://w3af.sourceforge.net/ [13/12/10] http://www.opensymphony.com/xwork/ [13/12/10]

Chapter 4. Finding Web Application Vulnerabilities by using Declarative Information

| 62

5 6 7 8 9 10 11 12 13 14 15

}

@RequiredFieldValidator ( type = ValidatorType . FIELD , message = " You must enter a value for bar . " ) @IntRangeFieldValidator ( type = ValidatorType . FIELD , min = " 6 " , max = " 10 " , message = " bar must be between $ { min } and $ { max } , current value is $ { bar }. " ) public void setBar ( final int b ) { bar = b ; } ...

Listing 4.1: Struts 2.x XWorks Validation Example with Annotations The other possibility is to use a XML configuration file instead of Java annotations. Listing 4.2 shows the same example from Listing 4.1 but configured through a XML file.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

< validators > < field name =" bar " > < field - validator type =" required " > < message > You must enter a value for bar . </ message > </ field - validator > < field - validator type =" int " > < param name =" min " >6 </ param > < param name =" max " >10 </ param > < message > bar must be between $ { min } and $ { max } , current value is $ { bar }. </ message > </ field - validator > </ field > </ validators >

Listing 4.2: Struts 2.x XWorks Validation Example with XML Validation rules can be specified: – Per Action class: in a file named <ActionName>-validation.xml – Per Action alias: in a file named <ActionName>-<alias>-validation.xml – Inheritance hierarchy and interfaces implemented by Action class: XWork searches up the inheritance tree of the Action to find default validations for parent classes of the Action and interfaces implemented. • Spring Spring 3 introduces several enhancements to its validation support. Most notably, the JSR-303 Bean Validation API4 is now fully supported. JSR-303 defines a metadata model and API for JavaBean validation. The default metadata source is annotations, with the ability to override and extend the metadata through the use of XML validation descriptors. Listing 4.3 shows a simple Java class with JSR-303 validation annotations. The name parameter has to fulfil the following requirements before it is accepted by the validation routine/framework: (1) The data must match the regular expression which states that only the characters a-z, A-Z are allowed; (2) the length of the name must be between 3 and 30 characters; (3) the name parameter is not allowed to be null.
4

http://jcp.org/en/jsr/detail?id=303 [13/12/10]

Chapter 4. Finding Web Application Vulnerabilities by using Declarative Information

| 63

1 2 3 4 5 6 7

public final class User { @Pattern ( regexp = " [a - zA - Z ]* " ) @Size ( min =3 , max =30) @NotNull private String name ; ... }

Listing 4.3: Spring 3.x Validation Example with Annotations The validation of the class can then be done inside Spring controller classes by injecting a Validator object. Listing 4.4 shows how the User class from Listing 4.3 is being validated after the user submitted the corresponding HTML form.
1 2 3 4 5 6 7 8 9 10 11 12 13

@Controller public final class UserController { @Autowired private Validator validator ; @RequestMapping ( value = " / userForm " , method = RequestMethod . POST ) public String post ( User user , BindingResult result ) { validator . validate ( user , result ) ; if ( result . hasErrors () ) { return " form " ; } // ... } ...

}

Listing 4.4: Spring 3.x Validation Example in a Controller Class Yet easier to use is the @Valid annotation which is not a Spring specific construct but rather part of the JSR-303 Bean Validation API. This obsoletes the injection of a Validator instance. All a developer has to do is rewrite the post() method from Listing 4.4 to include the @Valid annotation as shown in the following code snippet:

@RequestMapping(value = "/userForm", method = RequestMethod.POST) public String post(@Valid User user, BindingResult result) { if (result.hasErrors()) { return "form"; } // ... }
An alternative to annotations is writing custom class validators by implementing the org.springframework.validation.Validator interface and checking parameters respectively values programmatically. • MyFaces Validation in MyFaces can be achieved by using the ExtVal project as described in Section 2.9 and Listing 2.35. ExtVal provides several Java annotations for input validation purposes such as @Pattern, @Length, etc. • Wicket Wicket validation is done programmatically during the creation of a form. The code in Listing 4.5 adds a simple text field called age to a HTML form inside a Wicket page. The age field is required and the provided data by the user has to be greater than 18. The NumberValidator class provides several factory methods and ready-made validators for validating numbers. It has methods for longs and doubles, but they work equally well for integers and floats, respectively. Here is an example:

Chapter 4. Finding Web Application Vulnerabilities by using Declarative Information

| 64

add(new TextField("age").add(NumberValidator.minimum(18))); add(new TextField("handicap").add(NumberValidator.range(0, 3.5))); add(new TextField("duration").add(NumberValidator.POSITIVE));
The StringValidator class can be used to check strings, which also specifies several factory methods. Here are some examples of its use:

add(new TextField("userid").add(StringValidator.lengthBetween(8,12))); add(new TextField("comment").add(StringValidator.maximumLength(4000)));
There are several more validators available in the Wicket framework such as a validator to check user input against regular expressions etc. More information about validators can be found on the Wicket project page5 .
1 2 3 4 5 6 7 8 9 10 11 12 13 14

public class FormInput extends WicketPage { private class InputForm extends Form { public InputForm ( String name ) { super ( name , new CompoundPropertyModel ( new FormInputModel () ) ) ; form . add ( new TextField ( " age " ) . setRequired ( true ) . setLabel ( new Model ( " age " ) ) . add ( NumberValidator . minimum (18) ) ) ; ... } ... } ... }

Listing 4.5: Wicket Validation Example With the above mentioned input validation rules, that can be added declaratively to a web application by either providing a XML configuration file or by annotating the Java source code the following approaches could be used by a combined white- and black-box security testing tool to enhance the detection of parameters which might be vulnerable to Injection Attacks: • Validation Completeness Check: The XML configuration files and Java annotations could be used to check if every GET or POST parameter, found through a sole black box web spidering component or by statically analysing the Java source code of the application, is being referred to in a configured validation rule. If a GET or POST parameter has been discovered that is not being tackled by a configured validation rule inside a XML configuration file or a Java annotation, this might indicate that the found parameter suffers from a possible code injection vulnerability. This idea might generate false positives because not all found web application parameters such as session IDs have automatically to be validated and even maliciously tampering with them might not lead to a exploitation of the targeted web application. • SQLi Detection: The following suggestion is not a real testing scenario which uses declarative information but rather another interesting idea for finding injection vulnerabilities by using an AOP approach (see Section 3.2). By using AOP it would be possible to add aspects before typical JDBC database functions are being executed such as boolean execute(String query), ResultSet executeQuery(String query) and int executeUpdate(String query) (all defined in java.sql.Statement.* of the Java 6.0 API). These aspects would then
5

http://wicket.apache.org/apidocs/1.4/org/apache/wicket/validation/validator/packagesummary.html [07/04/11]

Chapter 4. Finding Web Application Vulnerabilities by using Declarative Information

| 65

be able to examine the queries issued to the database tier. In combination with a black box testing tool which tries to insert maliciously crafted SQL statements into GET and/or POST parameters, unvalidated input parameters could be detected with much more precision than common black box security testing tools. This approach is basically the same as the one proposed by Denver [12] but without the need of parsing database query log files. This aspect oriented approach would provide the same benefits and thoroughness as Denver’s proposal, the only prerequisite is that the existing web application would have to be recompiled with AspectJ to incorporate the newly proposed aspects which analyse the issued SQL statements to the database tier. Additionally, this approach could be enhanced to detect stored XSS flaws as well. Using this approach for detecting reflected XSS vulnerabilities makes no sense because sole black box security testing tools can detect them equally well without a deeper understanding of the inner workings of the targeted web application. • Detection of Unvalidated Redirects and Forwards: See Section 4.9. • Detection of OS Command Execution: By adding an aspect to the Process java.lang.Runtime.exec(String) method it would be possible to verify and check if arguments passed to the exec(String) method can be injected by a malicious user such as nc -l -p 3333 -v -e /bin/bash which opens a remote shell on the target host. Injecting malicious OS commands could be automated by using a spidering component similar to detecting SQLi vulnerabilities. • Detection of Dangerous File Access: By adding an aspect to the constructor of the java.io.File class it becomes possible for an aspect to verify if the file path can be supplied by a malicious user such as /etc/passwd on Linux operating systems. Other methods worth monitoring include the following: constructor of the java.io.FileReader class, constructor of the java.io.FileInputStream class, constructor of the java.io.BufferedInputStream class, constructor of the java.io.RandomAccessFile class through an automated spidering component similar to detecting SQLi vulnerabilities.

4.3 Broken Authentication and Session Management
Developers frequently build custom authentication and session management schemes, but building these correctly is hard. As a result, these custom schemes frequently have flaws in areas such as logout, password management, timeouts, remember me, secret question, account update, etc. Finding such flaws can sometimes be difficult, as each implementation is unique. Such flaws may allow some or even all accounts to be attacked. Once successful, the attacker can do anything the victim could do and thus privileged accounts are frequently targeted. Except for some simple automated tasks such as checking if the session IDs are chosen randomly enough to prevent an attacker from guessing session IDs and checking whether the session IDs are being reinitialised after a user authenticated himself successfully to the application for preventing Session Fixation attacks, detecting other session management vulnerabilities in an automated way becomes infeasible without doing a manual code review and is thus being ignored in this thesis.

4.4 Insecure Direct Object References
Applications frequently use the actual name or key of an object when generating web pages. Applications do not always verify if the user is authorised to access the targeted object. This results in an insecure direct object reference flaw. Testers can easily manipulate parameter values to detect such

Chapter 4. Finding Web Application Vulnerabilities by using Declarative Information

| 66

flaws and code analysis quickly reveals whether authorisation is properly verified. Such flaws can compromise all the data that can be referenced by the parameter. Unless the name space is sparse, it is easy for an attacker to access all available data of that type. The OWASP ESAPI security library as described in Section 2.4.4 provides a class called Access ReferenceMap. This class is especially designed for preventing insecure direct object references attacks. A simple example can be seen in Listing 4.6. The AccessReferenceMap interface is used to map from a set of internal direct object references to a set of indirect references that are safe to disclose publicly. This can be used to help protect database keys, file names and other types of direct object references. As a rule, developers should not expose their direct object references as it enables attackers to attempt to manipulate them. Indirect references are handled as Strings, to facilitate their use in HTML. Implementations can generate simple integers or more complicated random character strings as indirect references. Implementations should probably add a constructor that takes a list of direct references. Note that in addition to defeating all forms of parameter tampering attacks, there is a side benefit of the AccessReferenceMap. Using random strings as indirect object references, as opposed to simple integers makes it impossible for an attacker to guess valid identifiers. So if per-user AccessReferenceMaps are used, then request forgery (CSRF) attacks will also be prevented6 .
1 2 3 4 5 6 7 8 9 10 11 12 13

... Set fileSet = new HashSet () ; fileSet . addAll (...) ; // add direct references ( e . g . File objects ) AccessReferenceMap map = new AccessReferenceMap ( fileSet ) ; // store the map somewhere safe - like the session ! String indRef = map . getIndirectReference ( file1 ) ; String href = " http :// www . aspectsecurity . com / esapi ? file = " + indRef ) ; ... // if the indirect reference doesn ’t exist , it ’s likely an attack // getDirectReference throws an AccessControlException // you should handle as appropriate String indref = request . getParameter ( " file " ) ; File file = ( File ) map . getDirectReference ( indref ) ;

Listing 4.6: ESAPI Example on how to use the AccessReferenceMap Class Whether direct references are being used by a web application can easily be determined with a sole black box security testing application which is able to analyse GET and POST parameters for repeated patterns in URLs such as page1.php?userid=42&fileid=5, page1.php?userid=43&fileid=6, etc. None of the popular web application frameworks offer a feature for preventing insecure direct object references except for Wicket. Wicket is secure by design to these kind of attacks because of its unique MVC model which does not expose sensitive information in URLs and all component paths are session relative (see Section 2.10). Because there exists no declarative information in web application frameworks that would prevent insecure direct object references and who could be extracted by a security testing framework these kind of vulnerabilities are being ignored in this thesis.

6

http://owasp-esapi-java.googlecode.com/svn/trunk_doc/latest/org/owasp/esapi/ AccessReferenceMap.html [13/12/10]

Chapter 4. Finding Web Application Vulnerabilities by using Declarative Information

| 67

4.5 Security Misconfiguration
Security misconfiguration can happen at any level of an application stack, including the platform, web server, application server, framework and custom code. Developers and network administrators need to work together to ensure that the entire stack is configured properly. Automated scanners are useful for detecting missing patches, misconfigurations, use of default accounts, unnecessary services, etc. Such flaws frequently give attackers unauthorised access to some system data or functionality. Occasionally, such flaws result in a complete system compromise. Tools like Nessus7 or OpenVAS8 do a pretty good job on detecting security misconfigurations and thus these kind of security vulnerabilities are being ignored in this thesis. The Open Vulnerability Assessment System (OpenVAS) is a framework of several services and tools offering a comprehensive and powerful vulnerability scanning and vulnerability management solution. The Nessus vulnerability scanner features high-speed discovery, configuration auditing, asset profiling, sensitive data discovery and vulnerability analysis of a companies security posture.

4.6 Insecure Cryptographic Storage
The most common flaw in this area is simply not encrypting data that deserves encryption. When encryption is employed, unsafe key generation and storage, not rotating keys and weak algorithm usage is common. Use of weak or unsalted hashes to protect passwords is also common. External attackers have difficulty detecting such flaws due to limited access. They usually must exploit something else first to gain the needed access. Failure frequently compromises all data that should have been encrypted. Typically this information includes sensitive data such as health records, credentials, personal data, credit cards, etc. Whether data should or should not be encrypted inside the data store is a decision which can not be made by an automated security testing tool and thus this point from the OWASP Top Ten is being ignored in this thesis.

4.7 Failure to Restrict URL Access
Web applications are not always protecting page requests properly. Sometimes, URL protection is managed via configuration and the system is misconfigured. Sometimes, developers must include the proper code checks, and they forget. Detecting such flaws is easy. The hardest part is identifying which pages respectively URLs exist to attack. An attacker, who is an authorised system user, simply changes the URL to a privileged page. If there is a misconfiguration the access is granted to the attacker even if he does not provide the necessary user role to access that specific page respectively URL. By using declarative information provided by the Java web application frameworks the disadvantage of not knowing the privileged resources respectively URLs can be overcome by parsing specific configuration files. The following Sections elaborate which configuration files contain the necessary information for each of the popular Java web application frameworks:

7 8

http://www.nessus.org/nessus/intro.php [13/12/10] http://www.openvas.org/ [13/12/10]

Chapter 4. Finding Web Application Vulnerabilities by using Declarative Information

| 68

• Application Container The deployment descriptor as seen in Section 2.2.2 and Listing 2.8 contains valuable information about security constraints which are being enforced by the container and are being configured based on URLs. Additionally, the deployment descriptor might contain information about existing users and roles. So a white box security testing tool would know at least what kind of users and roles are existent and would be able to extract a list of restricted URL paths. This is only true if the web application uses container managed security features and the constraints are thus being configured in the deployment descriptor. If the web application rather uses Java annotations than a deployment descriptor then the same information can be extracted by parsing the source code of the Java web application. The annotations used to configure access constraints equal to the ones configured in the deployment descriptor are called @ServletSecurity as seen in Section 2.2.5 and @DeclareRoles as seen in Section 2.2.3. This is only true if the web application uses container managed security features and the constraints are thus being configured through Java annotations inside the source code of the Java web application. • Struts 1.x As seen in Section 2.7 the Struts Web Application Framework is being configured through a XML file called struts-config.xml. This particular configuration file contains Action mappings which have to be configured by a developer in order to use the web application framework at all. These mappings map relative URLs to Java classes implementing the desired functionality of the developer. By parsing the struts-config.xml configuration file a complete list of all the declaratively configured URLs a targeted web application serves to its users can be extracted. The struts-config.xml configuration file might even contain declarative information about existing user roles and privileges based on the roles element as seen in Section 2.7.1. • Struts 2.x Similar to Struts 1.x the Struts 2.x Web Application Framework provides the same information through a XML file called struts.xml as seen in Section 2.8 and Listing 2.32. Additionally to the XML configuration file, Struts 2.x offers Java annotations to configure the same information inside the source code of the web application. By parsing the source code of the targeted web application the following annotation is relevant @Result as seen in Listing 2.31. • Spring Spring similar to Struts 1.x and Struts 2.x provides declarative information about Action mappings which maps relative URLs to particular Java classes implementing the desired functionality by the developer through the use of Java annotations. The specific annotations are called @RequestMapping as seen in Section 2.6 and can be extracted by a security testing tool to gather a complete list of all the URLs a targeted web applications serves to its users. • Wicket In case of the Wicket framework (see Section 2.10) nothing like declarative Action mappings exist for mapping relative URLs to Java classes as it is the case in Struts 1.x or Struts 2.x based web applications. In Wicket, links are being programmatically created and added to the view through specific code instructions in the Java class files. The following snippet shows two ways how a common link is being created in the Wicket framework:

// Method 1 String url = urlFor(ConfirmRegistrationPage.class, new PageParameters("id=" + registrationId));

Chapter 4. Finding Web Application Vulnerabilities by using Declarative Information

| 69

// Method 2 add(new BookmarkablePageLink("link", ConfirmRegistrationPage.class));
In order to extract a complete list of possible URLs a Wicket application serves for its users it would be necessary to scan the source code of the targeted web application and look for urlFor(...) and new *link(...) commands. Besides the already mentioned web application frameworks, libraries used to secure Java web applications also provide configuration files containing declaratively added information about existing URLs a particular web application serves or rather URLs that have to be protected by the said security library as seen in Listing 2.2. Usually these configuration files do not contain a complete list of all the available URLs a particular web application serves but rather only the ones worth protecting from the public like an administrator interface. • Spring Security As seen in Section 2.4.2 the Spring Security library is being configured through a XML file called *-security.xml and might contain a non complete list of URLs that have to be protected as well as information about existing users and roles as Listing 2.13 shows. • ESAPI WAF The ESAPI web application firewall as seen in Section 2.4.4 is being configured through a XML policy file shown in Listing 2.19 and might contain a non complete list of URLs that are being served by the targeted web application and have to be protected by the web application firewall. • Apache Shiro Similar to the ESAPI WAF the Apache Shiro project (see Section 2.4.5) might provide a configuration file containing relative URLs that are being served to the users of the targeted web application but have to be protected by the Apache Shiro library as shown in the INI configuration file in Listing 2.20. Eventually, if the developer uses the roles filter chain then the configuration file also contains information about existing users and roles. • jGuard The example configuration file of jGuard shown in Listing 2.21 also contains a non complete list of relative URLs with declaratively added information about which roles are being allowed for a particular resources in the web application. After a white box security testing tool parsed the desired configuration files of a Java web application framework or a library used to secure a Java web application the following information is available to the testing tool: a (non) complete list of the URLs the targeted web application serves and a (non) complete list of existing users and roles the web application knows about. With the mentioned white box information extracted from the configuration files, the following approaches become feasible to detect URL access restriction vulnerabilities or access control misconfigurations more precisely than a sole black box security testing tool: • Enhancing Spidering Coverage: Spidering or crawling a web application is usually the first step a black box security testing tool performs prior to launching routines for finding vulnerabilities in the targeted web application. A web crawler is one type of bot or software agent. In general, it starts with a list of URLs to visit, called the seeds. As the crawler visits these URLs, it identifies all the hyperlinks in the page and adds them to the list of URLs to visit. Black box security testing tools usually want only one URL to start their spidering process, e.g.:

Chapter 4. Finding Web Application Vulnerabilities by using Declarative Information

| 70

http://www.example.com. How many pages and resources are then discovered by the spider solely depends on how well the spider is implemented and how well the spider component interacts with the targeted web application. This includes filling out forms and interpreting dynamic scripting languages such as JavaScript. The coverage (how many different pages the spider is able to find) can be increased by providing the spidering routine with a larger amount of seeds to start with. These seeds can be provided by parsing the previously mentioned configuration files or source code of the targeted web application. This largely reduces the issue that a black box security testing application has, namely identifying pages respectively URLs to attack for finding unprotected resources. • RBAC Policy Checks: Given the previously mentioned configuration files not only contain a (non) complete list of URLs but also information about which users and roles are allowed to access a particular resource (or a collection of URLs) it would be possible for a combined whiteand black-box security testing tool to conduct role based access control policy checks. Given such a security testing tool would be able to impersonate each existing and configured user role known to the web application, a map of the web application could be created by using an automated spidering component recording a list of all the URLs a particular user role has been granted access to and the allowed transitions between HTML pages. This would allow a developer or testing agent to easily spot URLs respectively pages which should have been protected by an access control mechanism but are nonetheless accessible to user roles with lower access rights due to a missing access control check or due to a misconfiguration.

4.8 Insufficient Transport Layer Protection
Applications frequently do not protect network traffic. They may use SSL/TLS during authentication, but not elsewhere, exposing data and session IDs to interception. Expired or improperly configured certificates may also be used. Detecting basic flaws is easy by observing the site’s network traffic. More subtle flaws require inspecting the design of the application and the server configuration. Such flaws expose individual user data and can lead to account theft. If an administrator account was compromised, the entire site could be exposed. Poor SSL setup can also facilitate phishing and Man-In-The-Middle (MITM) attacks. Configuration files with transport layer constraints include the following: • Application Container The deployment descriptor might contain an optional <user-data-constraint> element. A user data constraint establishes a requirement that the constrained requests be received over a protected transport layer connection. This guarantees how the data will be transported between client and server. The choices for type of transport guarantee include NONE, INTEGRAL and CONFIDENTIAL. If no user data constraint applies to a request, the container must accept the request when received over any connection, including an unprotected one. Listing 2.8 shows an example deployment descriptor with a user data constraint that protects all URLs with the prefix /catalog/cart/* through SSL/TLS connections. If the web application rather uses Java annotations than a deployment descriptor then the same information can be extracted by parsing the source code of the Java web application. The annotations used to configure access constraints equal to the ones configured in the deployment descriptor are called @ServletSecurity and @HttpMethodConstraint as seen in Section 2.2.5 and Listing 2.11. This is only true if the web application uses container managed secu-

Chapter 4. Finding Web Application Vulnerabilities by using Declarative Information

| 71

rity features and the constraints are thus being configured through Java annotations inside the source code of the Java web application. • Struts 1.x Struts 1.x does not offer a way to add declarative transport constraints in any form to an existing web application. The only way to add non container managed transport layer constraints is to write a custom RequestProcessor as described in Section 2.7.1. • Struts 2.x For the Apache Struts 2.x Framework (see Section 2.8) there exists a project called struts2-sslplugin9 which goal it is to provide a plugin to enforce HTTPS constraints through Java annotations. The following code snippet shows how to use the plugins @Secured annotation to enforce HTTPS connections for a particular Struts Action:

@Secured public class IndexAction extends ActionSupport { plubic String execute() { // ... return SUCCESS; } }
• Spring Spring based applications which use the Spring Security project contain a configuration file as shown in Listing 2.13. By adding requires-channel="https" to an URL constraint, Spring Security enforces that the data shared between the server and the user is being protected through SSL/TLS. • Wicket The Apache Wicket framework (see Section 2.10) also offers a Java annotation called @RequireHttps which can be used by a developer to annotate page classes. The framework will then enforce that these pages can only be accessed through the HTTPS protocol. • ESAPI WAF The ESAPI WAF XML configuration file offers a <enforce-https> element for specifying a list of URLs which can only be accessed through HTTPS, otherwise the WAF will redirect standard HTTP traffic to HTTPS. More on ESAPI can be found in Section 2.4.4. The following excerpt shows how to completely make a web application accessible over HTTPS only with the exception of the index.html page and the images folder:

<url-rules> <enforce-https path="/.*"> <path-exception>/index.html</path-exception> <path-exception type="regex">/images/.*</path-exception> </enforce-https> </url-rules>
• Apache Shiro The Apache Shiro configuration file as seen in Section 2.4.5 provides a SslFilter chain which enforces HTTPS connections for a specific set of URLs.

9

http://code.google.com/p/struts2-ssl-plugin/ [11/12/10]

Chapter 4. Finding Web Application Vulnerabilities by using Declarative Information

| 72

After a white box security testing tool parsed the desired configuration files of a Java web application framework or a library used to secure a Java web application, the following information is available to the testing tool: a (non) complete list of the URLs the targeted web application serves and a complete list of URLs that are being protected over SSL/TLS. With the mentioned white box information extracted from the configuration files, the following approaches become feasible for detecting missing transport layer constraints: • Transport Layer Policy Checks: Checking whether all or some specific pages are properly secured through SSL/TLS is easily done by spidering the targeted web application and comparing the pages which are only accessible through SSL/TLS and those who are not with a pre-defined list of URLs that should be protected by SSL/TLS. This would allow a developer/administrator to quickly spot misconfigurations in the way resources are being protected by the HTTPS protocol. Such a pre-defined URL list can either be provided by an administrator/developer or can be collected automatically from various configuration files as seen in Section 4.7. Whether spidering the targeted web application in order to find unprotected resources is really necessary or if it is sufficient to only parse the configuration files with transport layer constraints has to be evaluated later if necessary.

4.9 Unvalidated Redirects and Forwards
Applications frequently redirect users to other pages or use internal forwards in a similar manner. Sometimes the target page is specified in an unvalidated parameter, allowing attackers to choose the destination page. Detecting unchecked redirects is easy by finding POST or GET parameters containing a full URL or parameters where one can set a full URL. Unchecked forwards are harder to find since they target internal pages. Such redirects may attempt to install malware or trick victims into disclosing passwords or other sensitive information. Unsafe forwards may allow a malicious user to bypass access controls. Unvalidated redirects and forwards can easily be detected with sole black box security testing tools by analysing GET and POST parameters and looking for full URLs in parameters such as http://www.example.com/redirect.php?url=http://www.example.com/page2.php. Additionally to using a sole black box security testing tool as mentioned above, an AOP approach can also be used for detecting unvalidated redirects and forwards. Given the web application uses the void sendRedirect(String) method to programmatically issue redirects, an aspect could be used to check whether the parameter passed to the sendRedirect(String) function can be supplied by a malicious user. The sendRedirect(String) method is specified in the Servlet API in javax.servlet.http.HttpServletResponse. Again, the only prerequisite is that the existing web application needs to be recompiled with AspectJ to incorporate the newly proposed aspects which analyse the passed parameters to the sendRedirect(String) method.

4.10 Summary
This Chapter tried to identify possible ways to enhance the detection of security vulnerabilities in web applications with an automated security testing tool by incorporating declarative security information. The OWASP Top Ten project has been used to identify the most common web application vulnerabilities and for each one an appropriate solution has been proposed to enhance the detection of such vulnerabilities by using declarative information.

Chapter 4. Finding Web Application Vulnerabilities by using Declarative Information

| 73

The following common web application vulnerabilities have been covered in this Chapter: Injection Flaws, Broken Authentication and Session Management, Insecure Direct Object References, Security Misconfigurations, Insecure Cryptographic Storage, Failure to Restrict URL Access, Insufficient Transport Layer Protection and Unvalidated Redirects and Forwards. Not all of these vulnerabilities can be enhanced by incorporating declarative security information in an automated white box security testing tool such as Broken Authentication and Session Management, Security Misconfigurations, Unvalidated Redirects and Forwards, Insecure Direct Object References and Insecure Cryptographic Storage. Detecting Broken Authentication and Session Management vulnerabilities can only be spotted by a manual code review and there are no declarative security information in the configuration or source code files of a targeted web application which could be used by an automated security testing tool. Detecting Security Misconfigurations would be feasible if the approach would be narrowed down to a few specific cases where declarative security information is available in the targeted web application such as detecting weak cryptographic algorithms in SSL certificates used by the application container but there are already very good tools available to do these kind of checks (see Nessus, OpenVAS and openssl). Detecting Insecure Cryptographic Storage issues is another case where there is really no available declarative information available in the configuration- or source code files of a targeted web application and can only be resolved by a manual code review and thus is being ignored in this thesis. Detecting Unvalidated Redirects and Forwards issues can be rather easily spotted by an automated black box only security testing tool that spiders the targeted web application and can not really be enhanced by using white box information sources and thus is being ignored in this thesis. Detecting Insecure Direct Object References issues is also a candidate for an automated black box only security testing tool and the common Java web application frameworks do not offer declarative information regarding direct object references. A black box security testing tool could easily detect insecure direct object references by analysing URLs for repeated patterns between multiple URLs such as a repeated user id page.php?userid=5, page.php?userid=6 found through spidering the targeted web application. On the other hand issues such as Injection Flaws, Insufficient Transport Layer Protection and Failure to Restrict URL Access qualify for using white box information sources to enhance the precision and detection of security threats in an automated fashion in Java web applications. Injection Flaws can be detected more precisely by incorporating web application framework specific validation configuration files such as configuration files used by the Apache Commons Validator project or by scanning the Java source code of the web application for framework specific validation Java annotations. The automated security testing tool would then be able to find GET or POST parameters by spidering the targeted web application which have not been mentioned in one of the frameworks validation routines which would suggest an Injection Flaw. This is certainly a very promising approach because Injection Flaws are still the most prevalent vulnerabilities found in web applications. The WhiteHat Website Security Statistic Report (Fall 2010) estimates that Cross-Site Scripting (71%) and Information Leakage (70%) remain by far the most prevalent vulnerabilities occurring in 7 out of 10 websites10 . The biggest challenge of this approach is the statical code
10

http://www.whitehatsec.com/home/resource/stats.html [10/01/11]

Chapter 4. Finding Web Application Vulnerabilities by using Declarative Information

| 74

analysis for detecting unvalidated GET and POST parameters. The problem is that the frameworks do not offer one way to achieve object population and validation checks but rather several and it becomes difficult to distinguish between Java variables which fulfil a particular purpose in the code or are being provided by a user through a HTML form. Most web application frameworks such as Struts 1 & 2 and Spring automatically call Java setter methods in the Java classes associated with the HTML forms to populate the Java objects with the values entered by a user in the displayed HTML input elements. A static code scanning routine would have to know what to look for in the source code of the application depending on the used method. This rules out the detection of parameter validation code done through third party libraries such as ESAPI and the like and the approach can only be successful if the provided validation means of the framework are being used by the developer. If this approach is at all achievable in a feasible and tangible way is hard to tell but would nevertheless provide a testing agent or developer with relevant and interesting information about possible Injection Flaws. Even by using a crawling component to identify all available HTML forms respectively GET and POST parameters, the code or the configuration files would have to be checked whether the found parameters are being validated or not. This would simplify the problem only if all the validation rules and constraints have been configured in a single configuration file, otherwise the statical code analysis would still be needed to make educated decisions whether a parameter is being properly validated or not. Spring is certainly the most complicated framework in respect of the many ways parameter validation can be achieved and Struts 1.x is certainly one of the easier ones. Limiting the implementation to Struts 1.x only might not be a good idea because it would significantly reduce the usefulness of the security testing tools because newer Java web applications should not be developed using the relatively old Struts 1.x web application framework. In order to implement this approach the web application would have to meet several criteria such as used libraries and methodologies in parameter validation in order to guarantee useful results. Transport layer constraints can be configured in various ways such as in the deployment descriptor of an application container, through web application framework specific configuration files or framework specific Java annotations and thus qualifies for the use in an automated white-box security testing tool to detect Insufficient Transport Layer Protection issues. The security testing tool would be able to spider the targeted web application and record non SSL/TLS secured resources with SSL/TLS protected resources which would help a developer to spot misconfigurations and even discrepancies in configured transport layer constraints and actually deployed constraints. Failure to Restrict URL Access issues can be detected through the use of white box information sources coming from specific web application framework configuration files or the actual source code of the targeted web application in combination with a black box spidering component. This would allow a security testing tool do discover missing access control checks and misconfigurations in the way access constraints are being enforced throughout the Java web application. By spidering the targeted web application by impersonating user roles known to the web application a map of all resources a particular role has been granted access can be created. This map allows a developer or testing agent to easily spot misconfigurations in enforced access constraints and even discrepancies between intended access constraints and the actually enforced constraints in the deployed web application.

Chapter 4. Finding Web Application Vulnerabilities by using Declarative Information

| 75

5 Finding Privilege Escalation Vulnerabilities by using Declarative Information
The following Chapter finalises the main idea of this thesis, namely detecting privilege escalation vulnerabilities and misconfigurations in role based access control checks (RBAC) in web applications by using declarative information provided by web application framework specific configuration files. After analysing the current research in the field of enhancing web application security through the use of declarative information, as outlined in Chapter 3, did not reveal any suitable idea worth continuing for this thesis and evaluating possibilities of finding common web application vulnerabilities through the use of declarative information in Chapter 4, the idea of doing role based access control policy checks seems to be the most valuable for a developer or testing agent (see Section 4.7). Finding Injection Flaws through the use of declarative information as described in Section 4.2 would also be very beneficial for a developer or testing agent but this topic, especially the detection of SQLi vulnerabilities by using an automated as well as a combined white- and black-box security testing tool, has already been discussed and implemented by Denver [12].

5.1 Differences between RBAC and ABAC
Before the actual idea of this thesis is outlined in Section 5.2 it might be necessary to make a distinction between role based access controls (RBAC) and attribute based access controls (ABAC) because the proposed approach is currently only suited for web applications enforcing role based access constraints. The main difference between RBAC and ABAC is that ABAC also considers the object respectively data on which a subject wants to perform any kind of operation. Typical web application frameworks do not offer the ability to configure ABAC policies, they are instead providing RBAC policies based on URL constraints such as only allowing a user to access certain URLs based on his role (see Sections 2.8 for an example). There are quite a few Java libraries/frameworks for parsing and enforcing XACML policy files such as: Sun’s SunXACML project and PicketBox XACML. XACML is a declarative access control policy language implemented in XML and a processing model, describing how to interpret the policies. Instead of only using a role as the determining factor whether to grant access or not, many attributes can be used. Roles can be used in ABAC as well - since ABAC can use multiple attributes to make access control decisions, so ABAC can emulate RBAC perfectly while adding many additional advantages. This means that it is possible to add context to the access control decisions and adds for a finer granularity, tighter controls and more flexibility for the business. With XACML, policies can be woven around a combination of subjects (users/user-agents etc), resources (on which the access control is desired) and environment (IP address, Date, Time etc). For more information about this topic see Section 3.1. Web applications which do not use a XACML based approach usually enforce ABAC policies programmatically. Another way of enforcing ABAC policies, without using the XACML specification,

Chapter 5. Finding Privilege Escalation Vulnerabilities by using Declarative Information

| 76

can be achieved by implementing a custom AccessController provided by the ESAPI project (see Section 2.4.4). The AccessController interface defines a set of methods that can be used in a wide variety of applications to enforce access controls. In most applications, access control checks must be performed in multiple different locations across the various application layers. Finding privilege escalation vulnerabilities in web applications using ABAC policies is not a trivial task, especially when the policies are being programmatically enforced because of the wide variety of possible ways to implement such checks. An alternative way of doing completeness checks for ABAC policies involves a great deal of configuration effort for a testing agent. Because statical code analysis is infeasible, due to the complexity of programmatically enforced ABAC checks, a lot of additional white box information has to be provided by a testing agent such as when a particular resource is being protected based on a time constraint etc. and is thus being ignored for this thesis.

5.2 Detecting Privilege Escalation Vulnerabilities (RBAC)
Privilege escalation is the act of exploiting a configuration oversight to gain elevated access to resources that are normally protected from a user. The result is that a user with more privileges than intended by the application developer can perform unauthorised actions. This corresponds with one of the OWASP Top Ten points Failure to Restrict URL Access described in Section 4.7. The idea of this approach is to find misconfigurations in the way users respectively user roles have been granted access to specific parts of the web application. This boils down to a completeness check and allows a developer or testing agent to easily spot misconfigurations if given a list (or graphical representation such as a directed graph) of all the URLs and page transitions which are accessible to a specific user role (e.g.: GUEST, MANAGER, etc.). Any mismatch between declared access control rules and actual reachability is flagged as a security vulnerability. The testing tool would be able to determine if some parts of the web application can not be reached at all or if there are ways to reach a specific page that requires high access privileges via any path that allows access with lower privileges. As described in Section 4.7 the hardest part for a malicious user is identifying which pages respectively URLs exist to attack. An attacker, who is an authorised system user, simply changes the URL to a privileged page. If there is a misconfiguration the access is granted to the attacker even if he does not provide the necessary user role to access that specific page respectively URL. Additionally, to broaden the approach for a combined white- and black box security testing approach the developed tool should be aware of all the available users respectively roles the targeted web application knows about in order to check all available URLs with each available user role for privilege escalation attacks.

5.2.1 Extractable Declarative Information
The necessary information is being declaratively added by a developer during development to web application framework specific configuration files, deployment descriptors or even to the source code of the web application through Java annotations. The following Sections describe for each popular Java web application framework: (1) which configuration or source code files contain a (non) complete list of available URLs and (2) which configuration or source code files contain a (non) complete list of user roles. Some of the configuration files outlined here have already been discussed in Section 4.7 but are nonetheless repeated for the sake of completeness.

Chapter 5. Finding Privilege Escalation Vulnerabilities by using Declarative Information

| 77

Application Container

The deployment descriptor as seen in Section 2.2.2 and Listing 2.8 contains valuable information about security constraints which are being enforced by the container and are being configured based on URLs. These configured URL patterns give a first overview over the overall structure of the targeted web application. Important to note is that these configured URL patterns in the deployment descriptor represent constraints and might not contain a complete list of all available URLs the web application serves to its users. Additionally, the mentioned security constraints might contain information about which user roles are allowed access to configured URLs. Again, these are only security constraints and might not contain a complete list of all the available users. If the web application rather uses Java annotations than a deployment descriptor then the same information can be extracted by parsing the source code of the Java web application. The annotations used to configure access constraints equal to the ones configured in the deployment descriptor are called @ServletSecurity as seen in Section 2.2.5 and @DeclareRoles as seen in Section 2.2.3. All the above mentioned information sources only apply if the web application uses container managed security features and the constraints are thus being configured either through Java annotations or through the deployment descriptor. In case the web application is being deployed inside an Apache Tomcat Web Application Server then an additional configuration file called $CATALINA_BASE/conf/tomcat-users.xml might contain a (non) complete list of existing users and user roles. The tomcat-users.xml configuration file is being referred to as a UserDatabaseRealm in Tomcat’s documentation1 . UserDatabaseRealm is an implementation of the Tomcat Realm interface that uses a JNDI resource to store user information. By default, the JNDI resource is backed by a XML file. It is not designed for large-scale production use. At startup time, the UserDatabaseRealm loads information about all users and their corresponding roles, from an XML document. The users, their passwords and their roles may all be editing dynamically, typically via JMX. Changes may be saved and will be reflected in the XML file. The following snippet shows an excerpt from a tomcat-users.xml configuration file containing three different users:

<tomcat-users> <user name="tomcat" password="tomcat" roles="tomcat" /> <user name="role1" password="tomcat" roles="role1" /> <user name="both" password="tomcat" roles="tomcat,role1" /> </tomcat-users>
Struts 1.x

As seen in Section 2.7 the Struts Web Application Framework is being configured through a XML file called struts-config.xml. This particular configuration file contains Action mappings which have to be configured by a developer in order to use the web application framework at all. These mappings map relative URLs to Java classes implementing the desired functionality of the developer. By parsing the struts-config.xml configuration file a complete list of all the declaratively configured URLs a targeted web application serves to its users can be extracted. The struts-config.xml configuration file might even contain declarative information about existing user roles and privileges based on the roles element as seen in Section 2.7.1.
1

http://tomcat.apache.org/tomcat-7.0-doc/realm-howto.html#UserDatabaseRealm [08/04/11]

Chapter 5. Finding Privilege Escalation Vulnerabilities by using Declarative Information

| 78

Struts 2.x

Similar to Struts 1.x, the Struts 2.x Web Application Framework provides the same information through a XML file called struts.xml as seen in Section 2.8 and Listing 2.32. Additionally to the XML configuration file, Struts 2.x offers Java annotations to configure the same information inside the source code of the web application. By parsing the source code of the targeted web application the following annotation is relevant @Result as seen in Listing 2.31.
Spring

Spring, similar to Struts 1.x and Struts 2.x, provides declarative information about Action mappings which maps relative URLs to particular Java classes implementing the desired functionality by the developer through the use of Java annotations. The specific annotations are called @RequestMapping as seen in Section 2.6 and can be extracted by a security testing tool to gather a complete list of all the URLs a targeted web applications serves to its users.

5.2.2 Crawling the Web Application
Now that all declaratively added data is available to the security testing tool such as an (non) complete list of all available URLs and a (non) complete list of all users and user roles a developer or testing agent might have to manually complete the extracted information in order to receive a more precise test result. This is not at all necessary but guarantees that the targeted web application is more thoroughly tested by checking all the available URLs with each available user role. An automated web crawling component is now able to create a map of all the resources a particular user is being granted access to by the web application respectively which access controls are being enforced by the web application for a particular user on a particular resource. In order to get a complete picture of the targeted web application, the spidering process has to be done n times, depending on how many different user roles are defined in the web application. This of course implies that the spidering component is capable of acquiring the various found user roles, for example through a HTML based authentication form, and might have to be configured manually by a developer or testing agent if the information could not be extracted from various source or configuration files mentioned in Section 5.2.1. Additionally, the behaviour of the web spidering component has to be slightly modified in contrast to standard spiders which are most commonly used by black box security testing tools. The main difference between a sole black box security testing spider as seen in w3af is that discovered URLs are only tested once for common vulnerabilities such as Injection Flaws but in the here described approach it is very important to consider the used path and the enforced access control restrictions along this path to finally retrieve a discovered URL. It is utterly important to make this approach more thorough because it might be possible that there are ways to reach a specific page which requires high access privileges through a specific path that allows access to the same page with lower privileges and to make this approach more thorough all the discovered paths and resources have to be tested by the spidering component with all available user roles. Lets consider the following programmatically enforced RBAC policy: A user u with role r is allowed to access page A, B and C through the transition seen in Definition 5.1. A− B− C → →
ur ur

(5.1)

The provided white box information also contains a user b with role s which is programmatically allowed to access page C through the transition seen in Definition 5.2.

Chapter 5. Finding Privilege Escalation Vulnerabilities by using Declarative Information

| 79

A− C →

bs

(5.2)

The goal of the spidering component is to find deviations for the programatically enforced RBAC policies by trying to visit the pages A, B , C as seen in Definition 5.3. A− B →
bs bs

B− C →

bs

B− A →

bs

C− B →

bs

(C − A) →
bs

bs

(5.3)

The transition C − A which is the inverted version of the allowed transition A − C might or might → → not pose a security threat to the web application. The drawback of this approach is that it is only as precise and thorough as the spidering coverage. The coverage of the spidering component depends on how well the spider is implemented and how well the spider component interacts with the targeted web application. This includes filling out forms and interpreting dynamic scripting languages such as JavaScript. If the spidering component is not able to detect certain parts of the web application because it has to follow a specific work flow such as checking out from an e-commerce site where specific actions have to be made in a certain order to reach a specific part of the web application, then these parts respectively resources are not being tested for privilege escalation vulnerabilities. So it is important to make sure that the spidering component reaches a high coverage and one way to achieve this is by using declarative information found in the configuration files of the web application. There exist two ways to achieve a higher spidering coverage by using declarative information found in configuration files of the web application framework: • Seeding the Web Spidering Component: The spidering component can be seeded with a (non) complete list of all the available URLs known to the web application by parsing the mentioned configuration files in Section 5.2.1. These seeds reduce the issue that common black box security testing tools have, namely finding valid URLs to attack and they as well increases the coverage of the spidering component because the component is aware of valid and existing URLs and is able to query them. • Providing Test-Data: The idea is to provide the spidering component with a list of valid data which has to be entered into common HTML forms around the web application and is being accepted by the business logic of the targeted web application. This would guarantee that the automated spidering component is not blocked out of certain parts of the web application because of business logic constraints such as checking out of an e-commerce site where the spidering component would have to provide a valid credit card number in order to reach the checkout page. These kinds of information can not be automatically extracted from source or configuration files but rather have to be provided by a developer or testing agent in some kind of machine readable format such as a XML file. With these gathered information it is already possible to create a virtual model of the web application in form of a weighted directed graph whereas the weight correlates to the user role used to reach a particular resource. Tung et al. [34] already discussed this idea in their paper called A Novel Approach to Automatic Test Case Generation for Web Applications. They identified that a virtual model of a web application (a directed graph) has several dependencies such as data- and control dependencies. Data dependencies for web applications hold when a statement in a page defines variables, which are then used as a statement in another page. Based on the shared scope of variables of data dependencies, these can be classified into application-, session- or request-scope. Access control constraints, whether the user is in possession of the proper user role to access a given page, falls into

Chapter 5. Finding Privilege Escalation Vulnerabilities by using Declarative Information

| 80

the category of control dependencies. Furthermore, this model can be very helpful for the spidering process for identifying and removing cycles by using graph algorithms such as Tarjan’s Algorithm [32]. Additionally, it can be used to ensure that all available nodes and edges are being visited by the spider by using graph algorithms such as Breadth-first- or Depth-first.

5.2.3 Evaluation of the collected Data
After the spidering of the targeted web application is complete all the necessary information is available to analyse the collected data and spot privilege escalation vulnerabilities and misconfigurations in access control checks. There exist three possible ways to detect privilege escalation vulnerabilities and misconfigurations in access control checks from the collected data: • Comparing configured Access Controls with actual deployed Access Controls: Given the access control rules have been configured by a developer through the use of source code annotations or configuration files as described in Section 5.2.1 and Section 2.2.2 it becomes possible to detect discrepancies between configured access controls and actual access control checks in the deployed web application in an automated fashion. If no access control checks have been configured in the source code of the application or in the deployment descriptor it might be necessary for a developer to provide these informations manually in order to use this detection method in an automated fashion. Lets consider a configured access control constraint which specifies that only the user with an ADMINISTRATOR role is allowed to access URLs matching the following pattern: /admin/*. If the spidering component, while using a user with lower privileges, has been granted access to the following resource /admin/vulnerable.do clearly qualifies as a discrepancy between configured and actual deployed access restrictions and would be flagged as a misconfiguration in access control checks or even a privilege escalation vulnerability. • Outlier Detection: An outlier detection algorithm such as OutRank [25] which is a stochastic graph-based outlier detection framework for detecting outliers in data using random walks can be used. At the heart of the algorithm lies a Markov chain model that is built upon a graph, which assigns an outlier score to each object. By transforming the adjacency matrix of the graph into transition probabilities, they model the problem as a Markov chain process and find the dominant eigenvector of the transition probability matrix. The values in the eigenvector are then used to determine the outlierness of each object. Using such an algorithm would help finding privilege escalation vulnerabilities in an automated fashion. Consider an administration panel in a web application which should only be accessible to the user with an ADMINISTRATOR role. Due to a missing access control check a single page inside the administration panel is accessible to a user with lower privileges. By using the above mentioned OutRank algorithm it should become feasible to identify that particular resource with a missing access control check as an outlier in the administration panel. If and how good this outlier detection works with real life data would have to be evaluated first. • Visually: The collected data could be visually rehashed as a directed graph whereas nodes correspond to found URLs and edges represent transitions between URLs. By using clustering algorithms and re-colouring the collected data from the spidering component as described in [8] and [22] it would become possible for a developer or testing agent to quickly spot outliers visually.

Chapter 5. Finding Privilege Escalation Vulnerabilities by using Declarative Information

| 81

Depending on the declaratively information available to a combined white- and black-box security testing tool capable of detecting privilege escalation vulnerabilities and missing access control checks the approach can be used in a more automated fashion and this qualifies for integration into an automated security testing framework or continuous integration/testing tool.

5.3 Summary
This Chapter described the core approach of this thesis, namely detecting privilege escalation vulnerabilities or misconfigurations in access control checks through the use of declarative information in web application source- or configuration files. The approach consists of three phases: (1) the extraction of declarative information from the deployment descriptor and the web application framework’s source- or configuration files. (2) Spidering the deployed web application with an automated spidering component capable of acquiring all the user roles known to the web application. (3) Analysing the collected data for privilege escalation vulnerabilities or misconfigurations in access control checks. Phase 1: Extracting Declarative Information: The first phase extracts declarative information out of existing source- or configuration files of the targeted web application. The source code has to be available to the proposed security testing tool. This phase is tailored to the used web application framework used by the developer such as Struts 1.x, Struts 2.x, Spring, Wicket, etc. The following data is being extracted, if available: a (non) complete list of available URLs for seeding the web spidering component used in phase 2, a (non) complete list of available users and user roles, a (non) complete list of existing access constraints. A developer or testing agent might have to complete the collected data if he wants to have better test results but at least he has to provide the necessary information on how the web spidering component can acquire the existing user roles (e.g.: through a HTML based authentication form). Phase 2: Crawling the Web Application: The spidering phase consists of an automated web crawling component which spiders the deployed and targeted web application. The spidering phase is done multiple times depending on how many different users have been found and configured in phase 1. The spider tries to create a complete map of the existing access controls in place and thus follows each available path and tries to access all found resources with each available user role. Phase 3: Evaluation of the collected Data: Phase 3 analyses the collected data from phase 2 and is able to spot privilege escalation vulnerabilities or misconfigurations in access control checks in an automated fashion either by comparing configured access constraints with the actual deployed access constraints or by using an outlier detection algorithm. Additionally, if not enough information could be extracted from the web application’s source- or configuration files, the collected data is being rehashed as a directed graph and by using clustering algorithms and re-colouring of the collected data the developer or testing agent is able to visually spot outliers indicating privilege escalation vulnerabilities or misconfigurations in access control checks.

Chapter 5. Finding Privilege Escalation Vulnerabilities by using Declarative Information

| 82

6 Design
This Chapter finalises the design for the chosen approach, namely detecting privilege escalation vulnerabilities by using declarative information as described in Chapter 5. The design for the mentioned approach ought to be extensible and enough generic to be used with various web application frameworks such as Struts, Spring, etc. Additionally, this Chapter contains the design for the underlying storage solution as well as UML diagrams for the Java classes which have to be developed.

6.1 Framework
The basic architecture of the proposed security testing framework can be seen in Figure 4.1. The automated testing tool has to be configured (1) by an developer or testing agent for the specific web application. The automated security testing tool needs access to the actual source code of the web application (2) in order to extract further data which helps it to craft meaningful test cases. After all the information has been collected the actual process of finding misconfigurations and application vulnerabilities begins (3). A human readable report for the testing agent should be created. The to be developed security testing framework consists basically of four layers as seen in Figure 6.1. The layers are being described shortly (from bottom to top) in the following listing: • Meta Layer: The Meta Layer gathers the declarative information provided by a particular web application framework or developer and transforms the gathered data into a general purpose representation. This makes sure that the security testing framework stays generic enough and could be extended by providing a new parser for a particular web application framework and its configuration files. • Web Crawler: The crawler uses the collected data from the Meta Layer XML file and performs a spidering of the targeted web application. Based on the gathered data, the crawler can perform its task more from a black- or white box perspective depending on how much declarative information could be extracted from the web application’s configuration files or on how much additional information has been provided by a developer or testing agent. • Analysis: The Analysis Layer evaluates the collected data from the crawler and the Meta Layer and tries to spot misconfigurations in the way the targeted web application enforces RBAC policies. Some of the analysis can be done in an automated fashion and some will have to be done manually and visually by the developer or testing agent. • Presentation: The Presentation Layer summarises the findings visually or by creating a humanly readable report. The framework will in the end consist of three separate applications: (1) A command line tool for extracting declarative information out of the source- and configuration files of a targeted web application by using the Meta Layer XML file format. (2) A web crawler component which will use the previously created Meta Layer XML file to map the deployed web application and its access restrictions by performing several crawling attempts with varying principals configured in the Meta Layer XML file. The spidering will be more black- or white-box based depending on how much declarative information could be extracted. This might include test data for filling out HTML forms

Chapter 6. Design

| 83

Figure 6.1: Framework Layers

and usernames and passwords for various principals. (3) A graphical user interface for visually representing access restrictions and page transitions to the testing agent. The graphical user interface will combine the Analysis Layer as well as the Presentation Layer and offers the testing agent several view modes to easily spot possible privilege escalation vulnerabilities by presenting the collected data as a directed graph. These view modes include clustering of the mapped web application based on URL paths, clustering based on principals, re-colouring of single nodes based on their attributes and highlighting differences between two users by creating a diff based on their access restrictions. These three tools will provide a developer or testing agent with a framework for mapping access restrictions of a targeted web application in an automated fashion. By introducing the Meta Layer XML file this approach can be extended and enhanced for any number of web application frameworks and programming languages. This thesis focuses on web applications written in the Java programming language but can be easily extended for applications written in Ruby or PHP. Depending on how much information the developer is willing to provide in the Meta Layer XML file, the identification of possible security threats and misconfigurations can be done more automated and by reusing a previously set up Meta Layer XML file it is possible to do regression tests on the targeted web application. This is especially interesting for integrating the proposed security testing tool into a continuous integration tool such as Jenkins1 (formerly known as Hudson) or an automated security testing framework such as ASTF. Even if the developer or testing agent did not provide any information for the Analysis Layer he will be able to visually and manually verify the implemented access restrictions in the targeted web application by using the graphical user interface. The novelty of the described approach lies in the mapping and visual representation of enforced access restrictions by web applications in an automated fashion with the possibility to find privilege escalation vulnerabilities by using declarative information found in source- or configuration files. The four layers are being explained in more detail in Sections 6.2, 6.4, 6.7 and 6.8.

6.2 Meta Layer
The Meta Layer decouples web application framework specific parsing logic, which is needed for extracting declaratively extracted information, from the rest of the security testing framework. This
1

http://jenkins-ci.org/ [19/03/11]

Chapter 6. Design

| 84

decoupling keeps the security testing framework generic and extendible. The Meta Layer provides other layers (as seen in Figure 6.1) with data, such as a (non) complete list of available URLs for seeding the web crawling component described in Section 6.4, a (non) complete list of available users and user roles, a (non) complete list of existing access constraints, etc., gathered from source- or configuration files of a targeted Java web application. The gathered data is transformed by the Meta Layer into a general purpose format which is independent of the underlying web application framework the information originated from. Being more specific, the Meta Layer needs to provide interfaces for collecting information out of web application frameworks and classes for storing respectively representing the collected data. A general web application framework parser might consist of two components: a module for statically parsing the Java source code and a module for parsing framework specific XML configuration files. One way to achieve the decoupling from framework specific logic and the actual security testing tool is to define a general purpose XML structure which can be written by the framework specific parsers. Using a pre-defined XML structure to provide the Meta Layer with the collected data has several advantages as opposed to automatically loading the collected data into a set of pre-defined database tables: 1) A testing agent would be able to manually edit the generated XML file to make adjustments to the collected data if needed; 2) the data could be easily passed on to future security testing tools.

6.2.1 Meta Layer XML Structure
The following Section describes the layout of the previously mentioned general purpose XML structure which is used to present collected web application framework specific information to the Meta Layer. The XML file starts with a <web-meta> tag and the following tags such as <info> are enclosed within. The XML Document Type Definition (DTD) for the Meta Layer XML file can be found in Appendix B. Note: not all proposed XML tags in the Meta Layer XML file are needed for detecting privilege escalation vulnerabilities but are rather being added for completeness because they are easily extractable and might be useful for future tools. The minimum required fields in order for the approach to work are the: <users>, <roles> and <login> tag.

<info> Tag
The <info> tag contains some general information about the created Meta Layer XML file such as creation date and a version string of the tool which created the XML file. Additionally there exists the possibility for the developer or testing agent to add some notes. Listing 6.1 shows an example for the <info> tag.
1 2 3 4 5 6 7 8

< info > < created > Fri Jan 28 15:30:16 2011 </ created > < timestamp >1296225011 </ timestamp > < createdBy > Spring2FrameworkParser v1 .0 </ createdBy > < notes > ... </ notes > </ info >

Listing 6.1: Meta Layer XML Structure: <info> Tag

Chapter 6. Design

| 85

<classes> Tag
The <classes> tag contains a list of all the found Java classes in the source directory of the web application. A class is added to the XML structure by appending a <class> tag. The id attribute is later being used in the XML structure to make references to a particular class. The id uniquely identifies a particular class. The mentioned id attribute is being automatically generated by incrementing an integer starting by 1. Listing 6.2 shows an example of the <classes> tag.
1 2 3 4 5

< classes > < class id ="1" name =" com . app . LoginController " / > < class id ="2" name =" com . app . WelcomeController " / > ... </ classes >

Listing 6.2: Meta Layer XML Structure: <classes> Tag

<users> Tag
The <users> tag is being used to define users which have access to the targeted web application. A <user> tag can have an arbitrary number of <attribute> tags to add more values to a user object. Adding more attributes to a user object has to be done by the developer or testing agent and is not extractable from declarative information provided in source- or configuration files of the targeted web application. The id attribute is used to uniquely identify a user and the id is being used throughout the XML structure to make references to a particular user as seen in Listing 6.4. Listing 6.3 shows an example of the <users> tag. The optional <disable-form-submit> tag can be added to a <user> or <role> tag to instrument the spidering component to disable HTML form submits during the discovering of new URLs for a certain URL path. This is usually a good idea when the spidering component accesses the administration panel of a targeted web application, otherwise the spidering component might delete users or change passwords when trying to find new URLs by automatically filling out HTML forms as described in Section 6.4.5.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

< users > < user id ="1" name =" Alice " > < attribute name =" username " value =" alice11435 -6" / > < attribute name =" password " value =" wairaehoov7A " / > < attribute name =" email " value =" alice@example . com " / > ... < disable - form - submit pattern ="/ administrator /*" / > </ user > < user id ="2" name =" Bob " > < attribute name =" username " value =" bob11447 -9" / > < attribute name =" password " value =" UYa9wung7ooy " / > < attribute name =" email " value =" bob@example . com " / > ... </ user > ... </ users >

Listing 6.3: Meta Layer XML Structure: <users> Tag

Chapter 6. Design

| 86

<roles> Tag
The <roles> tag contains a list of all the found user roles in the configuration files or source code of the web application. The priority tag has to be set by the developer or testing agent, labeling the roles based on their abilities, meaning the role with the highest authority starts with a 1 and all other roles with less abilities are classified lower. This means that the administrator role usually is the role with the highest clearance and should be labeled with a priority attribute 1. This information is later being used by the web crawler component to identify parts of the web application which users with lesser privileges should not be allowed access to. The child node <user-id-ref> is used to map defined users to available user roles. Users are being defined through the <users> tag as seen in Listing 6.3. Listing 6.4 shows an example of the <roles> tag. The

<disable-form-submit> tag is described in Section 6.2.1. If the <disable-form-submit> tag is being added to a <role> tag instead of a <user> tag then
optional

the directive is being passed on to all users belonging to this particular role.
1 2 3 4 5 6 7 8 9 10 11 12

< roles > < role id ="1" name =" ADMINISTRATOR " priority ="1" > < user - id - ref >1 </ user - id - ref > < disable - form - submit pattern ="/ administrator /*" / > </ role > < role id ="2" name =" MANAGER " priority ="2" > < user - id - ref >1 </ user - id - ref > < user - id - ref >2 </ user - id - ref > ... </ role > ... </ roles >

Listing 6.4: Meta Layer XML Structure: <roles> Tag

<access-restrictions> Tag
The <access-restrictions> is an optional tag used to specify URL patterns that are being restricted and should only be accessible for a particular role or a group of roles similar to the constraints found in a deployment descriptor. The information has to be manually provided by a testing agent or developer if it could not be extracted from declarative information found in source- or configuration files. If the tag is present in the Meta Layer XML file the Analysis Layer can spot discrepancies between actual deployed access restrictions and configured access restrictions automated and more easily (as described in Section 5.2.3). Listing 6.5 shows an example of the <access-restrictions> tag.
1 2 3 4 5 6 7 8 9 10

< access - restrictions > < url pattern ="/ login . do " access =" permitAll " / > < url pattern ="/ country /*" > < role - id - ref >1 </ role - id - ref > < role - id - ref >2 </ role - id - ref > </ url > < url pattern ="/ admin /*" > < role - id - ref >1 </ role - id - ref > </ url > </ access - restrictions >

Listing 6.5: Meta Layer XML Structure: <access-restrictions> Tag

Chapter 6. Design

| 87

<login> Tag
The <login> tag has to be completely specified by the developer or testing agent and can not be extracted from declarative information. The tag is being used to describe how users are being authenticated by the targeted web application and is essential for the spidering process to acquire the configured user roles. The spidering component uses the provided information to automatically spider the web application with various users and roles. Listing 6.6 shows an example on how to configure a form based authentication scheme through the <form-based-login> tag for the following HTML form:

<form id="userLoginForm" method="POST"> <input type="text" id="inputLoginUsername" /> <input type="password" id="inputLoginPassword" /> <input type="hidden" id="x_1" /> <input type="submit" value="Sign in" /> </form>
The <form-url> tag (Listing 6.6, Line 3) points to the URL which contains the HTML login form. The <form-name> tag contains the name or the id of the form which specifies the input elements for the username and password. The <form-name> tag is used to identify the form which is needed for the login process of the web application and is important if the HTML page contains multiple HTML forms. This might be the case if the page contains a login form for users who already have an account and on the same page exists a form for user registration. The <data> tag contains the data that should be filled into the form. The data is being represented in the JavaScript Object Notation (JSON)2 format. The data can be configured in the following ways: • Static: {"<HTML Element Name or Id>":"<Value>", ...} This is the standard way of configuring data which should be filled into HTML elements. The key is the name or id of the HTML element and the second is the actual data that should be filled in: {"inputLoginUsername":"mail@example.com", ...}. • Dynamic: {"<HTML Element Name or Id>":"##<User Attribute Name>##", ...} This is the dynamic way of configuring the data which should be filled into HTML elements. The key is the name or id of the HTML element and the second is the name of a previously configured user attribute through the <user> tag as seen in Listing 6.3. This allows the developer or testing agent to use the users email as username instead of his username by configuring the data in the following way: {"inputLoginUsername":"##email##", ...}. The <submit-element> is used to specify the element that a user would have to click in order to submit the form. This does not have to be necessarily a HTML submit element but could also be an image or any other kind of HTML element. The <submit-element> can either contain an id or a name of a HTML element or the value of a HTML element provided in the HTML source through the value="..." attribute. If no <submit-element> is being specified, the login module tries to find the correct item to click by searching the HTML source code through an XPath Query for HTML elements that can be used to submit a form.
1 2 3 4

< login > < form - based - login > < form - url >/ login . do </ form - url > < form - name > userLoginForm </ form - name >
2

http://www.json.org/ [29/01/11]

Chapter 6. Design

| 88

5 6 7 8 9

< data >{" inputLoginUsername ":"## username ##" , " inputLoginPassword ":"## password ##" , " x_1 ":"5"} </ data > < submit - element > Sign in </ submit - element > </ form - based - login > </ login >

Listing 6.6: Meta Layer XML Structure: <login> Tag

<urls> Tag
The <urls> tag is used to provide a list of available URLs which have been extracted out of configuration files or the source code of the web application. These URLs can then be used by the spidering component as seeds in order to increase the coverage. The id parameter is being automatically generated by incrementing an integer starting at 1. Optionally, an URL can have an additional <class-id-ref> parameter. This additional parameter can be used to make a reference to the implementing class of that particular URL. Especially the Spring and Struts web application frameworks provide these kind of information in their Action mapping settings. Listing 6.7 shows an example of the <urls> tag.
1 2 3 4 5 6 7 8

< urls > < url id ="1" path ="/ login . do " > < class - id - ref >1 </ class - id - ref > </ url > < url id ="2" path ="/ welcome . do " / > < url id ="3" path ="/ editUser . do " / > ... </ urls >

Listing 6.7: Meta Layer XML Structure: <urls> Tag

<views> Tag
The <views> tag is optional and is used to list all the available files that contain the actual HTML, JSP etc. view generating code of a web application. Additionally it is possible to add two optional attributes called <class-id-ref> and <url-id-ref>. <class-id-ref> is used to connect a view file with the Java class who uses that particular view for displaying information to the user. The <url-id-ref> tag is used to make a connection between a view and the actual URL that returns that particular view to the users browser. Listing 6.8 shows an example of the <views> tag.
1 2 3 4 5 6 7 8 9

< views > < view path ="/ login . jsp " > < class - id - ref >1 </ class - id - ref > <url - id - ref >1 </ url - id - ref > </ view > < view path ="/ welcome . do " / > < view path ="/ editUser . do " / > ... </ views >

Listing 6.8: Meta Layer XML Structure: <views> Tag

<form-data> Tag
The <form-data> tag is also an optional tag but can be used by the developer or testing agent to provide the spidering component with valuable test data that guarantees page transitions and thus

Chapter 6. Design

| 89

increases the spidering coverage immensely as described in Section 5.2.2. Without such data the spidering component would fill randomly generated numbers, strings or email addresses into HTML forms (see Section 6.4.5). The data has to be provided in the exactly same way as described in the <logins> tag and shown in Listing 6.6. Listing 6.9 shows an example of the <form-data> tag.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

< form - data > < form > < form - url >/ selectCountry . do </ form - url > < form - name > selectCountry </ form - name > < data >{" ctry ":" United Kingdom "} </ data > < submit - element > Select </ submit - element > </ form > < form > < form - url >/ selectLanguage . do </ form - url > < form - name > selectLanguage </ form - name > < data >{" lang ":" German "} </ data > < submit - element > Select </ submit - element > </ form > ... </ form - data >

Listing 6.9: Meta Layer XML Structure: <form-data> Tag

Meta Layer API

Figure 6.2 shows a simplified UML diagram of the Meta Layer XML API. The particular classes for storing and setting the actual extracted information from the web applications source and configuration files has been omitted in Figure 6.2 for simplicity reasons. The following code snippet shows the usage of the Meta Layer API for writing a new Meta Layer XML file with the name metalayer.xml containing a <class> and <url> element:

MetaLayerWriter writer = MetaLayerFactory.getWriter(); Class class = writer.createClass(); class.setName("org.apache.struts.app.validator.CityStateZip"); Url url = writer.createUrl(); url.setPath("/exercise/welcome.do"); try { writer.saveTo(new File("metalayer.xml")); } catch (MetaLayerWriterException e) { e.printStackTrace(); }
Reading a Meta Layer XML file is as easy as writing a Meta Layer file. The following code snippet illustrates the process:

MetaLayerReader reader = MetaLayerFactory.getReader(); try { MetaLayerConfig config = reader.parse(new File("metalayer.xml")); } catch (MetaLayerReaderException e) { e.printStackTrace(); }

Chapter 6. Design

| 90

Figure 6.2: MetaLayer API

6.3 Framework Specific Parsers
With the Meta Layer XML structure in place it is easy to write parsers for specific web application frameworks that collect all the available declarative information and provide it to the actual security testing tool. This approach makes the actual security testing tool generic and it is easy to extend the approach to different Java or even non Java web application frameworks as long as the parsers produce a well formed Meta Layer XML file. Depending on the particular web application framework such a framework specific parser might consist of two components: a module that is capable of statically analysing the source code respectively the Java classes of the web application and a module that is able to parse framework specific XML configuration files. Such a framework parser then writes a valid and well formed Meta Layer XML file containing all the found declarative information. The developer or testing agent is then able to add additional information to the Meta Layer XML file such as test data (configured through the <form-data> tag as seen in Section 6.2.1) etc. before starting the actual tests by providing the Meta Layer XML file to web spidering component of the security testing tool. Figure 6.3 shows a simplified UML diagram of a framework specific parser capable of parsing web application configuration files and deployment descriptors. The concept is that every parser implements the FileListParser interface with the parse(List<File> fileList) method and then receives a list of all the files contained in the root- and sub-directories of the user provided path to the web application source and configuration files. The parser is then able to pick the files out of the provided list that it was designed to parse and add the extracted information to the MetaLayerConfig class which generates the Meta Layer XML file after all the classes implementing the FileListParser interface have been run.

6.3.1 Struts 1.x Example
The following Section describes a framework specific parser, implemented as a Java command line application, on the example of parsing source and configuration files of a Struts 1.x web application. Two command line parameters are going to be needed, namely: (1) path to the root directory of the Struts 1.x web application that contains all the source and configuration files and optionally (2) the path to the output file e.g. /home/userX/metaLayer.xml. Internally the framework specific parser

Chapter 6. Design

| 91

Figure 6.3: Framework Specific Parser UML consists of several modules capable of parsing WEB-INF/web.xml configurations and interpreting particular Struts 1.x source and configuration files such as WEB-INF/struts-config.xml. For Struts 1.x the following parser modules exist: • Deployment Descriptor Configuration Parser: This is actually not Struts 1.x specific because the web.xml deployment descriptor file is written in a standardised format as defined in the JSR-000053 Java Servlet 2.3 and JavaServer Pages 1.2 Specifications3 . From the deployment descriptor useful information such as configured user roles can be extracted and added to the Meta Layer <roles> tag. Role information can usually be found in the <security-constraint> tag embedded in the deployment descriptor as seen in Section 2.2.2. An anonymous user called ANONYMOUS is always and automatically added to the Meta Layer XML file when parsing a web application. • Struts Configuration Parser: Struts 1.x configuration files include all struts-config.xml files that can be found in sub directories of the user provided root directory of the web application. A single Struts 1.x Java web application can have multiple struts-config.xml configuration files for each sub-directory respectively sub URL. This means a Struts 1.x application which is reachable through the following URL: http://www.example.com/myApp/ has at least one struts-config.xml configuration file in the WEB-INF folder. Additionally it is common for a developer to split the Struts 1.x configuration file up into smaller chunks for easy maintainability and readability. Parts of the web application which for example can be reached by a user through the following URL: http://www.example.com/myApp/catalog would then be configured through the struts-config.xml configuration file located in the WEB-INF/catalog/ directory. A framework parsing application for Struts 1.x web applications has to consider this fact and collect the necessary information according to the configured subpaths. Struts 1.x configuration files contain the following data which can be extracted (as being described in Section 2.7): – Struts Actions: Struts Actions contain relative URLs, the class which is responsible for handling them and the name of the JSP file that contains the view respectively the HTML
3

http://jcp.org/aboutJava/communityprocess/first/jsr053/index.html [28/02/11]

Chapter 6. Design

| 92

code used for displaying the page to the user. Relative URLs mean that the user configures the URL relative to the executing context of the Struts 1.x application, e.g. a configured Action in the struts-config.xml file located in the WEB-INF/catalog folder might contain an action such as <action path="show.do" ...>. In order to actually execute the configured Action through typing an URL into a browser would be the following: http://www.example.com/myApp/catalog/show.do. These found URLs are being added to the <urls> section of the Meta Layer configuration as mentioned in Section 6.2.1. – Struts Forwards: The same rules apply for Struts Forwards as for Struts Actions. Struts Forward configurations also contain relative URLs which can be extracted and used by the web crawler as seeds. – Views: Struts Actions and Struts Forward configuration elements might contain a parameter indicating which JSP file should be loaded when the user requests a particular URL through his browser. This information is not being used by the security testing framework but might be handy for future tools. The extracted views are being added to the <views> tag in the Meta Layer XML file as described in Section 6.2.1. • Source Code Parser: The source code parser module is responsible for generating a list of all the found Struts 1.x classes with their particular package names. This information is not being used by the proposed security testing framework but might come in handy for future tools building up on the Meta Layer XML file. – Class List Generator: The class list is being generated by recursively visiting all sub folders of the initial user provided root directory of the Struts 1.x Java web application. – Package Name Extractor: The package names are being extracted from each found *.java plain source code file by using the parser provided by the Checkstyle project. Checkstyle uses a sophisticated parser to transform the Java code into an abstract syntax tree (AST). An abstract syntax tree is a tree representation of the abstract syntactic structure of source code written in a programming language. Each node of the tree denotes a construct occurring in the source code. The syntax is “abstract” in the sense that it does not represent every detail that appears in the real syntax. For instance, grouping parentheses are implicit in the tree structure and a syntactic construct such as an if-condition-then expression may be denoted by a single node with two branches. The found information is then appended to the <classes> tag in the Meta Layer XML file as seen in Section 6.2.1.
Meta Layer XML File for a Struts 1.x Application

Listing 6.10 shows a shortened version of the generated Meta Layer XML file from parsing a Struts 1.x application without any additional added information from a developer or testing agent. The scanned application is the example Struts 1.x application struts-1.3.10-apps.zip which can be downloaded from the Struts 1.x project site. The generated Meta Layer XML file contains <info>, <classes>, <users>, <urls> and <views> tags populated with the information extracted from the various source and configuration files found in the root- and sub-directories of the provided Java Struts 1.x web application folder. As mentioned before, the <classes> tag is not being used by the security framework and is merely there for completeness. An anonymous user is being added per default to the <users> tag and a testing agent would have to enhance the Meta Layer XML file with further information about existing users. The <urls> tag contains a list of all the found Struts Actions and Struts Forwards configured in the found

Chapter 6. Design

| 93

struts-config.xml files. This list is then added as starting seeds to the web spidering component
as described in Section 5.2.2 and Section 6.4.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57

<? xml version ="1.0" ? > <web - meta > < info > < created > Tue Mar 1 13:2:21 2011 </ created > < timestamp >1298980941755 </ timestamp > < createdBy > Struts 1. x Parser </ createdBy > < notes > </ notes > </ info > < classes > < class id ="1" name =" org . apache . struts . app . dispatch . ActionDispatcherExample "/ > < class id ="2" name =" org . apache . struts . app . dispatch . DispatchExampleAction "/ > < class id ="3" name =" org . apache . struts . app . dispatch . EventActionDispatcherExample "/ > < class id ="4" name =" org . apache . struts . app . dispatch . EventDispatchActionExample "/ > < class id ="5" name =" org . apache . struts . app . dispatch . LookupDispatchExampleAction "/ > < class id ="6" name =" org . apache . struts . app . dispatch . MappingDispatchExampleAction "/ > < class id ="7" name =" org . apache . struts . app . examples . CustomActionForm "/ > < class id ="8" name =" org . apache . struts . app . examples . CustomActionForward "/ > < class id ="9" name =" org . apache . struts . app . examples . CustomActionMapping "/ > < class id ="10" name =" org . apache . struts . app . examples . CustomFormBean "/ > < class id ="11" name =" org . apache . struts . app . exercise . HtmlSettersAction "/ > < class id ="12" name =" org . apache . struts . app . exercise . ImageAction "/ > < class id ="13" name =" org . apache . struts . app . exercise . SuccessAction "/ > < class id ="14" name =" org . apache . struts . app . exercise . TestBean "/ > < class id ="15" name =" org . apache . struts . app . upload . UploadAction "/ > < class id ="16" name =" org . apache . struts . app . upload . UploadForm "/ > < class id ="17" name =" org . apache . struts . app . validator . CityStateZip "/ > < class id ="18" name =" org . apache . struts . app . validator . EditTypeAction "/ > < class id ="19" name =" org . apache . struts . app . validator . LocaleAction "/ > < class id ="20" name =" org . apache . struts . app . validator . MultiRegistrationAction "/ > < class id ="21" name =" org . apache . struts . app . validator . RegistrationAction "/ > < class id ="22" name =" org . apache . struts . app . validator . RegistrationForm "/ > < class id ="23" name =" org . apache . struts . app . validator . ShowFileAction "/ > < class id ="24" name =" org . apache . struts . app . validator . TypeAction "/ >

Chapter 6. Design

| 94

58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106

< class id ="25" name =" org . apache . struts . app . validator . TypeForm "/ > < class id ="26" name =" org . apache . struts . actions . ForwardAction "/ > </ classes > < users > < user id ="1" name =" ANONYMOUS " / > </ users > < roles > </ roles > < access - restrictions > </ access - restrictions > < login > </ login > < urls > < url id ="1" path ="/ dispatch / welcome . do " / > < url id ="2" path ="/ dispatch / dispatch . do " / > < url id ="3" path ="/ dispatch / dispatch - submit . do " / > < url id ="4" path ="/ dispatch / dispatch - noparam . do " / > < url id ="5" path ="/ dispatch / dispatchLocale . do " / > < url id ="34" path ="/ exercise / welcome . do " / > < url id ="35" path ="/ exercise / bean - write -2. do " / > < url id ="36" path ="/ exercise / html - cancel . do " / > < url id ="37" path ="/ exercise / html - cancel - true . do " / > < url id ="38" path ="/ exercise / html - cancel - false . do " / > < url id ="64" path ="/ welcome . do " / > < url id ="65" path ="/ upload / upload . do " / > < url id ="66" path ="/ upload / upload - submit . do " / > < url id ="67" path ="/ validator / viewBundleExamples . do " / > < url id ="68" path ="/ validator / validateBundleExamples . do " / > < url id ="69" path ="/ validator / viewJsBundleExamples . do " / > < url id ="70" path ="/ validator / validateJsBundleExamples . do " / > < url id ="71" path ="/ validator / showStrutsConfigBundles . do " / > </ urls > < views > < view path ="/ dispatch / index . jsp " > <url - id - ref >1 </ url - id - ref > </ view > < view path ="/ dispatch / dispatch . jsp " > <url - id - ref >2 </ url - id - ref > </ view > < view path ="/ dispatch / dispatch . jsp " > <url - id - ref >3 </ url - id - ref > < class - id - ref >2 </ class - id - ref > </ view > < view path ="/ dispatch / dispatch . jsp " > <url - id - ref >4 </ url - id - ref > < class - id - ref >2 </ class - id - ref > </ view > </ views > < form - data > </ form - data > </ web - meta >

Listing 6.10: Meta Layer XML File generated from a Struts 1.x Sample Application

6.4 Web Crawler
This Section describes the open source HtmlUnit4 project and the benefits it provides for spidering web applications automatically. Additionally, the design and performance of the web spider component, which has been developed as part of a previous thesis [10] and builds upon HtmlUnit, are being
4

http://htmlunit.sourceforge.net/ [02/02/11]

Chapter 6. Design

| 95

discussed and visualised.

6.4.1 HtmlUnit Overview
HtmlUnit is a “GUI-Less browser for Java programs” and is released under the Apache License Version 2. It models HTML documents and provides an API that allows developers to invoke pages, fill out forms, click links, etc... just like one does in a “normal” browser. The JavaScript support (which is constantly being improved) is fairly good and is able to work even with quite complex AJAX libraries, simulating either the Mozilla Firefox or Microsoft Internet Explorer web browser. HtmlUnit is typically used for testing purposes although HtmlUnit is not a generic unit testing framework such as JUnit for Java. It is especially a way to simulate a browser for testing purposes and is intended to be used within another testing framework such as JUnit or TestNG5 . HtmlUnit uses the Mozilla Rhino engine6 to provide JavaScript support. Rhino is an open source implementation of JavaScript written entirely in Java. Rhino is an implementation of the core JavaScript language only and does not contain objects or methods for manipulating HTML documents. The methods for manipulating HTML documents is instead provided by HtmlUnit. Rhino contains (as of version 1.6): All the features of JavaScript 1.7, allows direct scripting of Java, a JavaScript shell for executing JavaScript scripts, a JavaScript compiler to transform JavaScript source files into Java class files and a JavaScript debugger for scripts executed with Rhino. The JavaScript language itself is standardised by Standard ECMA-262 ECMAScript7 : A general purpose, cross-platform programming language. Rhino 1.3 and greater conform to Edition 3 of the Standard. JavaScript support is essential for web spider components which want to crawl today’s web applications thoroughly. There are no absolute trends about the use of JavaScript in common web applications. Some users have scripting turned off, some browsers do not support scripting. However W3C’s browser statistics show that 95% of all browsers on the Internet have JavaScript support enabled (as of January 2008)8 . It is safe to say that there are more web applications being built using some kind of JavaScript functionality than there are applications completely avoiding the use of JavaScript. The goal of web spider components especially embedded in automated web application security related applications is to find and unveil as many HTML pages containing attack vectors as possible. Web spiders without JavaScript support such as the webSpider plugin of w3af would return an empty HTML document in case of stumbling upon the HTML page shown in Listing 6.12. This results in a reduced accuracy because pages of the targeted web application which rely on extended JavaScript functionality are never being checked for security related issues in an automated scan. By using the HtmlUnit API and its JavaScript support it is relatively easy to write a multi threaded web spider module which is able to cope with modern web applications using extended JavaScript functionality for their web applications. The next Section shows how HtmlUnit is being used in a simple scenario: to retrieve a HTML page with embedded JavaScript, fill out some HTML text input fields and in order to actually submit the form by using the HtmlUnit API to click on the submit button.

from http://htmlunit.sourceforge.net/[05/04/10] http://www.mozilla.org/rhino/ [05/04/10] 7 http://www.ecma-international.org/publications/standards/Ecma-262.htm [05.04.10] 8 http://www.w3schools.com/browsers/browsers_stats.asp [11/06/10]
5 Taken 6

Chapter 6. Design

| 96

6.4.2 Submitting a HTML Form with HtmlUnit
The API of HtmlUnit is very accommodating for filling out and submitting HTML forms. Listing 6.11 shows a sample Java method which retrieves a HTML page (the source code of the retrieved HTML page is shown in Listing 6.12) through a proxy and populates the different HTML text input fields. The received HTML page contains a login form which has been dynamically created by a JavaScript routine (see Listing 6.12, lines 5-15). HtmlUnit (as of version 2.8) is able to simulate the following browsers: Microsoft Internet Explorer versions 6, 7 and 8, Netscape and Mozilla Firefox versions 2 and 3. Selecting a specific browser version changes the HTTP user agent header which is being sent by the HtmlUnit API. Additionally some specific JavaScript instructions are being interpreted differently based on the selected browser version9 . The specific browser version is being set upon creation of the HtmlUnit WebClient object (see Listing 6.11, Line 4). Retrieving a HTML page is done by calling the WebClient.getPage("http://www...."); method as seen in Listing 6.11 on line 13. The HtmlUnit library retrieves the HTML page located at the given location and silently loads any additional resources such as Cascading Style Sheets (CSS) and JavaScript source files embedded with the <script type="text/javascript" src="remote.js"></script> directive from the remote host. HtmlUnit now uses the Mozilla Rhino engine to interpret all the downloaded JavaScript instructions and changes the HTML document structure accordingly (if necessary). The mentioned getPage() method returns a Java HTML page object which now contains the finished rendered HTML page which now can be used for further processing. The HTML page object can now be used to manipulate the contents of the retrieved HTML page. To give an impression of what functionality HtmlUnit is offering through its API, the following list contains some of the more interesting methods which are being offered by the HTML page object (as of HtmlUnit version 2.8). For a complete documentation of the API visit HtmlUnit’s project homepage: • List<HtmlAnchor> getAnchors() Returns a list of all <a href="">...</a> anchors contained in a received HTML page. • List<HtmlElement> getElementByName(String name) Returns the HTML element with the specified name. • HtmlElement getFocusedElement() Returns the element with the focus or null if no element has the focus. • List<HtmlForm> getForms() Returns a list of all the forms in a received HTML page. • List<FrameWindow> getFrames() Returns a list containing all the frames (from <frame> and <iframe> tags) in a received HTML page. Some of these functions return a HtmlElement object which is an abstract class provided by HtmlUnit. This abstract class is being used by other HtmlUnit classes to model HTML elements such as
9

http://htmlunit.sourceforge.net/apidocs/com/gargoylesoftware/htmlunit/BrowserVersion.html
[11.06.10]

Chapter 6. Design

| 97

text- or password input fields, radio buttons, checkbox buttons etc. Again a short list of interesting methods which all HtmlElement objects have in common: • <P extends Page> P click() Simulates clicking on this element, returning the page in the window that has the focus after the element has been clicked. • <P extends Page> P dblClick() Simulates double-clicking on this element, returning the page in the window that has the focus after the element has been clicked. • void focus() Sets the focus on this element. • Page mouseOver() Simulates moving the mouse over this element, returning the page that this element’s window contains after the mouse move. • void setAttribute(String attributeName, String attributeValue) Sets the value of the attribute specified by name. The example method shown in Listing 6.11 manipulates a HTML text- and a HTML password input element on lines 20-29 by calling the setText() method (which is equivalent to setAttribute("value", "myString")) and finally submits the filled out form by “clicking” on the submit button on line 32 by calling the click() method. The example in Listing 6.11 uses the names of the HTML elements in order to retrieve them from the previously retrieved HTML page. This is just one of several ways to retrieve and modify HTML elements embedded in the HTML page object. Other methods include: retrieving HTML elements by their id attribute, iterating through a list of all available HTML elements in the HTML page or by using the XML Path Language (XPath). XPath is a query language for selecting nodes from an XML document. The XPath language is based on a tree representation of a XML document and provides the ability to navigate around the tree, selecting nodes by a variety of criteria. In popular use (though not in the official specification), an XPath expression is often referred to simply as an XPath. HtmlUnit’s HtmlPage object allows the developer to retrieve HTML elements embedded in the HTML page by issuing XPath queries such as the following code snippet which retrieves all HTML text- and password input elements embedded in a HTML page object.

String xPathQuery = "//input[@type=’text’] | //input[@type=’password’]"; List<HtmlElement> nodeList = htmlPage.getByXPath(xPathQuery);
These XPath queries are extensively used in the multi threaded web spider component written for this thesis (see Section 6.4.4). The XPath queries are being used to extract new URLs pointing to pages on the targeted web application which have not been visited in an ongoing web spider run.
1 2 3 4 5 6

public void submittingForm () throws Exception { // Creates a new browser object using a proxy server // and simulating Mozilla Firefox version 3 final WebClient webClient = new WebClient ( BrowserVersion . FIREFOX_3 , " http :// myproxyserver " , 8080) ;

Chapter 6. Design

| 98

7 8 9 10

// Set proxy username and password final DefaultCredentialsProvider credentialsProvider = ( DefaultCredentialsProvider ) webClient . getCredentialsProvider () ; credentialsProvider . addProxyCredentials ( " proxyUsername " , " myProxyPassword123 " ) ; // Get the first page final HtmlPage page1 = webClient . getPage ( " http :// www . example . com / login . php " ) ; // Get the form that we are dealing with and within that form , // find the submit button and the field that we want to change . final HtmlForm form = page1 . getFormByName (" loginForm " ) ; final HtmlSubmitInput button = form . getInputByName ( " loginButton " ) ; final HtmlTextInput textFieldUsername = form . getInputByName ( " username " ) ; final HtmlPasswordInput textFieldPassword = form . getInputByName ( " password " ) ; // Change the value of the text fields textFieldUsername . setText ( " john " ); textFieldPassword . setText ( " gaephah6MueD " ) ; // Now submit the form by clicking the button and get back the second page final HtmlPage page2 = button . click () ;

11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33

}

Listing 6.11: Short HtmlUnit Example for submitting a HTML Login Form through a Proxy Server
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

< html > < head > < title > Login Form </ title > </ head > < body > < form action = " checkLogin . php " name = " loginForm " > < script type = " text / javascript " > function writeInputElement ( inputType , inputName ) { document . write ( ’ < input type = " ’ + inputType + ’" name = " ’ + inputName + ’" / > ’) ; } writeInputElement ( " text " , " username " ) ; writeInputElement ( " password " , " password " ) ; writeInputElement ( " submit " , " loginButton " ) ; </ script > </ form > </ body > </ html >

Listing 6.12: Dynamically created HTML Login Form with embedded JavaScript Code

6.4.3 HtmlUnit JavaScript Performance
The only drawback of HtmlUnit is its JavaScript execution speed. HtmlUnit is noticeably slower in processing downloaded HTML documents when its JavaScript support is enabled than other state of the art browsers like Mozilla Firefox and its competitors. In order to measure the performance of the HtmlUnit library a test case from the Mozilla Dromaeo JavaScript Performance Test Suite has been

Chapter 6. Design

| 99

used10 . The test case has been slightly modified for testing the execution speed of the HtmlUnit API. The selected test case involves modifying the HTML Document Object Model (DOM) by creating and appending several hundred new HTML elements to a retrieved page. The following functions are being used by the test case to measure the execution speed of HTML DOM modifications: • createElement(tagName) This method returns an Element object. The tagName parameter is of type String. This method can raise a DOMException object. • createTextNode(data) This method returns a Text object. The data parameter is of type String. • cloneNode(deep) This method returns a Node object. The deep parameter is of type Boolean. • document.body.appendChild(newChild) This method returns a Node object. The newChild parameter is a Node object. This method can raise a DOMException object. • document.body.insertBefore(newChild, refChild) This method returns a Node object. The newChild parameter is a Node object. The refChild parameter is a Node object. This method can raise a DOMException object. • document.body.innerHTML This variable can be used to modify the rendered content of a HTML document after it has been fully loaded inside a web browser. Figure 6.4 shows the relative runtimes in milliseconds (logarithmic scale) between Mozilla Firefox 3.6.3, Chromium 6.0.431.0 and HtmlUnit 2.7. In comparison HtmlUnit is almost 4x slower than Mozilla Firefox 3.6.3 and even 19x slower than Chromium 6.0.431.0 in doing HTML DOM modifications. Whereas the creation of new HTML elements is the most time consuming task for HtmlUnit. The test case has been run five times for every browser and the runtimes in Figure 6.4 are the arithmetic means of these times. See Appendix E for a detailed breakdown of the measured runtimes and Appendix F for the code used to measure the JavaScript performance.

6.4.4 Design of the Web Spider Component
The following Sections describe the overall design of the web spider component which has been developed as part of this thesis. The web spider uses the HtmlUnit library described in Sections 6.4.1, 6.4.2 and 6.4.3 to use its JavaScript interpreter and the ability to simulate state of the art web browsers. The main goal throughout the design and implementation of the web spider was to build it: • Scalable: As seen in Section 6.4.3 the HtmlUnit library is not the fastest when it comes to processing JavaScript instructions especially with HTML DOM modifications. In order to increase the crawling speed without disabling JavaScript support the web spider has to be developed in a multi threaded fashion. • Easily extensible: The web spider needs to be easily extensible if more functionality should be added to the code base without any major hassle.
10

https://wiki.mozilla.org/Dromaeo [11/06/10]

Chapter 6. Design

| 100

Figure 6.4: JavaScript Execution Time (Logarithmic Scale) in various Browsers for HTML DOM Modifications

• JavaScript aware: The main advantage over other web spiders is the JavaScript support of the HtmlUnit library. The full potential of the HtmlUnit API regarding JavaScript support should be used. • Comprehensive: The web spider should be crawling a web application as thoroughly as possible. • Configurable: The user should be able to alter the behaviour of the web spider through several parameters which he can set prior to starting the web spider. The configurable parameters are being described in Section 6.4.4 and Section 9.1.4. Appendix C and D contain a simplified UML 2 class diagram of the developed web spider component and its classes for extracting new URLs.
Configuration Settings

In order to alter the behaviour of the web spider the user can set various settings through command line parameters. These settings mostly influence how long the web spider will be crawling through a web application before terminating the crawling process and passing the results to other modules. Additionally, a user is able to tell the web spider which resources it either should follow or ignore. • Setting a maximum crawling level: This setting allows the user to tell the web spider how “deep” into the web application it should crawl. The first page the web spider retrieves from the web application is on level 0. Any links embedded in that first page lead to other pages which are located on level 1 and so on. So by setting a maximum crawling level n, the web spider will stop crawling any further if it reached level n. • Setting a follow regular expression: If this setting is being used then the web spider will only follow links matching the given regular expression and will ignore all other links not matching the provided expression. • Setting an ignore regular expression: If this setting is being used then the web spider will crawl every link it finds but will ignore those that match the provided expression.

Chapter 6. Design

| 101

• Setting a time limit: The user is able to set a maximum time limit for the spidering process. The web spider will stop spidering when the time limit has been exceeded. Of course the spidering process will be terminated before the time limit has been exceeded if either the web spider is not able to find any more resources on the targeted web application or the disk space is becoming low. • Staying on the same domain: Usually the web spider follows any found link embedded in a received HTML page even if the link leads to a completely different domain than the initial address of the targeted web application. For example a web application reachable under the following address http://web1.example.com has an embedded link to a completely different domain such as http://www.example2.org. This setting tells the web spider if domains differing from the initial domain of the web application should be ignored or not. • Setting an ignore HTML element regular expression: The user is able to provide a regular expression matching HTML element name or id tags. The spidering component will ignore HTML elements whose name or id matches the provided regular expression when trying to automatically fill out HTML forms. This might be useful when spidering an administration panel of a targeted web application, otherwise the spidering component might delete users or change passwords.
Program Flow

Figure 6.5 shows an UML 2 activity diagram with a birds eye view on the implemented program logic of the multi threaded, JavaScript aware web spider. The steps following the Start Spidering node are being executed in multiple threads thus speeding up the whole spidering process. A received HTML page is being processed by executing different link extraction routines. These extraction routines scan the retrieved HTML code for links pointing to previously undiscovered HTML pages of the web application. The currently implemented extraction routines include the following: • A HREF link extraction: This routine simply extracts the src attribute from common HTML <a href="...">...</a> links. • Frame extraction: This routine scans the retrieved HTML code for HTML <frame src="..."/> and <iframe src="..."/> tags and extracts the src attribute. • Mouse event extraction: Scans the retrieved HTML code for onmouseover and onmouseout tags and invokes the event with HtmlUnit’s JavaScript support. If the event triggers a page redirect to a previously undiscovered HTML page, then the page will be added to the web spider queue for further processing. • OnClick extraction: This routine scans the retrieved HTML code for onclick tags and invokes the event with HtmlUnit’s JavaScript support. If the event triggers a page redirect to a previously undiscovered HTML page, then the page will be added to the web spider queue for further processing. • OnChange extraction: This routine scans the retrieved HTML code for onchange tags and invokes the event with HtmlUnit’s JavaScript support. If the event triggers a page redirect to a previously undiscovered HTML page, then the page will be added to the web spider queue for further processing. • HTML comment extraction: Scans the retrieved HTML code for URLs between HTML comments with a sophisticated regular expression. The following example shows a common HTML comment and a embedded URL which will be detected by this routine. The identified URLs inside HTML comments are being added to the web spider queue for further processing.

Chapter 6. Design

| 102

Figure 6.5: Simplified UML 2 Activity Diagram showing the Main-Flow of the Web Spider

<!-This is a common HTML comment... and this is an embedded URL: https://web1.example.com/index.php -->
• Submitting HTML forms: If the retrieved HTML page contains one or multiple HTML forms, this routine tries to fill out the existing input fields, drop down boxes, radio buttons etc. to successfully submit the form. If the submitted form triggers a page redirect to a previously undiscovered HTML page, then the page will be added to the web spider queue for further processing. For more details on how forms are being submitted and how input fields are being populated see Section 6.4.5.

6.4.5 Smartly filling out HTML Forms
Common HTML forms are widely used in today’s web applications for passing information from the user to the web application. A form can contain input elements like text fields, checkboxes, radio-buttons, submit buttons and more. A form can also contain select lists, textarea, fieldset, legend and label elements. Of course there are other technologies such as Adobe Flash, Java Applets, JavaFX, Microsoft Silverlight etc. that can be used to create expressive, feature-rich web application user interfaces. These technologies should not be discussed here further. The web spider plugin developed for this project thesis is able to detect HTML forms embedded in retrieved HTML pages. The discovered forms are being analysed and the web spider tries to populate the form elements such as text fields, radio-buttons, checkboxes etc. with reasonable data to circumvent any business logic restrictions in a non invasive way. The main difficulty in filling out HTML forms in an automated manner is to circumvent any restrictions the business logic of the web application might impose on a particular HTML form. Lets

Chapter 6. Design

| 103

say a user has to create a new user profile in an e-commerce application. The user has to enter his name, address and phone number in order to use the e-commerce application. Most probably the user will be shown a HTML form with the necessary input fields such as a text field for entering his name, address, phone number and e-mail address and a drop down list for entering his birthday and country. The business logic of the web application probably checks the input fields and helps the user in filling out the necessary data. The business logic makes sure that the text field used for entering the users phone number only contains numeric values such as numbers ranging from 0 to 9 before accepting the data. The main goal is to guess such restrictions based on various indicators like the name of the HTML element and others if no form data is being provided through the Meta Layer XML file as described in Section 6.2.1. The routine for smartly filling out HTML forms is being implemented as a helper library. How business logic restrictions are being guessed and what the differences between those two mentioned modes are will be explained in the following sub Sections 6.4.5.
Non Invasive

Non invasive means that no hidden HTML elements are being changed and no existing values except for text fields are being overwritten. This reflects the behaviour of a normal and friendly user using the web application. The following indicators are being used to guess the imposed business logic on HTML elements embedded in a retrieved HTML form: • Buzzwords: If the name or the id of a HTML element matches a predefined buzzword such as [number, phone, telephone, mobile, zip, postal, day, month, year, hour, minute, second] then the input field will be populated with a unique random numeric value. Other buzzwords such as [mail, email] are being used to identify email address fields. Unique and random e-mail addresses will be created in the following format [a-zA-Z0-9]@[a-zA-Z0-9].com if the name or id of the HTML element matches the e-mail buzzword. • Existing value detection: If the name or the id of a HTML element does not match any predefined buzzword the value of the element is being analysed. It might be possible that the input field has been pre filled out with an example by the web application developer. Let’s say the user has to enter his e-mail address and the according HTML element is filled out with an example such as email@example.com. The routine detects the @ by using regular expressions and generates a random unique e-mail address. If the value attribute matches the following [0-9]* regular expression a unique random numeric value is being generated. If the value attribute matches the following [a-zA-Z0-9]* regular expression a unique random alphanumeric value is being generated. If none of the predefined regular expressions match, a unique random alphanumeric value is being generated per default. • Max length detection: After a unique random value has been created whatsoever the HTML element is being checked for length restrictions. HTML input fields can be limited to a specific size with the maxlength attribute. If such an attribute exists and the generated value is longer than the maximum allowed size, the random value is being reduced to not violate the maximum length restriction. • Meta Layer: If a Meta Layer XML file has been provided by a developer or testing agent as described in Section 6.2.1 the spidering component will use the pre-configured data enclosed in the <form-data> tag to populate a found HTML form. The <form-data> tag is being described in Section 6.2.1.

Chapter 6. Design

| 104

Figure 6.6: Simplified UML 2 Activity Diagram for the Non-Invasive-Smartly-Filling-Out HTML Form Module

Figure 6.6 shows an UML 2 activity diagram of the main program flow of the non invasive mode.

6.4.6 Spidering with different User Roles
For the described approach to detect privilege escalation vulnerabilities it is essential for the spidering component to spider the targeted web application with different user roles configured through <roles> and <users> tags in the Meta Layer XML file as seen in Section 6.2.1. The HTML pages and resources that can be accessed with the currently used user credentials during the spidering process are being stored as a directed graph in a graph optimised database as later described in Section 6.5.1. Spidering is done several times with each of the configured user roles and it is then the task of the Analysis Layer to detect any misconfigurations and eventual privilege escalation vulnerabilities as later described in Section 6.7. The web spider component will be using the provided information in the Meta Layer XML file to gain access to the various user roles as described in Section 6.2.1 through the use of the information configured in the <login> tag. The spidering is done in three phases whereas phase three is optional: 1. The deployed web application is being spidered with each configured user in the Meta Layer XML file. The users are being sorted on the basis of their Meta Layer XML file configured priority tag. Ideally the user with the highest configured priority is an administrator account who has access to the complete web application. It is essential that the first user that is being used to spider the targeted and deployed web application is the one who can access the most resources in order to increase the thoroughness of the approach. Following spidering phases have to rely on the previously discovered resources and try to access them with principals having lower privileges. If the spidering is done in the following order it ensures that most if not all resources can be verified whether they are accessible by principals with lower privileges. The first spidering phase starts at the command line configured target URL and additionally uses the configured URLs in the Meta Layer XML file as seeds.

Chapter 6. Design

| 105

2. The second spidering phase completes the first spidering phase by checking all the resources for accessibility that have been discovered by other users in the first spidering phase but not by the currently used principal. This further enhances the thoroughness and completeness of the approach and ensures that all resources are being checked with every configured user. Accessing such resources is not simply done by trying to access the resources directly, rather by following the hole chain from the starting URL to the actual resource which is being requested by the current principal. This might be necessary if the resource can only be accessed by following a certain flow of actions that are being enforced by the business logic of the web application such as a shopping cart checkout system where a user first has to provide his home address before he can enter his payment options and finish the checkout process. 3. The third and last spidering phase is optional and can be enabled by specifying a command line argument before the spidering process starts. The third phase considers all previously found resources and tries to access them directly and thus ignoring any kind of underlying business logic or flow that might prevent the access. This can be seen as a flavor of Forced Browsing as described here: http://www.owasp.org/index.php/Forced_browsing. Whether a certain resource is accessible or not is determined based on two factors: (1) A resource is considered not accessible by a user with certain privileges if the web server response to a sent HTTP(S) request contains an unusual HTTP status code other than 200 OK in spidering phases 2 and 3 and only if the user did not provide any regular expression through a command line parameter. HTTP status codes indicating an access denied error might include one of the following: 204 No Content, 301 Moved Permanently, 302 Moved Temporarily, 400 Bad Request, 401 Unauthorized, etc. A complete list of all available HTTP/1.1 status codes can be found in RFC 2616[15]. (2) The user has the possibility to specify a regular expression through a command line argument that matches a given string such as access denied on a custom made error HTML page by the developer. The spidering component checks the URL, title and body content of every retrieved response against the provided regular expression and if it matches the resource is being considered as inaccessible by the currently used principal.

6.4.7 Saving the Crawled Data
Figure 6.7 shows the Entity Relationship Model (ERM) used for saving discovered URLs during the spidering process. Table 6.2 describes the meaning of the particular fields in the UrlQueue and HttpLog table. The data contained in these tables are being transferred to a graph optimised database for the later analysis of the results as described in Section 6.7. The UrlQueue table is mainly used by the spidering module as a synchronised queue to keep track of discovered URLs and URLs which have been discovered but have not been processed yet. Additionally there exists a HttpLog table which simply logs all sent HTTP requests for statistic reasons and regardless if the retrieved content are binary files, CSS style sheets etc.

6.5 Working with Graphs
Before the actual Analysis Layer is being designed and described in Section 6.7 it is important to store the gathered data from the spidering process appropriately and make it accessible to the Analysis Layer. One way to achieve this is to store the spidered data as a graph on which graph algorithms can be used to make statements about possible misconfigurations and eventual security vulnerabilities. Graphs can naturally be stored in various ways, for example in a relational database. A more convenient way is to use graph optimised databases which usually offer a sophisticated API for working and querying the data stored in the graph. One such graph optimised database is Neo4j.

Chapter 6. Design

| 106

Field

id urlqueue_id level user_id url referer discovered_at fed missing method statuscode access_denied

web_request web_response

Field

id charset method url referer contenttype statuscode statusmessage sent_at user_id web_request web_response

Chapter 6. Design

Field

id name value

UrlQueue - Description Primary Key The id of the previous UrlQueue entry used to discover this URL On which level respectively how deep into the wep application the URL was found Corresponds to the user id configured in the Meta Layer XML file The discovered URL The URL of the previous UrlQueue entry used to discover this URL Timestamp marking the data and time this URL has been discovered true or false depending on whether the spidering module parsed this resource for further URLs true or false depending on whether the URL has been added as a seed or if the spidering module discovered it The used HTTP method to retrieve the HTML resource, either POST or GET The HTTP response from the server, e.g.: 200, 404 etc. Whether the received HTTP page correlates to a custom made access denied page previously configured by the user through a command line argument. Serialized Java com.gargoylesoftware.htmlunit.WebRequest object Serialized Java com.gargoylesoftware.htmlunit.WebResponse object HttpLog - Description Primary Key The used charset for the HTTP response, e.g.: UTF-8 The used HTTP method to retrieve the HTML resource, either POST or GET The URL of the resource The URL of the previous resource The content type the URL points to, e.g.: text/html, image/png etc. The HTTP response from the server, e.g.: 200, 404 etc. The message sent along the statuscode, e.g.: OK, Not Found etc. Date and time time the HTTP request has been sent Corresponds to the user id configured in the Meta Layer XML file Serialized Java com.gargoylesoftware.htmlunit.WebRequest object Serialized Java com.gargoylesoftware.htmlunit.WebResponse object HttpLogParameter - Description Corresponds with the id in the HttpLog table Name of the attribute sent with a POST request The actual data sent with the POST request Table 6.2: Description of the UrlQueue Entity Relationship Model

| 107

Figure 6.7: UrlQueue Database Table

6.5.1 Graph Databases
Graph databases are a type of data store which treat the relationship between things as equally important to the things themselves. Examples of data sets that are natural fits for graph databases: Friend links on a social network, “People who bought this also bought...” recommendation engines as seen on Amazon and the world wide web. In most types of databases, the records stored in the database are nodes and edges (relationships) are derived from a field on a node. In a SQL database, for example, you might have a table called “people” that includes a field friend_id. The field friend_id is a reference to another record in the people table. The weakness with reference fields becomes apparent as soon as one wants to do many-to-many relationships or store data about the relationship. A person can have many friends; and one might want to track the date the friendship link was created, or whether the two people are married. The solution to this in a SQL database is a join table. In the people/friends example, the join table might be called friendships. But this method has some weaknesses. One is that it can greatly increase the number of tables in the database and might make it hard to tell apart standard tables (nodes) from join tables (edges) - which makes it more difficult for new developers to comprehend the database architecture. Another problem is that object oriented databases, which work quite well for mapping node (model) tables, generally have a much harder time mapping edges. But the biggest weakness is that queries against relationship data - be it in join table or a reference link - are extremely unwieldy. In a SQL database it typically leads to recursive joins, which tend to lead to long, incomprehensible SQL statements and unpredictable performance. A graph database is designed to represent this type of information, so it models the data more naturally. It is also designed to query it such as walking the data in a convenient and performant manner. Neo4j11 is such a graph databases which has been under development for 10 years and in production for over 7 years. Neo4j is released under a dual free software/commercial license model (which basically means that it is open source under the AGPLv3 but if someone is interested in using it with closed source software, then he will have to buy a commercial license). It is a mature and robust graph database that provides: • an intuitive graph-oriented model for data representation. Instead of static and rigid tables, rows and columns, the developer can work with a flexible graph network consisting of nodes, relationships and properties.
11

http://neo4j.org/ [23/01/11]

Chapter 6. Design

| 108

• a disk-based, native storage manager completely optimised for storing graph structures for maximum performance and scalability. • massive scalability. Neo4j can handle graphs of several billion nodes/relationships/properties on a single machine and can be shared to scale out across multiple machines. • a powerful traversal framework for high-speed traversals in the node space. • can be deployed as a full server or a very slim database with a small footprint (~500KB Java JAR archive). • a simple and convenient object-oriented API. In addition, Neo4j includes the usual database features: ACID transactions, durable persistence, concurrency control, transaction recovery, high availability, etc. Of course there exist several open source and freely available graph databases (for example the WebGraph Framework [4]) but in comparison Neo4j seemed the most mature and is greatly supported by graph algorithm libraries such as Gremlin12 (see Section 6.5.2).

6.5.2 Architecture
The following Section shortly outlines the architecture used to work and process graph structures in the proposed security testing tool. The architecture makes heavy use of open source libraries which are being described in the following listing: • Gremlin: Gremlin is a domain specific programming language for graphs 13 . It can be used for graph query, analysis and manipulation. • Pipes: Pipes is a data flow framework using process graphs. A process graph is composed of Pipe vertices connected by communication edges. A Pipe implements a simple computational step that can be composed with other Pipe objects to create a larger computation. Larger computations form a process graph (i.e. Pipeline). Such graphs allow for the splitting, merging, and in general, the transformation of data from input to output 14 . • Blueprints: Blueprints is a collection of interfaces, implementations, ouplementations and test suites for the property graph data model. Blueprints is analogous to the JDBC, but for graph databases. As such, it provides a common set of interfaces to allow developers to plug-andplay their graph database backend. Moreover, software written atop Blueprints works over all Blueprints-enabled graph databases 15 . • JUNG Framework: see Section 6.8.

6.6 Saving the Crawled Data in Neo4j
The crawled data is being stored in two places: a relational database and a graph optimised database. The relational database is exclusively being used during the spidering of the targeted web application. The database acts as a synchronised queue for the spidering component and feeds new threads
https://github.com/tinkerpop/gremlin/wiki/ [26/01/11] https://github.com/tinkerpop/gremlin/wiki/ [26/01/11] 14 https://github.com/tinkerpop/pipes/wiki/ [27/01/11] 15 https://github.com/tinkerpop/blueprints/wiki/ [27/01/11]
12 13

Chapter 6. Design

| 109

with not yet processed URLs which have been found in HTML pages through the spidering process described in Section 6.4.7. How the data is being stored in the graph optimised database is shown in Figure 6.8 which shows the root node of the graph and two so called reference nodes. Reference nodes are being suggested by the Neo4j Wiki and development guide because having reference nodes for each node type makes later adding of new nodes and querying much simpler and faster. The edge relationships are being called Principals and Pages for accessing the reference nodes from the root and Principal and Page for accessing particular pages and user nodes from the reference nodes. The following code snippet shows how the pages reference node can be accessed from Neo4j:

Node pagesReferenceNode = null; pagesReferenceNode = graphDb.getReferenceNode() .getSingleRelationship(RelationshipTypes.PAGES, Direction.OUTGOING).getStartNode();
Pages are linked through Transition relationships, indicating that two particular pages are linked in the sense that Pagen +1 can be accessed through a link embedded in Pagen . Access restrictions of a particular user are being stored in the graph by using Has_Access relationships connecting particular page nodes with a user node for each stored page a user could successfully access. Additionally to the shown relationships between nodes, nodes can have an arbitrary number of attributes. These additional information might be later used to visualise certain aspects of the collected data differently. Page nodes represent actual discovered HTML pages respectively discovered URLs during the spidering process. Along side the actual node the following additional data is being stored in the graph database: • URL: The actual URL used to retrieve the resource. • Method: Whether it was a HTTP GET or POST request. • SSL: Whether the connection was secured using SSL. Principal nodes represent the configured users from the Meta Layer XML file and principal nodes are being stored in the graph with the following additional attributes: • Name: The name of the principal. • Id: The id corresponding to the id set in the Meta Layer XML file (see Section 6.2.1). • Attributes: Arbitrary number of attributes corresponding to the attributes configured in the Meta Layer XML file as described in Section 6.2.1. Neo4j has no indexing features built-in since the graph structure of the stored data eliminates many of the needs for indexes existing when using other underlying data models. Simple key based lookup of nodes is however something that is not easily done through the graph structure, instead one has to manage a lookup index manually. IndexService is a means of providing those indexing capabilities for a Neo4j graph and integrate it as tightly as possible. With an IndexService a developer can associate any number of key-value pairs to any node and do fast lookups given such key-value pairs. The Index Service is being used in the security testing framework to find stored page nodes based on their URL. This enhances the speed of inserting and linking new page nodes greatly. Adding and storing a value into the IndexService is very easy and is shown in the following code snippet:

Chapter 6. Design

| 110

Figure 6.8: Storing the crawled data in a graph optimised Database

// Adding to the Index Service Index<Node> idx = graphDb.index().forNodes("idxPages"); idx.add(pageNode, "url", pageNode.getUrl()); // Retrieving from the Index Service IndexHits<Node> pageNodeHits = idx.get("url", "http://www.example.com"); pageNode = pageNodeHits.getSingle();

6.7 Analysis Layer
The Analysis Layer is now finally able to make statements about misconfigurations and eventual privilege escalation vulnerabilities by interpreting the collected data from the spidering process that has been stored in the relational- and graph database. The following Sections describe possible ideas for making sense of the collected data and ways to point out possible security issues.

6.7.1 Comparing crawled Data with the <access-restrictions> Tag
The Meta Layer XML file contains a so called <access-restrictions> tag as being described in Section 6.2.1. If a developer or testing agent added <access-restrictions> data to the Meta Layer XML file it is relatively easy for the Analysis Layer to compare the crawled results against the provided data and report any mismatches between intended restrictions and actual restrictions found in the deployed web application. This is done by comparing the URL patterns configured in the Meta Layer XML file and their user role restrictions with the crawled data of the existing users and their role membership. The following scenarios could occur: • A user has access to a resource which is forbidden in the <access-restrictions> tag: This scenario clearly states that there is a difference between the developers intentions and the actual deployed web application and is a strong indication that there might exist either an error in the Meta Layer XML file or that a particular resource can be accessed by a user with a different user role than actually intended. This event can be flagged as a possible vulnerability, misconfiguration or security threat. • A user has no access to an allowed resource in the <access-restrictions> tag: If this event occurs it can mean several things: either the Meta Layer XML is outdated and does not correspond any more with a newer version of the deployed web application or the user has been

Chapter 6. Design

| 111

updated to be in a different role than previously configured in the Meta Layer XML file. This does not necessarily have to be a security threat and would have to be sorted out manually by a developer or testing agent. So this event can be flagged as a warning that needs manual inspection. • A resource has been found which is not mentioned in the <access-restrictions> tag: The following event also indicates that there is a misconfiguration between the Meta Layer XML file and the actual deployed web application. The resource might be accessible by all users and has just been forgotten to be added to the Meta Layer XML file by the developer. Nevertheless, this event should also be flagged as a warning that needs manual inspection by a developer or testing agent. In summary this is a really simple and straightforward approach for detecting misconfigurations, possible security threats in a targeted web application given the developer or testing agent keeps the Meta Layer XML file up to date and adds the necessary access restrictions to the <access-restrictions> tag. Shortcomings are clearly the need for specifying these restrictions once for the hole web application. Depending on how many users, roles and resources the application has, this can be a tedious task if it could not be extracted from source- or configuration files.

6.7.2 Graph Differences - Clustering by User / User Roles
Clustering algorithms can be used to partition a set of records in our case HTML resources and group them in a meaningful order to make it easier for a user to visually detect patterns and eventual misconfigurations and even possible privilege escalation vulnerabilities. Clustering is being described by Ramakrishnan as follows: The goal of a clustering algorithm is to partition a set of records into groups such that records within a group are similar to each other and records that belong to two different groups are dissimilar. Each such group is called a cluster and each record belongs to exactly one cluster although there are clustering algorithms that allow overlapping clusters, where a record could belong to several clusters [29]. The proposed clustering attribute on how HTML resources are being partitioned will be the principal accessibility in this analysis method. In comparison with the previously proposed analysis method in Section 6.7.1 the user has to manually and visually examine the data and clusters to find potential security threats whereby he can be visually supported by the Presentation Layer as described in Section 6.8. The following proposed clustering algorithm will consider each found HTML resource and partition them based on which user with a specific user role is able to access a particular resource successfully. Considering the web crawler completed the spidering process and checked the available resources R1 , R2 , R3 , ..., Rn against three configured users with distinctive user roles U1 , U2 , U3 the following Clusters C might exist: • A single cluster Cn for each configured user Un containing all the HTML resources Rn that are only accessible to user Un , e.g.: 1 = {R2 , R3 , R4 , ...} • A single cluster C0 for each HTML resource that can not be accessed by any configured user, this also includes those resources for which the web server responded with a 404 Not Found or similar status code.

Chapter 6. Design

| 112

• Multiple clusters containing those resources that could be accessed by multiple users. There might for example exist a cluster C12 containing all the URLs that users with id’s 1 and 2 could access successfully. Furthermore, clusters C123 and C1234 containing the URLs that users with id’s 1, 2, 3 and 1, 2, 3, 4 could access without receiving an access denied exception. The maximal number of clusters to be expected is dependent on how many users n have been configured in the Meta Layer XML file. The maximal number of clusters can be calculated by using the formula in Equation 6.1.
n

1+
k =1

n! (n − k )! · k !

(6.1)

The described formula in Equation 6.1 is standard and basic permutation theory [17] plus one additional cluster for the resources that could not be accessed by any user. A medium-sized web application with n = 4 users all having different user roles would result in a maximum of 16 clusters by using the formula in Equation 6.1. This is even visually inconvenient for a user to analyse and therefore it makes sense to only compare two users at a time resulting in a maximum of 4 clusters and provide the user with an artful interface in the Presentation Layer as described in 6.8. By clustering the HTML resources according to their accessibility it is relatively easy for a developer or testing agent to see to what kind of resources a particular user has access to and investigate on those resources that interest him further. Clustering the resources is certainly more convenient than looking at the complete graph consisting of all the resources and transitions between the HTML pages without any kind of colouring or ordering. Clustering based on accessibility is not the only attribute available for clustering, it is possible to cluster a web application graph based on URL paths as described in Section 6.7.3. Figure 6.9 shows a possible visual representation of the user clustering algorithm. In the example two users are being compared and thus 4 clusters are being created: a cluster containing the nodes only User 1 can access, a cluster containing all the nodes only User 2 can access, a cluster with the nodes both users can access and a fourth cluster containing all the nodes neither User 1 and 2 can access.

6.7.3 Edge Analysis - Clustering by URL Paths
Clustering based on the enclosed path of an URL is a convenient way to separate different areas or rather function domains of the web application. The only drawback of this clustering algorithm is that it only works if the targeted application uses a URL scheme to separate different areas of the web application like /admin/* for an administration area. Other web applications might use an URL scheme such as /index.php?s=<NAVIGATION> for its navigation and clustering on URL paths is then pointless because there are no paths enclosed in the URL to cluster on. Lets consider a simple e-commerce application accessible through the URL http://www.example.com/shop/ and with the following URL naming scheme: • /shop/*: This path contains a welcome page for customers and tells them something about this particular e-commerce application. • /shop/products/*: All pages in this path contain descriptions about the available products customers can buy. • /shop/products/checkout/*: Contains all the necessary pages for a customer to complete his order and pages for verifying his payment options.

Chapter 6. Design

| 113

Figure 6.9: Clustering by Users / Roles • /shop/help/*: The following path consists of all the available help pages such as FAQ’s and shipment details. • /shop/admin/*: The administration area where only users with administration rights have access to modify prices of products and similar activities. Clustering on these URL paths would result in a set of 5 clusters namely: := {x | x ε All URLs in path /shop/*}, := {x | x ε All URLs in path /shop/products/*}, 2 ..., 5 := {x | x ε All URLs in path /shop/admin/*}
1

The maximum number of expected clusters generated with this approach greatly depends on how the targeted web application is structured in regard to used URL paths. Clustering based on URL paths is a convenient way to analyse from which resources transitions can be made into specific function domains of a web application. Especially interesting for a developer or testing agent would be the cluster containing all the /admin/* resources and he would then be able to analyse from which other clusters transitions can be made into the administration area. To even enhance the visual experience it is possible to combine the ideas presented in 6.7.2 and 6.7.3 by clustering and colouring graph vertices respectively HTML resources as described in Section 6.7.4. Figure 6.10 shows a possible visual representation of the URL path clustering algorithm. For each URL path found in the collected data a new cluster is being created as being illustrated through a circle of nodes with an uniform colour. So it is especially easy to spot connections respectively transitions between function domains of the web application such as edges between a globally visible part and the administration domain.

Chapter 6. Design

| 114

Figure 6.10: Clustering by URL Paths

6.7.4 Clustering by URL Paths and Re-Colouring
This approach combines the ideas described in 6.7.2 and 6.7.3 by using a cluster algorithm to cluster the vertices first and then additionally colourise them based on their accessibility attributes. By combining both approaches a user has the advantages of seeing both clustering algorithms combined in one illustration, given it is possible to cluster based on URL paths and there exists more than one user role in the targeted web application. This allows the developer or testing agent to quickly check if a given URL path cluster such as the administration area is homogeneous which would be the case if all the contained vertices in that particular cluster have the same colour respectively are all accessible by the same user only. If the cluster is not uniform respectively the cluster contains vertices with a different colour than most other nodes this indicates that another user role can gain access to that particular HTML resource. In case of the administration area this case would indicate that there might exist a possible security threat because a user with a different user role can gain access to the administration area. This is an efficient and convenient way for a testing agent to visually and manually detect outliers rather quickly. Figure 6.11 shows an illustration of the combined visualisations of Figures 6.9 and 6.10. The clustering is done on URL paths and the nodes are being coloured based on user accessibility. Nodes having two colours or a gradient are accessible by both users. This allows a developer to easily spot outliers in the admin/* domain because the colours of the vertices are not uniform and contain vertices which are coloured with two colours which means that another user has access to these particular resources. Whether this is a security issue or not would have to be verified manually if no access restrictions as described in Section 6.7.1 have previously been configured.

6.7.5 Showing Differences by Re-Colouring
This view mode offers the user or testing agent a convenient way to quickly analyse the differences between two graphs in regard to found resources between two users. The algorithm used is similar to the diff command on Unix operating systems for comparing the differences between two files. The graph algorithm for showing differences between two users USER1 and USER2 works as follows: • If a vertex is present in the graph of USER1 and USER2 then the vertex will be coloured grey. • If a vertex is present in the graph of USER1 but not in the graph of USER2 then the vertex will be coloured red.

Chapter 6. Design

| 115

Figure 6.11: Clustering by URL Paths and Re-Colouring

Figure 6.12: Clustering by URL Paths and showing differences • If a vertex is not present in the graph of USER1 but is present in the graph of USER2 then the vertex will be coloured green. Figure 6.12 shows a sample illustration of the above described algorithm. This can be used to quickly find URLs respectively resources that are specifically tailored to a user e.g.: a user preference dialog accessible through the following URL: /user/showPreferences.do?userid=42. Which turns out to be a valid method for detecting Insecure Direct Object References as described in Section 4.4.

6.7.6 Software Testing Lifecycle
Important for the Software Development Lifecycle (SDLC) which also contains a Software Testing Lifecycle is to use the described and implemented security testing tool in regression tests. Regression testing identifies when code modifications cause previously-working functionality to regress, or fail, ultimately allowing you to catch regression errors as soon as they are introduced. Most organizations verify critical functionality once and then assume it continues to work unless they intentionally modify it. However, even routine and

Chapter 6. Design

| 116

minor code changes can have unexpected side effects that might break previously verified functionality. 16 The proposed security testing tool should allow a developer or testing agent to mark and generate access restriction rules of marked vertices as foreseen in the Meta Layer XML file as described in Section 6.2.1. This allows a developer to do regression tests over time by using previously configured access restrictions and enables him to identify misbehaviour of previously working code in the web applications access restrictions constraints. This is especially interesting for integrating the proposed security testing tool into a continuous integration tool such as Jenkins or an automated security testing framework such as ASTF to do continuous testing. More about the benefits of continuous testing can be found in Duvall’s et al. book called Continuous Integration: Improving Software Quality and Reducing Risk [14]. In order to make the security testing tool integrateable into a continuous integration tool it needs the ability to be run in an automated fashion and preferably produces a humanly or machine readable report for further analysis. This is the responsibility of the Analysis Layer and can be solved by generating a XML file from the issues found by comparing the user provided access restrictions in the Meta Layer XML file as described in Section 6.7.1 - Comparing crawled Data with the <access-restrictions> Tag. This is the only data that can be extracted by the Analysis Layer in an automated fashion without the need of a user to manually and visually analysing the data. Listing 6.13 shows a proposed XML layout for such an issue report as described above which simply contains all the found issues. The elements are really self explanatory and such a XML report might contain <error>, <warning> and <information> elements, each with a description, title and an URL pointing to the resource that caused the issue.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

<? xml version = ’1.0 ’ encoding =’ UTF -8 ’? > < rbacvisualiser > < info > < created > Sat Mar 26 13:32:53 2011 </ created > < timestamp >1301142773050 </ timestamp > </ info > < error name =" Access Violation " url =" http :// localhost :8080/ administrator / v1 . do " username =" EMPLOYEE " > EMPLOYEE has access but should have been denied ! </ error > < error name =" Access Violation " url =" http :// localhost :8080/ administrator / v1 . do " username =" MANAGER " > MANAGER has access but should have been denied ! </ error > < error name =" Access Violation " url =" http :// localhost :8080/ administrator / v1 . do " username =" CLERK " > CLERK has access but should have been denied ! </ error > </ rbacvisualiser >

Listing 6.13: Sample XML Report created from found Issues

6.8 Presentation Layer
This Section is concerned with how the gathered data from the Meta- (Section 6.2), Crawling- (Section 6.4) and most importantly from the Analysis-Layer (Section 6.7) are being visually represented to the
16

http://www.wrox.com/WileyCDA/Section/id-291252.html - Adam Kolawa, Co-Founder of Parasoft [17/03/11]

Chapter 6. Design

| 117

Figure 6.13: GUI Design Prototype

user for verification and analysis. A good resource for ideas on how to present security related data is Raffael Marty’s book Applied Security Visualization [22] as well as [8]. He suggests to use Visual Transformation techniques [22, p.157] to represent the data visually to the user and different sise, shape and colours to visually highlight attributes of the collected data. Additionally aggregation of the data plays a crucial role in reducing the complexity of the collected data and makes it possible to identify patterns. These steps have already been discussed in Sections 6.7.2, 6.7.3 and 6.7.4 where clustering algorithms are being used to aggregate the data and different colours to highlight different user roles. Because the collected data of the Web Crawling Layer is in its purest form already a directed graph it makes sense to represent the data as a directed graph to the user with the mentioned aggregation functions at his fingertips.

6.8.1 Java Swing Application
The obvious choice for writing such a visual analysis tool is a Java Swing application because all of the other tools are written in Java and already written code for accessing the crawled data can be shared between applications. Figure 6.13 shows a design prototype of a tool capable of presenting the collected data to a developer or testing agent. The GUI design is split up in three main parts: (1) a common menu bar with menu items such as File, Help etc, (2) a panel displaying the collected data from the web crawler as a directed graph and (3) a tabbed pane where various information can be dynamically displayed to the user such as a list of all the identified issues by comparing the collected data with the configurations made in the Meta Layer XML file or information about particular vertices and user accessibility. Visualising graphs is not a trivial task and a lot of available libraries fall short when the graphs get too complex and too dense. Either the tools crash with out of memory exceptions because they try to store the hole graph in memory or they become sluggish and unresponsive. After evaluating several graph libraries (including libraries without Java Bindings) such as NodeBox17 ,
17

http://nodebox.net/ [17/03/11]

Chapter 6. Design

| 118

GraphViz18 , Protovis19 and Jung20 it became clear that the JUNG framework is very well suited and enough performant to display even complex graph structures. JUNG stands for “Java Universal Network/Graph Framework” and is an open source software library that provides a common and extensible language for the modeling, analysis and visualisation of data that can be represented as a graph or network. It is written in Java, which allows JUNG-based applications to make use of the extensive built-in capabilities of the Java API, as well as those of other existing third-party Java libraries. The JUNG architecture is designed to support a variety of representations of entities and their relations, such as directed and undirected graphs, multi-modal graphs, graphs with parallel edges and hypergraphs. It provides a mechanism for annotating graphs, entities and relations with metadata. This facilitates the creation of analytic tools for complex data sets that can examine the relations between entities as well as the metadata attached to each entity and relation. JUNG also provides a visualisation framework that makes it easy to construct tools for the interactive exploration of network data. Users can use one of the layout algorithms provided or use the framework to create their own custom layouts. In addition, filtering mechanisms are provided which allow users to focus their attention, or their algorithms, on specific portions of the graph.

6.9 Summary
This Chapter outlined the ideas and design of a four piece security testing tool able to analyse web applications and their access restrictions consisting of a Meta-, Crawling-, Analysis- and PresentationLayer. The described approach is able to spot privilege escalation vulnerabilities by analysing the collected data in a more or less automated fashion depending on how much additional white box information has been provided by a developer or testing agent or how much declarative information could be extracted from source- or configuration files. A Meta Layer XML file has been described which decouples the security testing tools from a concrete programming language or web application framework. The Meta Layer XML file is being used to provide higher layers with the collected white box information. The Meta Layer XML file is being created by command line tools especially tailored for a specific web application framework and which are capable of parsing these particular configuration- and/or source code files. Hence the security testing tool is not limited to Java web applications only but can be used for web applications written in PHP, Ruby or any other programming language. The crawling component is responsible for creating a as much as detailed map of the targeted web application and its implemented access restrictions as possible. The Analysis Layer is responsible for finding privilege escalation vulnerabilities by considering the collected data and developer configured access restrictions which is being done in an automated fashion. Other analysis features include visual representation of the collected data for manual verification by a user. Several cluster- and colour-algorithms as well as combinations thereof have been proposed to help a user to spot outliers in the implemented access restrictions more quickly and visually. The user is able to create new access restriction rules which are being added to the used Meta Layer XML file from the representation of the data through the Presentation Layer. This loop allows the testing agent to do regression tests of a deployed web application whenever modifications to the application have been made. The Presentation Layer consists of a Java Swing application using the JUNG framework for displaying and transforming the collected data from the spidering process as a directed graph.

http://www.graphviz.org/ [17/03/11] http://vis.stanford.edu/protovis/ [17/03/11] 20 http://jung.sourceforge.net/ [17/03/11]
18 19

Chapter 6. Design

| 119

7 Testing
The Testing Chapter is concerned with functionality testing to verify whether the security testing tool meets the intended specifications and functional requirements laid out in the development and design specifications. The goal is to verify that it is indeed possible and feasible to detect privilege escalation vulnerabilities and misconfigurations in real life web applications by using the developed security testing tool.

7.1 Web Crawler - WIVET Results
WIVET1 is a benchmarking framework that aims to statistically analyse web link extractors. In general, web application vulnerability scanners fall into this category. These vulnerability scanners, given a URL, try to extract as many input vectors as they possibly can to increase the coverage of the attack surface. WIVET provides a good sum of input vectors to any extractor and presents the results. The WIVET project is released under the GNU General Public License Version 2.
Crawler High Scores

This Section compares the high scores scored by other commercial or free web spiders whether they have JavaScript support or not. The high scores have been taken from the WIVET homepage (see http://code.google.com/p/wivet/wiki/CurrentResults. Only w3af has been verified with SVN revision 3438. Other scores could not be verified because the mentioned products in the high score miss any versioning information. • w3af - 50% Open Source web application attack and audit framework. Version: 1.1 (from SVN server) Revision: 3438. See http://w3af.sourceforge.net/ • Acunetix - 94% Commercially available web application vulnerability scanner. Acunetix has pioneered the web application security scanning technology: Its engineers have focused on web security as early as 1997 and developed an engineering lead in web site analysis and vulnerability detection. See

http://www.acunetix.com

• Cenzic Hailstorm - 88% Commercially available web vulnerability scanner. Protect your Web applications by using Cenzic’s desktop, black box scanning solution. See http://www.cenzic.com/ • HP Webinspect - 94% Commercially available automated web application security testing and assessment tool. Get innovative assessment technology for web services and web application security and automate web application security testing and assessment. See https://www.fortify.com/

products/web_inspect.html

1

http://code.google.com/p/wivet/ [24/03/11]

Chapter 7. Testing

| 120

• IBM AppScan - 83% Commercially available desktop solution to automate web application security testing. Rational AppScan Standard Edition significantly reduces costs associated with manual vulnerability testing and helps to protect against the threat of cyber-attack by automating security analysis to detect exploitable vulnerabilities. See http://www-01.ibm.com/ software/awdtools/appscan/. • MavitunaSecurity Netsparker - 92% Commercially available web application vulnerability scanner. Netsparker can crawl, attack and identify vulnerabilities in all custom web applications regardless of the platform and the technology they are built on, just like an actual attacker. It can identify web application vulnerabilities like SQL Injection, Cross-site Scripting (XSS), Remote Code Execution and many more. See http://www.mavitunasecurity.com/netsparker/. As one can easily see, w3af performed the worst of the tested web spiders although one has to note that w3af is the only free software project amongst the tested products. For a list of other commercially or freely available web application security tools see [11]. Because of the JavaScript support provided by the web spider component, the security testing tool is able to collect considerably more pages from the WIVET benchmark framework than w3af. The web crawling component of the developed security testing tool achieved 87%. The achieved results of the commercially available products have not been verified during this thesis. The developed crawling component missed the following test cases from the WIVET benchmark: • 3_45589.php: multi-page form with a single path to final destination • 9_26dd2e.php: unattached JS function document.location • 16_2f41a.php: 302 redirection link in response body • 17_2da76.php: XHR with a busy mode page 2 • 18_1a2f3.php: heavy JS library standard form creation • 19_1f52a.php: link attached to a SWF simple button onclick event • 19_2e3a2.php: link attached to a SWF simple button parameterised onclick event Except test cases 19_1f52a.php and 19_2e3a2.php which would require some kind of Adobe Flash support it might be possible to enhance the spidering component to detect the missing test cases in a future release.

7.2 Test Environment & Application
The test environment consists of a tailored Struts 1.3.10 web application containing various test cases to test and demonstrate the abilities and limitations of the proposed security testing tool for finding misconfigurations and privilege escalation vulnerabilities in access control checks in a targeted web application. The sample web application consists of the following four user roles: ADMINISTRATOR, MANAGER, EMPLOYEE and CLERK. Each user role has his own URL path consisting of the following: /administrator/*, /manager/*, /employee/* and /clerk/*. Each path in turn consists of 10 HTML pages. Figure 7.1 shows the overall layout of the tailored web application. Clearly visible are

Chapter 7. Testing

| 121

Figure 7.1: Struts 1.3.10 Test Web Application - URL Layout

the 4 function domains containing the 10 HTML resources. Depending on which principal is being supplied in the form based authentication, reachable under /login.do, the user will be redirected to the appropriate path for his user role. Access restrictions are managed programmatically and not through features provided by the container. The tailored Struts application implements a custom Command for checking user privileges prior to displaying content to the user. In Struts application prior to 1.3.x this would have been a custom RequestProcessor as described in Section 2.7.1. Struts 1.3.x changed the behaviour of the RequestProcessor to a configurable chain of commands. Rather than subclassing a monolithic object, it is possible not just to replace commands, it is also possible to insert or remove commands, if needed, to extend or streamline the request processing gauntlet. The implemented custom command class checks if a user has previously been authenticated prior to displaying any requested content, if that is not the case the user will be redirected to the login page at /login.do. If the user has been previously authenticated the user’s request will be forwarded. Additional access restrictions are being managed programmatically inside each JSP view before displaying any HTML content. The access restrictions are being verified in every single JSP page by checking the principals stored in the user’s session. The user object in the session has been set previously through the login form action. Listing 7.1 shows a particular JSP page which is only accessible with the administrator role and access will be denied otherwise. Lines 3 and 4 in Listing 7.1 do the actual verification of the principal stored in the session. Table 7.1 shows the various access rights for each user role. The table has to be read from the top down and whether a particular user role has access to a particular HTML resource (Pages 1 - 10) is indicated by a Y if the access is being granted and a N if it is not. For example: a user incorporating the MANAGER role is able to access all of the HTML resources of users incorporating the following roles: MANAGER, EMPLOYEE and CLERK.

Chapter 7. Testing

| 122

1 2 3 4 5 6 7 8

... <% User user = ( User ) session . getAttribute (" USER ") ; if ( user . role . equalsIgnoreCase ( User . ADMINISTRATOR ) ) { % > < jsp : include page ="/ jsp / administrator / content . jsp " / > <% } else { % > < jsp : include page ="/ accessdenied . jsp " / > <% } % >

Listing 7.1: Checking Access Restrictions in JSP Pages

7.3 Struts Test Application
For testing purposes the sample application described in Section 7.2 contains an additional 11th HTML resource reachable through the following URL: /administrator/v1.do. This JSP page is deliberately missing any access restrictions checks explained in Listing 7.1. This ultimately leads to a privilege escalation vulnerability where any user that has been properly authenticated with any kind of existing user role can access that particular page in the ADMINISTRATOR restricted area. The following Sections describe how the proposed and implemented security testing tool is being handled by a common testing agent and shows how that particular vulnerability is being detected by the tool.

7.3.1 Phase 1: Parsing Struts Source-/Configuration Files
The first step involves creating the Meta Layer XML file as described in Section 6.2 by parsing the web application’s source- and configuration files. This is done by running the following command in a command shell:

# java -jar struts1parser.jar -o metalayer.xml <ROOT DIR OF WEB APPLICATION>
The automatically generated metalayer.xml file is shown in Listing 7.2. This file can now be enhanced by the developer to increase the precision of the approach. The changes made to the metalayer.xml file can be seen in Listing 7.3. The creation of the metalayer.xml file only takes a couple of seconds and contains the found classes, URLs, JSP and a standard user called ANONYMOUS.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

<? xml version ="1.0" ? > <web - meta > < info > < created > Tue Mar 22 15:19:16 2011 </ created > < timestamp >1300803556499 </ timestamp > < createdBy > Struts 1. x Parser </ createdBy > < notes > </ notes > </ info > < classes > < class id ="1" name =" com . wastf . rbacstruts1 . CheckUserCommand " > </ class > < class id ="2" name =" com . wastf . rbacstruts1 . Constants " > </ class > < class id ="3" name =" com . wastf . rbacstruts1 . LogOffAction " > </ class > < class id ="4" name =" com . wastf . rbacstruts1 . LoginAction " > </ class > < class id ="5" name =" com . wastf . rbacstruts1 . LoginForm " > </ class > < class id ="6" name =" com . wastf . rbacstruts1 . SuccessAction " > </ class > < class id ="7" name =" com . wastf . rbacstruts1 . User " > </ class > </ classes > < users > < user id ="1" name =" ANONYMOUS " > </ user >

Chapter 7. Testing

| 123

20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80

</ users > < roles > </ roles > < access - restrictions > </ access - restrictions > < login > </ login > < urls > < url id ="1" path ="/ administrator / page1 . do " > </ url > < url id ="2" path ="/ administrator / page2 . do " > </ url > < url id ="3" path ="/ administrator / page3 . do " > </ url > < url id ="4" path ="/ administrator / page4 . do " > </ url > < url id ="5" path ="/ administrator / page5 . do " > </ url > < url id ="6" path ="/ administrator / page6 . do " > </ url > < url id ="7" path ="/ administrator / page7 . do " > </ url > < url id ="8" path ="/ administrator / page8 . do " > </ url > < url id ="9" path ="/ administrator / page9 . do " > </ url > < url id ="10" path ="/ administrator / page10 . do " > </ url > < url id ="11" path ="/ manager / page1 . do " > </ url > < url id ="12" path ="/ manager / page2 . do " > </ url > < url id ="13" path ="/ manager / page3 . do " > </ url > < url id ="14" path ="/ manager / page4 . do " > </ url > < url id ="15" path ="/ manager / page5 . do " > </ url > < url id ="16" path ="/ manager / page6 . do " > </ url > < url id ="17" path ="/ manager / page7 . do " > </ url > < url id ="18" path ="/ manager / page8 . do " > </ url > < url id ="19" path ="/ manager / page9 . do " > </ url > < url id ="20" path ="/ manager / page10 . do " > </ url > < url id ="21" path ="/ clerk / page1 . do " > </ url > < url id ="22" path ="/ clerk / page2 . do " > </ url > < url id ="23" path ="/ clerk / page3 . do " > </ url > < url id ="24" path ="/ clerk / page4 . do " > </ url > < url id ="25" path ="/ clerk / page5 . do " > </ url > < url id ="26" path ="/ clerk / page6 . do " > </ url > < url id ="27" path ="/ clerk / page7 . do " > </ url > < url id ="28" path ="/ clerk / page8 . do " > </ url > < url id ="29" path ="/ clerk / page9 . do " > </ url > < url id ="30" path ="/ clerk / page10 . do " > </ url > < url id ="31" path ="/ employee / page1 . do " > </ url > < url id ="32" path ="/ employee / page2 . do " > </ url > < url id ="33" path ="/ employee / page3 . do " > </ url > < url id ="34" path ="/ employee / page4 . do " > </ url > < url id ="35" path ="/ employee / page5 . do " > </ url > < url id ="36" path ="/ employee / page6 . do " > </ url > < url id ="37" path ="/ employee / page7 . do " > </ url > < url id ="38" path ="/ employee / page8 . do " > </ url > < url id ="39" path ="/ employee / page9 . do " > </ url > < url id ="40" path ="/ employee / page10 . do " > </ url > < url id ="41" path ="/ accessdenied . do " > </ url > < url id ="42" path ="/ login . do " > </ url > < url id ="43" path ="/ logoff . do " > </ url > </ urls > < views > < view path ="/ jsp / administrator /1. jsp " > <url - id - ref >1 </ url - id - ref > < class - id - ref >6 </ class - id - ref > </ view > < view path ="/ jsp / administrator /2. jsp " > <url - id - ref >2 </ url - id - ref > < class - id - ref >6 </ class - id - ref > </ view > ...

Chapter 7. Testing

| 124

81 82 83 84 85 86 87 88 89 90 91 92

< view path ="/ jsp / manager /1. jsp " > <url - id - ref >11 </ url - id - ref > < class - id - ref >6 </ class - id - ref > </ view > < view path ="/ jsp / manager /2. jsp " > <url - id - ref >12 </ url - id - ref > < class - id - ref >6 </ class - id - ref > </ view > ... </ views > < form - data > </ form - data > </ web - meta >

Listing 7.2: Initial metalayer.xml File for the Test Application A testing agent now enhances the Meta Layer XML file to his needs and adds the programatically checked user roles, users and eventually desired respectively intended access restrictions. The changes made to the originally created file can be seen in Listing 7.3.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

< users > < user id ="1" name =" ANONYMOUS " > </ user > < user id ="2" name =" ADMINISTRATOR " > < attribute name =" username " value =" adminUser " / > < attribute name =" password " value =" password " / > </ user > < user id ="3" name =" MANAGER " > < attribute name =" username " value =" managerUser " / > < attribute name =" password " value =" password " / > </ user > < user id ="4" name =" EMPLOYEE " > < attribute name =" username " value =" employeeUser " / > < attribute name =" password " value =" password " / > </ user > < user id ="5" name =" CLERK " > < attribute name =" username " value =" clerkUser " / > < attribute name =" password " value =" password " / > </ user > </ users > < roles > </ roles > < access - restrictions > </ access - restrictions > < login > </ login > < roles > < role id ="1" name =" ADMINISTRATOR " priority ="1" > < user - id - ref >2 </ user - id - ref > </ role > < role id ="2" name =" MANAGER " priority ="2" > < user - id - ref >3 </ user - id - ref > </ role > < role id ="3" name =" EMPLOYEE " priority ="3" > < user - id - ref >4 </ user - id - ref > </ role > < role id ="4" name =" CLERK " priority ="4" > < user - id - ref >5 </ user - id - ref > </ role > </ roles > < access - restrictions > < url pattern ="/ login . do " access =" permitAll "/ > < url pattern ="/ accessdenied . do " access =" permitAll "/ > < url pattern ="/ administrator /*" > < role - id - ref >1 </ role - id - ref >

Chapter 7. Testing

| 125

42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70

+ + + + + + + + + + + + + + + + + + + + + + + + + + +

</ url > < url pattern ="/ manager /*" > < role - id - ref >1 </ role - id - ref > < role - id - ref >2 </ role - id - ref > </ url > < url pattern ="/ employee /*" > < role - id - ref >1 </ role - id - ref > < role - id - ref >2 </ role - id - ref > < role - id - ref >3 </ role - id - ref > </ url > < url pattern ="/ clerk /*" > < role - id - ref >1 </ role - id - ref > < role - id - ref >2 </ role - id - ref > < role - id - ref >3 </ role - id - ref > < role - id - ref >4 </ role - id - ref > </ url > </ access - restrictions > < login > < form - based - login > < form - url >/ login . jsp </ form - url > < form - name > loginForm </ form - name > < data > {" username ":"## username ##" , " password ":"## password ##"} </ data > < submit - element > Sign in </ submit - element > </ form - based - login > </ login > < urls > < url id ="1" path ="/ administrator / page1 . do " > </ url >

Listing 7.3: Additional Information added to the metalayer.xml File (displayed as a diff -up FILE1 FILE2 output) In this case the testing agent added several pieces of information to the Meta Layer file including the following: • <roles>: The available roles in the web application. Because these roles are being programmatically verified in the source code of the web application these different roles do not show up in a standardised configuration file such as a deployment descriptor and thus have to be added by the testing agent (Listing 7.3, Lines 24-37). The <roles> tag is being described in Section 6.2.1. • <users>: The available users incorporating the previously defined roles. Again, because the users are being programmatically verified and are not being declared using a deployment descriptor these have to be added manually by a testing agent (Listing 7.3, Lines 4-19). Ideally, the testing agent configures a distinct user for each defined role. The <users> tag is being described in Section 6.2.1. • <access-restrictions>: Access restrictions help the Analysis Layer to find vulnerabilities and misconfigurations in an automated fashion. In this case the testing agent defined the various privilege restrictions based on the available URL paths of the web application (Listing 7.3, Lines 38-59). The information provided here correlates with Table 7.1 and Section 7.2. The <access-restrictions> tag is being described in Section 6.2.1. • <login>: Because the web application can not be spidered before a user is not properly authenticated the testing agent defined the HTML form used by the web application to authenticate users (Listing 7.3, Lines 60-67). In this case the additional attributes defined in the <user>

Chapter 7. Testing

| 126

elements are being used to populate the input fields of the HTML login form. The <login> tag is being described in Section 6.2.1. With this enhanced Meta Layer XML file in Listing 7.3 the next step is to use the web crawling component and map the deployed web application’s access restrictions as they have actually been implemented in the application. Important to note is that this first step has to be done only once and the enhanced Meta Layer XML file can be kept as long as the web application does not significantly change.

7.3.2 Phase 2: Crawling the deployed Web Application
With the prepared Meta Layer XML file the crawling of the deployed web application can be started. The crawling component uses all the available users to map all the existing resource transitions and privilege restrictions. The crawling component has to be additionally configured over the command line interface by the testing agent.

# java -jar rbacspider.jar \ --access-denied-page "access denied" \ --forceful-browsing \ --ignore-regex "logoff.do" \ metalayer.xml \ http://localhost:8080/
The −−access-denied-page property contains a regular expression used by the web crawler to identify if a user has been denied access to a requested resource. This is especially important if the web application uses custom designed error pages as it is the case in the sample Struts 1.3.10 application described previously in Section 7.2. If the web application would send standard failing HTTP status codes when access to a particular resource is not being granted then this parameter could be omitted. The −−forceful-browsing flag adds another loop to the spidering process as described in Section 6.4.6 in order to request every single found URL without following the previously collected transitions. Spidering the test application with different user roles as described in Section 6.4.6 took roughly 3 hours. This is because the test application shows the applications site map2 on every HTML resource and the spidering component tries to reach each page from all available angles respectively paths. This is intentional and should give an idea how performant the spidering and the visualisation of the data is if there are a lot of page transitions present. After the spidering is complete the following archive 2011-03-24-localhost.rbacspider has been generated which then can be opened in the Presentation Layer for further analysis.

7.3.3 Phase 3: Analysing the Results
The final step is to analyse the collected data from the spidering phase in the graphical user interface provided. Starting the application is done by double clicking the rbacvisualiser.jar file or by entering the following command into a terminal:
2A

site map is a list of pages of a web site accessible to crawlers or users. It can be either a document in any form used as a planning tool for web design or a web page that lists the pages on a web site, typically organised in hierarchical fashion. This helps visitors and search engine bots find pages on the site. - http://en.wikipedia.org/wiki/Site_map [25/03/11]

Chapter 7. Testing

| 127

Figure 7.2: Initial view of the collected Data from the Struts Test Application

# java -jar rbacvisualiser.jar
Figure 7.2 shows the initial view of the visualisation tool presenting the collected data after the user opened the previously generated archive. The default view is based on the URL path clustering algorithm described in Section 6.7.3. Of course, this view is not all that informative in the beginning but interesting for a testing agent or developer is the Problems tab. The Problems tab contains the findings based on the user provided <access-restrictions> information and the actual findings of the spidering component based on the Analysis Layer described in Section 6.7.1. Figure 7.3 shows the Problems tab and the particular HTML resource vulnerable to a privilege escalation vulnerability as has been intended (see Section 7.3). The Problems tab is the first source a testing agent consults in finding problems in the implementation of the targeted web application. Afterwards he is able to analyse the data further visually by hiding edges or vertices, clustering the data based on users or by re-colouring the data. Additionally he is able to generate edit or add new rules to the <access-restrictions> section of the Meta Layer XML file which then can be used in future scans of the web application. If the testing agent would not have added the additional information in the

<access-restrictions> section of the Meta Layer XML file he would still have been able to
detect the privilege escalation vulnerability in the targeted web application by re-colouring the collected data based on users as described in Section 6.7.4. Figure 7.4 shows the collected data after it has been re-coloured by using the ADMINISTRATOR and CLERK user. For the sake of clarity all of the main edges have been hidden and now one can clearly see that the one vertex in the /administrator/* cluster is coloured differently than the rest and this indicates that that particular vertex, namely /administrator/v1.do, is accessible by both the ADMINISTRATOR and CLERK user which is our intended privilege escalation vulnerability. This test case has shown that the Presentation Layer is enough performant to handle even web applications with a lot of vertices and transitions respectively edges between them and that the design and idea of a security testing framework for detecting misconfigurations and vulnerabilities in non-

Chapter 7. Testing

| 128

Figure 7.3: Automatically detected Access Restriction Violations in the Struts Test Application

Figure 7.4: Re-Colouring of the Data based on Users

container managed access restriction checks in an automated fashion actually works.

7.4 WordPress 3.1
The following test case tries to show that the developed security testing framework is also capable of scanning rather large real life web applications successfully. Finding misconfigurations or privilege escalation vulnerabilities in a real life community tested web application is rather unlikely but it might be interesting to see whether the security testing framework reports false positives or not. This test case involves the freely available blogging software WordPress3 . WordPress started as a blogging system, but has evolved to be used as full content management system and so much more through the thousands of plugins, widgets and themes. WordPress has been chosen for this test case to show that the developed approach in this thesis can be used with real life web applications and that the approach is not limited to web applications written in Java because WordPress is entirely written in PHP. The test setup involves WordPress 3.1, PHP Version 5.3.3-1ubuntu9.3, Apache/2.2.16 (Ubuntu) web server and a MySQL Ver 14.14 Distrib 5.1.49 database. Test data is being provided by importing the XML file from http://svn.automattic.com/wpcom-themes/demo/test-data.2011-0117.xml.

3

http://wordpress.org/ [31/03/11]

Chapter 7. Testing

| 129

Administrator

Manager

Employee

Clerk

1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10

1 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y

2 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y

3 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y

Administrator 4 5 6 7 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y

8 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y

9 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y

10 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y

1 N N N N N N N N N N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y

2 N N N N N N N N N N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y

3 N N N N N N N N N N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y

4 N N N N N N N N N N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y

Manager 5 6 N N N N N N N N N N N N N N N N N N N N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y

7 N N N N N N N N N N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y

8 N N N N N N N N N N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y

9 N N N N N N N N N N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y

10 N N N N N N N N N N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y

1 N N N N N N N N N N N N N N N N N N N N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y

2 N N N N N N N N N N N N N N N N N N N N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y

3 N N N N N N N N N N N N N N N N N N N N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y

4 N N N N N N N N N N N N N N N N N N N N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y

Employee 5 6 7 N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y

8 N N N N N N N N N N N N N N N N N N N N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y

9 N N N N N N N N N N N N N N N N N N N N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y

10 N N N N N N N N N N N N N N N N N N N N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y

1 N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N Y Y Y Y Y Y Y Y Y Y

2 N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N Y Y Y Y Y Y Y Y Y Y

3 N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N Y Y Y Y Y Y Y Y Y Y

4 N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N Y Y Y Y Y Y Y Y Y Y

Clerk 5 6 N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y

7 N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N Y Y Y Y Y Y Y Y Y Y

8 N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N Y Y Y Y Y Y Y Y Y Y

9 N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N Y Y Y Y Y Y Y Y Y Y

10 N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N Y Y Y Y Y Y Y Y Y Y

Table 7.1: Access Rights of the Sample Struts 1.3.10 Web Application A vulnerable PHP page has been added to the /wp-admin/* folder which holds the administration backend and is only accessible to a user with administrator privileges. This vulnerable PHP page called /wp-admin/vulnerable.php is missing any access control checks and is accessible to all available users. This is to demonstrate that the described security testing tool is indeed capable of detecting privilege escalation vulnerabilities in non Java applications properly and that the tool works reliably.

7.4.1 Phase 1: Preparing the Meta Layer XML File
Listing 7.4 shows the used Meta Layer XML file. The difference between this Meta Layer XML file and the one previously used during the Struts 1.3.10 test application described in Section 7.3.1 is that several parts are missing such as: the <classes>, <urls> and <views> tag elements. The <classes> and <views> tags are merely optional and do not affect the thoroughness of the approach as described in Section 6.2.1, the missing <urls> tag on the other hand does. The approach does still work even if no URLs are being used to seed the crawling component as described in Section 6.2.1 but it is very likely that not all URLs are properly being found and tested by the web crawling component. Nonetheless, it is possible to conduct a security analysis of the targeted web application and it is possible to detect missing access control checks in the web application as shown later on in Section 7.4.3.
1 2 3 4 5 6

<? xml version ="1.0" ? > <web - meta > < info > < created > Tue Apr 12 11:28:43 2011 </ created > < timestamp >1302604104 </ timestamp > < createdBy > Manual </ createdBy >

Chapter 7. Testing

| 130

7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44

< notes > </ notes > </ info > < classes > </ classes > < users > < user id ="1" name =" ANONYMOUS " > </ user > < user id ="2" name =" ADMINISTRATOR "> < attribute name =" username " value =" admin " / > < attribute name =" password " value =" admin " / > </ user > </ users > < roles > < role id ="1" name =" ADMINISTRATOR " priority ="1" > < user - id - ref >2 </ user - id - ref > < disable - form - submit pattern ="/ wordpress / wp - admin /*" / > </ role > < role id ="2" name =" ANONYMOUS " priority ="2" > < user - id - ref >1 </ user - id - ref > </ role > </ roles > < access - restrictions > < url pattern ="/ wordpress / wp - admin /*" > < role - id - ref >1 </ role - id - ref > </ url > </ access - restrictions > < login > < form - based - login > < form - url >/ wordpress / wp - login . php </ form - url > < form - name > loginform </ form - name > < data > {" user_login ":"## username ##" , " user_pass ":"## password ##"} </ data > < submit - element > Log In </ submit - element > </ form - based - login > </ login > < urls > </ urls > < views > </ views > < form - data > </ form - data > </ web - meta >

Listing 7.4: metalayer.xml File for a Standard WordPress 3.1 Installation As intended by the developers of WordPress, the Meta Layer XML file contains an <access-restrictions> element stating that only the administrator user should have access to the /wp-admin/* folder. This should later on detect the vulnerable.php file that has been added to the WordPress administration folder in the Analysis Layer. The <login> tag describes the HTML form used by WordPress to authenticate users and which gives them access to the administration panel of WordPress. The name of the form and the used HTML input elements have to be manually extracted by a testing agent by looking at the HTML source code of the /wordpress/wp-login.php file. Listing 7.5 shows the HTML source code of the /wordpress/wp-login.php file with the embedded HTML form.
1 2 3 4 5 6 7

... < form name =" loginform " id =" loginform " action =" http :// localhost / wordpress / wp - login . php " method =" post " > <p > < label > Username < br / > < input type =" text " name =" log " id =" user_login " class =" input " value ="" size ="20" tabindex ="10" / > </ label >

Chapter 7. Testing

| 131

8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

</p > <p > < label > Password < br / > < input type =" password " name =" pwd " id =" user_pass " class =" input " value ="" size ="20" tabindex ="20" / > </ label > </p > <p class =" forgetmenot " > < label > < input name =" rememberme " type =" checkbox " id =" rememberme " value =" forever " tabindex ="90" / > Remember Me </ label > </p > <p class =" submit " > < input type =" submit " name =" wp - submit " id =" wp - submit " class =" button - primary " value =" Log In " tabindex ="100" / > < input type =" hidden " name =" redirect_to " value =" http :// localhost / wordpress / wp - admin /" / > < input type =" hidden " name =" testcookie " value ="1" / > </p > </ form > ...

Listing 7.5: /wordpress/wp-login.php HTML Source Code containing the HTML based Authentication Form of WordPress

7.4.2 Phase 2: Crawling the deployed Web Application
With the prepared Meta Layer XML file (see Listing 7.4) the crawling of the deployed web application can be started. The crawling component uses all the available users to map all the existing resource transitions and privilege restrictions. The crawling component has to be additionally configured over the command line interface by the testing agent. The following commands have been used to crawl the installed WordPress application:

# java -jar rbacspider.jar \ --ignore-regex ".*logout.*" \ --access-denied-page ".*Lost your password?.*" \ --ignore-htmlelements-regex ".*search.*" \ --forceful-browsing \ --max-threads 10 \ metalayer.xml http://localhost/wordpress/
The −−ignore-regex provided regular expression prevents the spidering component to unintentionally click on the sign out button and thus terminating the users session. The −−access-denied-page regular expression matches the /wordpress/wp-login.php file which is shown to a user with lower privileges when trying to access resources in the /wordpress/wp-admin folder. This is default WordPress behaviour and by providing this regular expression the spidering component is aware when a particular user has been denied access to a requested resource. The −−ignore-htmlelements-regex provided regular expression prevents the spidering component to fill out any search fields which are provided by WordPress to search for blog entries. This prevents the spidering component to enter an endless loop which would only be terminated by exceeding the −−max-level spidering constraint which defaults to 25. The endless loop is caused because whenever the spidering component automatically fills out such a search form on a WordPress page as described in Section 6.4.5 the returned results are interpreted as a new resource respectively path in the targeted web application. The therein found URLs are then again crawled

Chapter 7. Testing

| 132

for possible new paths and URLs. The particular problem in this test case is that such a search field is present on every WordPress page and thus leads to an endless amount of transitions between pages. To prevent this from happening a −−max-level constraint could have been lowered from the default value of 25 to a smaller number to leave such a loop more quickly or by adding a −−ignore-htmlelements-regex regular expression to ignore such search fields altogether. The −−forceful-browsing flag adds another loop to the spidering process as described in Section 6.4.6 in order to request every single found URL without following the previously collected transitions. The −−max-threads 10 flag tells the spidering component to use 10 spidering threads instead of 5 which is the default value. Depending on the available random access memory this value can even be increased further and thus speeding up the spidering process. Nonetheless, the default value of 5 threads is a good mix between performance and speed. Spidering the installed WordPress web application with two different user roles as described in Section 6.4.6 has been manually terminated after each user spidered the application for roughly an hour. After the spidering is complete the following archive 2011-04-01-localhost.rbacspider has been generated which then can be opened in the Presentation Layer for further analysis.

7.4.3 Phase 3: Analysing the Results
The final step is to analyse the collected data from the spidering phase in the graphical user interface provided. Starting the application is done by double clicking the rbacvisualiser.jar file or by entering the following command into a terminal:

# java -jar rbacvisualiser.jar
Figure 7.5 shows the clustered and re-coloured graph of the collected data as described in Section 6.7.4. The Problems tab contains the findings based on the user provided <access-restrictions> information and the actual findings of the spidering component based on the Analysis Layer described in Section 6.7.1 and Section 7.4.1. Figure 7.5 additionally shows the Problems tab and the particular HTML resource /wp-admin/vulnerable.php that has been added with missing access control checks. The detection of the intentionally added privilege escalation vulnerability has been successful and the spidering component as well as the Analysis Layer correctly found the vulnerability. By clustering and re-arranging the collected data the issue is also clearly visible on the right. The violation is clearly visible in the cluster on the right which should only be accessible by the administrator user (coloured red) but one node is also accessible by a user with fewer privileges (gradient coloured). This test case showed that the designed and developed approach for finding possible privilege escalation vulnerabilities works even with large and real life applications visually by inspecting the collected data as well as automated given the <access-restrictions> rules exist in the Meta Layer XML file. However, one issue noticed during the execution of the test case showed that the Analysis Layer might generate false positives in certain circumstances. These circumstances are explained in more detail in Section 7.6.

Chapter 7. Testing

| 133

Figure 7.5: Automatically detected Access Restriction Violations in the WordPress Test Application

7.5 PrivaSphere Secure Messaging Service
PrivaSphere’s Secure Messaging Service4 provides a reliable and secure email communication over the Internet. It provides advanced features such as digitally stamped receipts (certified) and can be used for sending offers, delivery of client statements (financial, personal) and delivery of medical data. The Secure Messaging Service provides innovative secure and authenticated Internet messaging technologies and service to corporations and individual users. The following test case has been chosen in order to verify that the developed approach reliably works with a rather large real life and productive web application. The following four users respectively user roles have been set up and have been available during the spidering and analysing of the enforced access control constraints in the Secure Messaging Service web application: a normal user, a first-level helpdesk user, a signature service administrator and a super user. Figure 7.6 shows the email inbox for a standard user after logging in.

7.5.1 Phase 1: Preparing the Meta Layer XML File
In the first phase the Meta Layer XML is being set up and configured. Because the source code of the application has not been available, the Meta Layer XML has been set up manually and eventual privilege escalation vulnerabilities and misconfigurations in enforced access control constraints will have to be identified visually by a testing agent. The approach works nonetheless even if no declarative information has been extracted from the source- and configuration files of the targeted
4

https://www.privasphere.com/ [12/04/11]

Chapter 7. Testing

| 134

Figure 7.6: Standard User Secure Messaging Service Email Inbox web application but the results might not be as precise and thorough when using available declarative information and eventual vulnerabilities have to be visually and manually verified by a developer. Listing 7.7 shows the prepared Meta Layer XML file containing the four mentioned user roles and the HTML based authentication form which is being used by the spidering component to acquire the different user roles. Listing 7.6 shows the simplified HTML source code of the HTML based authentication form from which the <login> tag has been configured in the Meta Layer XML file in Listing 7.7.
1 2 3 4 5 6

< form id =" loginForm " name =" loginForm " method =" post " action ="../ login ." > < input type =" hidden " name =" routeTo " value =" index " / > < input name =" login " id =" login " type =" text " / > < input name =" password " type =" password " / > < input type =" image " name =" submit " value =" submit " src ="...." / > </p > </ form >

Listing 7.6: Secure Messaging Service HTML based Authentication Form
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

<? xml version ="1.0" ? > <web - meta > < info > < created > Tue Apr 12 11:28:43 2011 </ created > < timestamp >1302604104 </ timestamp > < createdBy > Manual </ createdBy > < notes > </ notes > </ info > < classes > </ classes > < users > < user id ="1" name =" ANONYMOUS " > </ user > < user id ="2" name =" NORMAL_USER " > < attribute name =" username " value =" M8R - p4ov6i@mailinator . com " / > < attribute name =" password " value =" si9eiBahz1 " / > </ user > < user id ="3" name =" HELPDESK_USER " > < attribute name =" username " value =" M8R -6 tiadc@mailinator . com " / > < attribute name =" password " value =" Ahbeineip5 " / > </ user > < user id ="4" name =" SIGNATURE_USER " >

Chapter 7. Testing

| 135

21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61

< attribute name =" username " value =" M8R - b6wfch@mailinator . com " / > < attribute name =" password " value =" Amaesa3eu5 " / > </ user > < user id ="5" name =" SUPER_USER "> < attribute name =" username " value =" M8R - c19ob11@mailinator . com " / > < attribute name =" password " value =" shoow4Chae " / > </ user > </ users > < roles > < role id ="1" name =" SUPER " priority ="1" > < user - id - ref >5 </ user - id - ref > < disable - form - submit pattern =".*" / > </ role > < role id ="1" name =" SIGNATURE " priority ="2" > < user - id - ref >4 </ user - id - ref > </ role > < role id ="1" name =" HELPDESK " priority ="3" > < user - id - ref >3 </ user - id - ref > </ role > < role id ="1" name =" NORMAL " priority ="4" > < user - id - ref >2 </ user - id - ref > </ role > < role id ="1" name =" ANONYMOUS " priority ="4" > < user - id - ref >1 </ user - id - ref > </ role > </ roles > < access - restrictions > </ access - restrictions > < login > < form - based - login > < form - url >/ hp / index . php </ form - url > < form - name > loginForm </ form - name > < data > {" login ":"## username ##" , " password ":"## password ##"} </ data > < submit - element > submit </ submit - element > </ form - based - login > </ login > < urls > </ urls > < views > </ views > < form - data > </ form - data > </ web - meta >

Listing 7.7: metalayer.xml File for the Secure Messaging Service

7.5.2 Phase 2: Crawling the deployed Web Application
With the prepared Meta Layer XML file (see Listing 7.7) the crawling of the deployed web application can be started. The crawling component uses all the available users to map all the existing resource transitions and privilege restrictions. The crawling component has to be additionally configured over the command line interface by the testing agent. The command line arguments used to run the crawling component can be seen in Listing 7.8.
1 2

3

4

# java - jar rbacspider . jar \ -- ignore - regex ".* logoff . d *|.* hp .*|.* delete .*|.* flush .*|.* reload .*|.* remove .*|.* close .*|.* forgot .*|.* reg1 .*|.* a ? clientCert .*|.* index .*" \ -- ignore - htmlelements - regex ".* search .*|.* logoff .*|.* save .*|.* update .*|.* remove .*|.* delete .*|.* flush .*" \ -- sessionId - regex " jsessionid " \

Chapter 7. Testing

| 136

5 6

metalayer . xml \ https :// www - dev . privasphere . com \

Listing 7.8: Crawling Command Line Arguments for the Secure Messaging Service The −−ignore-regex provided regular expression prevents the spidering component to unintentionally click on the logoff button and thus terminating the users session. Additionally, commands such as deleting email messages and removing contacts from the address book are being disabled. The −−ignore-htmlelements-regex provided regular expression prevents the spidering component to fill out any search fields which are provided by the Secure Messaging Service as described in Section 7.6. Additionally, HTML elements used to update, remove and flush user accounts are being ignored which is especially important when the spidering component uses high privileged users to map enforced access restrictions. An easier solution to this problem would be to disable form submits based on URL patterns as described in Section 6.2.1. This is not possible with the overall structure of the Secure Messaging Service because the administration panel is not separated through a distinctive URL path such as /administrator/*. The −−sessionId-regex provided regular expression is necessary in case of the Secure Messaging Service because the application does not use cookies to store session IDs, they are being appended to the URL instead. Common and valid URLs in the Secure Messaging Service look like the following:

/adminHlpDsk_en.jsp;jsessionid=93951EA9E450BFCA6321BBBF8C1A66BB?...
Especially for spidering phases 2 and 3 as described in Section 6.4.6 it becomes necessary to alter the discovered URLs by removing and replacing session IDs in order to make them comparable and interchangeable between different users and roles. For example when an URL discovered by User 1 is being checked if it is accessible for User 2 then the embedded session ID of User 1 has to be replaced with the current session ID of User 2, otherwise the access is automatically denied because the session of User 1 might already have timed out. The −−access-denied-page regular expression is missing in case of the Secure Messaging Service because no customised error messages are being shown to a user when the access to a particular resource has been denied by the application. The Secure Messaging Service sends 302 Moved Temporarily HTTP status codes instead. As described in Section 6.4.6 HTTP status codes other than 200 OK are being considered as blocked requests in the spidering phases 2 and 3 if no regular expression has been given through the −−access-denied-page command line argument. Due to a bug in the Secure Messaging Service the resources available to the super user role could not be correctly and fully mapped. If the HTML page which is being received after a successful login is being re-requested by the spidering component then the user’s session will be immediately terminated. Doing the same with the other user roles (a normal user, a first-level helpdesk user and a signature service administrator) worked just fine. Spidering the Secure Messaging Service web application with four different user roles has been manually terminated after each user spidered the application for roughly an hour. After the spidering is complete the following archive 2011-04-13-www-dev.privasphere.com.rbacspider has been generated which then can be opened in the Presentation Layer for further analysis.

Chapter 7. Testing

| 137

7.5.3 Phase 3: Analysing the Results
The final step is to analyse the collected data from the spidering phase in the graphical user interface provided. Starting the application is done by double clicking the rbacvisualiser.jar file or by entering the following command into a terminal:

# java -jar rbacvisualiser.jar
Figure 7.7 shows the initial view of the collected data from the spidering component described in Section 7.5.2. As mentioned before, the Secure Messaging Service does not use URL paths to separate function domains of the web application such as making all resources belonging to the administration panel available trough URLs such as /administrator/*. Therefore most off the discovered resources belong to one large cluster as seen in Figure 7.7. Yet, it is still possible to analyse the data manually and spot misconfigurations in access controls and eventual privilege escalation vulnerabilities visually (because there have no <access-restriction> rules been configured in the Meta Layer XML file as seen in Listing 7.7) by clustering and comparing different user roles as described in Section 6.7.2. Figure 7.8 shows the same data collected from the spidering of the Secure Messaging Service but clustered on the signature service administrator and a normal user. The four clusters being: (1) the resources that are only accessible to the signature service administrator (coloured red), (2) the resources that are only accessible to a normal user (coloured blue), (3) the resources that are accessible by both users (gradient coloured) and (4) the 4 resources that are neither accessible by the signature service administrator nor a normal user (coloured pink). The developer or testing agent is now able to spot privilege escalation vulnerabilities or misconfigurations in enforced access controls by checking whether the cluster containing the resources accessible to both user roles contain URLs which should have been protected. The developer additionally has the possibility to add <access-restriction> rules based on the crawled data in order to automate the process of finding security issues in future scans of the Secure Messaging Service. This test case proved that the approach works with a large real life and productive web application in a feasible and tangible way. Although few declarative information have been provided to the spidering component it is still possible to get an overview over the enforced access restrictions and spot eventual security issues in the targeted web application despite the fact that the eventual issues have to be visually detected by a developer or testing agent. Additionally, a developer or testing agent has to be aware of the fact that the spidering component did not find all available resources and thus the visualised resources might not be complete.

7.6 False Positives in the Analysis Layer
During the execution of the test case described in Section 7.4 it became clear that the Analysis Layer might generate false positives depending on how well the spidering component is able to access and process the discovered URLs and on the termination criteria provided to the spidering component through command line arguments as described in Section 7.4.2 and Section 9.1.4 by a developer or testing agent. The false positives noticed during the execution of the test case described in Section 7.4 have been in the form of: User X should have access but has not! and arouse from how the spidering component is either manually terminated by the user or how the termination criteria have

Chapter 7. Testing

| 138

Figure 7.7: Analysing the collected Data from the Secure Messaging Service (Clustered by URL Paths)

Figure 7.8: Analysing the collected Data from the Secure Messaging Service (Clustered by Users)

Chapter 7. Testing

| 139

been set through command line arguments. Considering the developer or testing agent added an <access-restrictions> rule to the Meta Layer XML file such as the following:

<access-restrictions> <url pattern="/wordpress/wp-admin/*"> <role-id-ref>1</role-id-ref> </url> </access-restrictions>
If the spidering component is prematurely terminated either manually by the user using the interactively available command x followed by an ENTER to immediately terminate the currently running spidering process (as described Section 9.1.4) or by reaching a termination criteria such as limiting the spidering process to 10 minutes by providing the −−max-time 10 command line argument, false positives might occur because the spidering component might not have reached any URLs matching the above <access-restrictions> rule. This issue matches the case A user has no access to an allowed resource in the

<access-restrictions> tag as described in Section 6.7.1 and leads to the generation of a
warning that would not have occurred if the spidering process terminated properly or if the spidering component had enough time to examine all the discovered URLs.

A way to minimise the generation of these kind of messages is either to terminate the spidering component manually by entering c followed by an ENTER which instructs the spidering component to process the already discovered URLs but ignore newly found URLs (as described in Section 9.1.4) or by increasing the termination criteria that the spidering component has more time to discover all the available URLs.

7.7 Summary
The test cases described in this Chapter showed that the proposed security testing tool is indeed capable of detecting privilege escalation vulnerabilities in an automated way in real life applications if the appropriate <access-restrictions> rules have been configured by the testing agent in the Meta Layer XML file. Additionally, the test cases showed that the Presentation Layer is a very nifty tool for a testing agent to visually identify issues in the collected data from the spidering process and lets him add and edit <access-restrictions> rules which then can be used in future scans of a targeted web application. Two issues have been discovered during the execution of the test cases that a testing agent needs to be aware of, namely the creation of false positives if the spidering component is prematurely terminated and possible spidering loops through provided search functionality in the targeted web application but which usually are being resolved when the spidering component reaches the maximum allowed crawling level which defaults to 25 but nonetheless can take several hours to reach depending on the overall structure of the targeted web application. In general, the more <access-restrictions> rules are being configured by the testing agent the more issues can be found in an automated fashion by the Analysis Layer when the actual collected data is being compared with the <access-restrictions> rules as described in Section 6.7.1.

Chapter 7. Testing

| 140

8 Summary
This Chapter recapitulates critically the findings and limitations of this thesis and proposes a list of further steps for the project.

8.1 Conclusion
This thesis is concerned with improving security in web applications by using declarative information provided in web application frameworks. The security testing tool that was researched and developed in this thesis is especially useful for detecting privilege escalation vulnerabilities and misconfigurations in access control checks in web applications with at least two distinguishable user roles in an automated fashion whereas one can be anonymous. This thesis has shown that it is indeed possible to detect misconfigurations and possible privilege escalation vulnerabilities in an automated fashion. The approach has been designed to be extensible and enough generic to use it with different web application frameworks and even programming languages. Depending on the overall design of the web application, adding access control checks to each accessible resource can be a very tedious task and it is very likely that some checks might have gone forgotten, rendering the application vulnerable to privilege escalation attacks. By using declarative information provided in source- or configuration files of a developed web application it becomes possible to detect privilege escalation vulnerabilities and misconfigurations in access control checks in an automated fashion allowing a developer to make the web application more secure. The developed security testing tool has been designed to be integrateable into a continuous build environment and fulfills the requirements for doing automated regression tests for a deployed web application. Finding misconfigurations in access control checks and thus possible privilege escalation vulnerabilities is being done in three phases: (1) The source code and configuration files of a targeted web application are being parsed by a Java command line Framework Parser specifically tailored to support a particular used web application framework such as Struts 1.x, 2.x, Spring etc. The Framework Parser’s output is a general Meta Layer XML file containing all the found data such as a (non) complete list of available URLs for seeding the web crawling component used in phase 2, a (non) complete list of available users and user roles, a (non) complete list of existing access constraints. A developer or testing agent then has the opportunity to enhance the generated Meta Layer XML file with additional information such as adding more information about users and roles, user passwords, configuring how users have to authenticate themselves to the targeted web application e.g.: HTML form based authentication and URL based access restrictions as they have been intended during the development of the web application. Such a particular access restriction rule might state that only users with an Administrator role should have access to HTML resources accessible through URLs beginning with /administrator/*. (2) The second phase consists of a web crawling component which takes the Meta Layer XML file as an input. The spidering of the targeted web application is done for each configured user respectively role configured in the Meta Layer XML file to find all possible transitions and enforced access restrictions. The output of the spidering component is a ZIP archive containing all the collected data. (3) The third and final phase consists of a Java Swing Visualisation Application which visualises and analyses the collected data from the

Chapter 8. Summary

| 141

spidering component as a directed graph whereas nodes correspond to found HTML resources and edges correspond to transitions between HTML resources. The analysis is done in an automated fashion if the testing agent provided access restrictions rules which can be compared with the actual collected data from the spidering. If no information has been provided regarding access restrictions the user has the possibility to spot privilege escalation vulnerabilities or misconfigurations visually and manually by various means provided by the Visualisation Application such as clustering the collected data based on URLs, clustering based on users or by re-colouring nodes according to their accessibility. The test cases showed that the developed security testing tool is indeed capable of detecting privilege escalation vulnerabilities in an automated fashion in real life applications if the appropriate <access-restrictions> rules have been configured by the testing agent in the Meta Layer XML file or have been extracted from source- or configuration files. Additionally, the test cases showed that the Presentation Layer is a very handy tool for a testing agent to manually and visually identify issues in the collected data from the spidering process and lets him add and edit <access-restrictions> rules which then can be used in future scans of a targeted web application. Two issues have been discovered during the execution of the test cases that a testing agent needs to be aware of, namely the creation of false positives if the spidering component is prematurely terminated and possible spidering loops through provided search functionality in the targeted web application but which are usually being resolved when the spidering component reaches the maximum allowed crawling level which defaults to 25 but nonetheless can take several hours to reach depending on the overall structure of the targeted web application. In general, the more <access-restrictions> rules are being configured by the testing agent the more issues can be found in an automated fashion by the Analysis Layer when the actual collected data is being compared with the <access-restrictions> rules as described in Section 6.7.1 and the generated report of the automatically found issues can be used in a software testing lifecycle as described in Section 6.7.6.

8.2 Further Steps
Another interesting approach that has been looked at during this thesis is the detection of misconfigurations and eventual privilege escalation vulnerabilities in attribute based access control checks. It might be possible to enhance the here developed approach further to incorporate XACML files (see Section 3.1). Additionally, the developed Visualisation Application could certainly be enhanced with more techniques to visualise the collected data from the spidering process. Besides the two previously mentioned books for inspiration [8] and [22], there are certainly more clever ways to help a user find outliers from the collected data regarding missing or wrongly enforced access control checks. As described in Section 7.1 the spidering component could be further enhanced that the component scores more than the currently 87% by looking into the 7 test cases that it missed.

Chapter 8. Summary

| 142

9 User Manual & Development Guide
This Chapter contains the user manual and the development guide for the developed security testing tools. The user manual leads new users through the installation process of the tools and helps to get familiar with the various command line arguments. The development guide is for advanced users who want to extend the functionality of the testing tools and build them from source.

9.1 User Manual
The user manual is targeted at users who want to install and run the security testing tools on their machines. The manual outlines the software and hardware requirements, installation procedures and how to start the applications.

9.1.1 Hard- & Software Requirements
The three developed tools are purely written in Java and therefore compatible with various operating systems such as Microsoft Windows, Linux and Mac OS as long as these provide a Java Virtual Machine (JVM) respectively the Java Runtime Edition (JRE). The security testing tools have been tested and noted to work with the following Java version:

java version "1.6.0_24" Java(TM) SE Runtime Environment (build 1.6.0_24-b07) Java HotSpot(TM) 64-Bit Server VM (build 19.1-b02, mixed mode)
The security testing tools are packaged for distribution in a single JAR archive so that they contain all of the free/open source third-party libraries necessary to run them successfully. Building from source and testing has primarily been done on a Linux Ubuntu 10.10 machine. The hardware requirements include the following: • Java(TM) SE Runtime Environment >= 1.6.0 (32 or 64 bit) • >= 2 GB Random Access Memory (RAM)

9.1.2 Installation Instructions
The following Section outlines the necessary steps for installing and running the described testing tools which are enclosed on the attached CD-ROM: Prerequisites: As mentioned before a working Sun JRE >= 1.6.0 is necessary in order to run the tools correctly. Check the manual of your operating system on how to install the Sun JRE. Installation: The installation is fairly simple. Insert the enclosed CD-ROM and double click on the install.sh executable shell script in the CD-ROM’s root directory and follow the on screen instructions. Alternatively open a terminal and enter the following commands:

Chapter 9. User Manual & Development Guide

| 143

(a) Welcome Screen

(b) Readme Screen

(c) License Screen

Figure 9.1: Installer

# cd /media/<CD-ROM DRIVE> # java -jar Binaries/Installer.jar
If you have to install the tools on a remote machine (e.g. accessed through a SSH client) where no graphical user interface is available to you, you can alternatively install them in a text-only mode by adding the -console parameter to the installer like shown in the following snippet:

# java -jar Installer.jar -console
Figure 9.1 shows the first three steps of the installer. The installer has been created using the free IzPack project. IzPack is a one-stop solution for packaging, distributing and deploying applications. It is fully cross-platform and generates a single installer. As such, it is an alternative to native solutions such as platform-specific installers and package managers. IzPack-generated installers only require a Java virtual machine to run1 .

9.1.3 Running the Meta Layer Extraction Tool
Using a framework specific parser to generate a Meta Layer XML file is pretty straightforward and can be run by issuing the following command inside a terminal:

# jar struts1parser.jar java -jar struts1parser.jar [options...] <DIRECTORY> -o FILE : write the Meta Layer XML structure to this file
Whereas the <DIRECTORY> is the top level directory of a web application’s source and configuration files. The output of such a framework specific parser is a Meta Layer XML file as described in Section 6.2. Listing 9.1 shows a typical output shown when the Meta Layer XML file is being created from a provided web application’s source and configuration files.
1 2 3 4 5 6 7 8

[+] [+] [+] [+] [+] [+] [+] [+]
1

Found 95 files Creating PicoContainer ... Wiring PicoContainer ... Found Java class : ../ rbacstruts1 / CheckUserCommand . java Found Java class : ../ rbacstruts1 / Constants . java Found Java class : ../ rbacstruts1 / LogOffAction . java Found Java class : ../ rbacstruts1 / LoginAction . java Found Java class : ../ rbacstruts1 / LoginForm . java

http://izpack.org/ [13/03/11]

Chapter 9. User Manual & Development Guide

| 144

9 10 11 12 13 14

[+] [+] [+] [+] [+] [+]

Found Java class : ../ rbacstruts1 / SuccessAction . java Found Java class : ../ rbacstruts1 / User . java Found Web XML Config : ../ webapp / WEB - INF / web . xml Found Struts Config : ../ webapp / WEB - INF / struts - config . xml Writing XML file to : metalayer . xml Stopping PicoContainer ...

Listing 9.1: Typical output from a Framework specific Parser while generating the Meta Layer XML File

9.1.4 Running the RBAC Spider
Listing 9.2 contains a list of all the options available to the user for tweaking the spidering process done by the RBAC spidering tool to retrieve and process HTML pages and transitions.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34

# java - jar rbacspider . jar java - jar rbacspider . jar [ options ...] FILE TARGET -- access - denied - page VAL : Matches the content of a received HTML resource against the following expression and marks it as an access denied if a match has been found -- check - ssl : Whether SSL certificates should be checked -- css - enabled : Whether CSS parsing should be enabled or not -- debug : Starts the application in debug mode -- export - graph FILE : Exports the spidered data in the GraphML format -- follow - regex VAL : Only follow the URLS matching this expression -- forceful - browsing : Whether forceful browsing should be enabled or not -- ignore - htmlelements - regex VAL : Html elements to ignore when spidering -- ignore - regex VAL : The spider will ignore URLS matching the following expression -- javascript - enabled : Whether JavaScript parsing should be enabled or not -- max - spidering - level N : How many levels the spider should examine -- max - spidering - time N : Stops the spidering after x minutes -- max - threads N : Number of max spidering threads -- only - forward : Whether the spider should stay on the same domain -- proxy - host VAL : Proxy Host -- proxy - password VAL : Proxy Password -- proxy - port N : Proxy Port -- proxy - username VAL : Proxy Username -- sessionId - regex VAL : Regular expression used to find session identifiers in the URL

Listing 9.2: Command Line Options for the RBACSpider The available command line options are pretty self explanatory and reasonable default values have been chosen so that a user in most cases and for most standard web application does not have to provide any arguments to the RBAC Spider. Additionally, the user has the possibility to interactively manipulate the crawler during a spidering process. This might be necessary for example when the user did not specify a termination criteria

Chapter 9. User Manual & Development Guide

| 145

through command line arguments such as −−max-spidering-level, −−max-spidering-time prior to launching the process but wants to stop the procedure now. The following options are available to the user: • Stop Spider threads by entering ’x’: If the user enters x followed by pressing ENTER the current spidering process will be immediately terminated and the next step in the spidering procedure is being started as described in Section 6.4.6. Forcefully stopping the spidering procedure with this feature is not encouraged and leads to false positives because this prevents the testing framework to map and access all the discovered URLs with each user role. This missing information leads to false positives in the Analysis Layer when the <access-restrictions> rules are being compared with the collected data as described in Section 6.7.1 and Section 7.6. The most likely false positive error message is User X should have access but has not! and is indeed because the information is simply missing due to premature termination of the spidering process. • Disable discovering new URLs by entering ’c’: This feature is similar to stopping the spidering process as described above but reduces the number of false positives. The difference is that the spidering process is not terminated immediately but rather the already discovered URLs are being properly processed as described in Section 6.4.6 and solely newly discovered URLs are being ignored and not saved in the spidering queue. This generally leads to fewer missing information and thus reduces the creation of false positives in the Analysis Layer as described in Section 6.7.1 and Section 7.6. • Show Spider statistics by entering ’s’: If the user enters s followed by pressing ENTER, a short statistic is displayed about how man threads are currently running and how much more URLs have to be processed by the spidering component before the next phase is being initiated as described in Section 6.4.6. A typical output might look like the following: Running threads [10] Remaining URLs [200].

9.1.5 Running the RBAC Visualiser
The RBAC Visualiser can be started by issuing the command java -jar rbacvisualiser.jar or by double clicking on the JAR archive. Additionally the RBAC Visualiser provides one command line argument which is essential for integrating the developed security testing tool into an automated security testing framework such as ASTF. The available command line options are shown in Listing 9.3.
1 2 3 4

# java - jar rbacvisualiser . jar java - jar rbacvisualiser . jar [ - - export < OUTPUTFILE > < ARCHIVE >] -- export FILE : Exports the found problems as a XML file

Listing 9.3: Command Line Options for the RBACVisualiser If the −−export flag is being provided the RBAC Visualiser does not show a graphical user interface to the user, instead it behaves like a common command line only application and analyses the previously collected data from the spidering process and generates a XML report containing found issues which normally are being displayed to the user as seen in Figure 7.3. The following snippet shows how such a XML file is being created:

# java -jar rbacvisualiser.jar \ --export out.xml 2011-03-24-localhost.rbacspider

Chapter 9. User Manual & Development Guide

| 146

Listing 9.4 shows an example of such a created XML report containing the found issues by comparing the <access-restrictions> information from the Meta Layer XML file with the actual collected data from the spidering process.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

<? xml version = ’1.0 ’ encoding =’ UTF -8 ’? > < rbacvisualiser > < info > < created > Sat Mar 26 13:32:53 2011 </ created > < timestamp >1301142773050 </ timestamp > </ info > < error name =" Access Violation " url =" http :// localhost :8080/ administrator / v1 . do " username =" EMPLOYEE " > EMPLOYEE has access but should have been denied ! </ error > < error name =" Access Violation " url =" http :// localhost :8080/ administrator / v1 . do " username =" MANAGER " > MANAGER has access but should have been denied ! </ error > < error name =" Access Violation " url =" http :// localhost :8080/ administrator / v1 . do " username =" CLERK " > CLERK has access but should have been denied ! </ error > </ rbacvisualiser >

Listing 9.4: Sample XML Report created from the found Vulnerabilities

Selecting and Editing Graph

The RBAC Visualiser provides mouse and keyboard shortcuts for modifying the visualised graph. Most important is the Graph menu in the menu bar as seen in Figure 9.2 (a). A user is able to cluster the graph differently, apply colours, filter vertices and show or hide vertex lables. The visualiser provides two mouse modes, namely Transforming and Picking. The mouse modes can be changed by right clicking the yellow graph panel as seen in Figure 9.2 (b) and Figure 9.6. The Transforming mode is used to move the hole graph on the screen by holding the left mouse button pressed and moving the mouse. The Picking mode provides the ability to select single or multiple vertices by left clicking them. If the graph has previously been clustered, the user is able to select all vertices belonging to a particular cluster by selecting one vertex from the cluster by simultaneously pressing CTRL as seen in Figure 9.2 (c). Double clicking a particular vertex opens an additional tab in the lower tab bar containing information about that particular vertex such as: which user had access to that resource, how many in or out transitions can be made etc. Additionally, the user is able to zoom in or out by using the mouse wheel no matter what mouse mode has been set.
Exporting the Graph

For interoperability reasons the RBAC Visualiser offers the ability to export the collected data from the spidering process as a GraphML file. GraphML is a comprehensive and easy-to-use file format for graphs. It consists of a language core to describe the structural properties of a graph and a flexible ex-

Chapter 9. User Manual & Development Guide

| 147

(a) Graph Options

(b) Changing Mouse Mode

(c) Selecting Vertices

Figure 9.2: RBAC Visualiser tension mechanism to add application-specific data2 . This is very convenient for a user who wants to visualise the data in another tool such as Gephi. Gephi is an interactive visualisation and exploration platform for all kinds of networks and complex systems, dynamical and hierarchical graphs. Gephi has at one point been considered as a basis for the RBAC Visualiser but its code base is not loosely coupled enough to allow other tools use its visualisation capabilities. Of course by using a different graph visualisation software the user looses the unique features of the RBAC Visualiser for displaying discrepancies in the access control checks.

9.1.6 Video Tutorials
The enclosed CD-ROM contains various tutorial videos in the OGG video format. These videos show how to configure the command line tools and how to start them. The videos are located in the root directory of the CD-ROM in a folder labelled Videos. The videos have been recorded using gtk-recordmydesktop version v0.3.8.1 on a Ubuntu 10.10 Linux operating system.

9.2 Development Guide
The development guide is targeted at users who want to extend the functionality of the tools and build them from source on their own machines. The following Sections contain detailed instructions on how to build the security testing tools and what kind of tools and libraries have been used during the initial development phase.

9.2.1 Prerequisites
The following Section outlines the prerequisites needed for building the tools from source. Sun Java 1.6 JDK: Development and testing has been done on a Ubuntu 10.10 Linux operating system with the Java(TM) SE Runtime Environment (build 1.6.0_24-b07) and Java(TM) SE Development Kit (1.6.0_20) 32 and 64 bit. Check the manual of your Linux operating system on how to install the Sun JRE and JDK. Maven 2: Apache Maven is a software project management and comprehension tool. Based on the concept of a project object model (POM), Maven can manage a project’s build, reporting and documentation from a central piece of information. If you are using the Ubuntu operating system you can install Maven by simply typing: sudo apt-get install maven into a terminal. See the Maven
2

http://graphml.graphdrawing.org/ [27/03/11]

Chapter 9. User Manual & Development Guide

| 148

documentation on how to install Maven for your specific operating system (http://maven.apache.org/download.html#Installation).

9.2.2 Installing the Development Environment
Insert the enclosed CD-ROM and double click on the install.sh executable shell script in the CDROM’s root directory and follow the on screen instructions of the installer. Alternatively open a terminal and enter the following commands:

# cd /media/<CD-ROM DRIVE> # java -jar Binaries/Installer.jar
Make sure that you select the Sources option in the package selection panel in order to install the source files needed for extending and building the tools from source. If you have to install the sources on a remote machine (e.g. accessed through a SSH client) where no graphical user interface is available to you, you can alternatively install them in a text-only mode by adding the -console parameter to the installer like shown in the following snippet:

# java -jar Installer.jar -console
After successfully installing the Java source files, several Java libraries needed for building the security tools have to be added to the local Maven repository on the development machine. This is necessary because these specific libraries are not part of an official and online available Maven repository. Open a terminal and enter the following commands in order to add the necessary libraries to the local Maven repository:

# cd <Installation Directory>/src/ # ./install-missing-maven-deps.sh
The install-missing-maven-deps.sh script has to be executed only once after installing the sources. See Section 9.2.4 for a complete list of used free/third party libraries that have been used.

9.2.3 Building with Maven
Building the security testing tools from source is fairly easy and is taken care of by Maven. Enter the following commands into a terminal for building the projects from the previously installed source files:

# cd <Installation Directory>/src/ # mvn clean package source:jar -Dmaven.test.skip
Alternatively, if you want to skip the JUnit tests which are being automatically executed by Maven after the build process you can provide the -Dmaven.test.skip parameter to the mvn command. For more information about specific Maven commands visit http://maven.apache.org/.

9.2.4 Free/Third-Party Libraries Used
The security testing tools rely heavily on several free and third party libraries that provide the needed functionality. Table 9.2 provides a list of all the libraries used and their specific licensing scheme.

Chapter 9. User Manual & Development Guide

| 149

9.2.5 Use logback for debugging purposes
The developed tools use the free SLF4J project to provide extensive logging capabilities for its users. The Simple Logging Facade for Java (SLF4J) serves as a simple facade or as an abstraction for various logging frameworks, e.g. java.util.logging, log4j and logback, allowing the end user to plug in the desired logging framework at deployment time. By making modifications to the logback.xml file inside the packaged jar files, the logging details can be increased in order to create a more verbose output for debugging and program error identification purposes.

9.3 RBACVisualiser Screenshots
Figures 9.3 to 9.16 show the developed visualisation tool and the most interesting features. Some additional features such as filtering based on users and other non clustering view modes have intentionally been left out but can be enabled by clicking on the Graph menu after opening a rbacpsider archive file. • Figure 9.3: Shows the initial view after opening the collected data from a previous spidering with the web crawler component. The collected data visualised in these screenshots are coming from the test case described in Section 7.4. The data is clustered based on the URL paths of the found URLs as described in Section 6.7.3 and is the default view mode after opening a project. At the bottom the URL List Tab is clearly visible which contains all the discovered URLs in the targeted application and the HTTP responses received from the web server during the spidering process. • Figure 9.4: At the bottom the Problems Tab contains the issues found during the analysis of the collected data as described in Section 6.7.1. Severe issues are being marked with a red exclamation mark when an <access-restriction> rule has been clearly violated. Other non severe issues are being marked with a yellow exclamation mark and might involve false positives and HTTP error codes received from the web server. False positives might occur when the spidering process has been terminated early by the user and the crawling component could not access all the discovered URLs with all available users which leads to an incomplete mapping of the targeted web application. • Figure 9.5: At the bottom the Access Restrictions Tab contains the configured <access-restriction> rules from the Meta Layer XML file used during the spidering process as described in Sections 6.2.1 and 6.7.1. Figure 9.10 shows how rules can be added or edited.

• Figure 9.6: The visualisation tool has two mouse modes used to select and edit the displayed graph as described in Section 9.1.5. The visualiser provides two mouse modes, namely Transforming and Picking. The mouse modes can be changed by right clicking the yellow graph panel. • Figure 9.7: Shows the context menu of the URL List Tab which can be accessed by selecting one or multiple entries from the URL List table and clicking on the right mouse button. The user then has the possibility to highlight the corresponding nodes in the graph which belong to the selected entries or inspect the HTTP response and request sent by crawling component to query the selected URLs as seen in Figure 9.8. • Figure 9.8: Shows the HTTP request and response dialog from a selected URL List entry as seen in Figure 9.8. The dialog shows the raw request and response received from the web server when the crawling component tried to access the selected URL with a particular user.

Chapter 9. User Manual & Development Guide

| 150

• Figure 9.9: Shows the context menu of the Problems Tab which can be accessed by selecting one or multiple entries from the Problems table and clicking on the right mouse button. The user then has the ability to highlight the corresponding nodes in the graph which belong to the selected issues. • Figure 9.10: Shows the context menu of the Access Restrictions Tab which can be accessed by selecting one or multiple entries from the Access Restrictions table and clicking on the right mouse button. The user then has the ability to highlight the corresponding nodes in the graph which belong to the selected <access-restriction> rule configured in the Meta Layer XML file. The user then furthermore has the ability to edit or add new <access-restriction> rules as seen in Figure 9.11. • Figure 9.11: The <access-restriction> dialog shown in this Figure allows a user to edit or add new <access-restriction> rules to the previously loaded project and Meta Layer XML file. Changes made to the <access-restriction> rules stay in memory and are not being saved until the user clicks on File → Export Meta Layer XML File... and saves the changes to a new Meta Layer XML file. • Figure 9.12: This Figure shows the Graph → Layouts menu which allows the user to cluster and re-arrange the displayed graph according to his preferences. These view modes are especially important when the user wants to manually or rather visually inspect the collected data from the spidering process for any issues that have not yet been covered with an <access-restriction> rule. The various view modes are being shown in Figures 9.13 to 9.16. • Figure 9.13: The Cluster By User view mode clusters the collected data based on two users as described in Section 6.7.2. The here shown dialog lets the user choose on which two users he wants the graph to be clustered on. The re-arranged and clustered graph is shown in Figure 9.14. This feature is only available if there have two or more users been configured in the Meta Layer XML file. • Figure 9.14: Shows the re-arranged, clustered and re-coloured graph after the user has selected two users from the dialog shown in Figure 9.13. The exact details on the clusering algorithm is described in Section 6.7.2. • Figure 9.15: The following view mode is based on the algorithm described in Section 6.7.4 which clusters the graph based on URL paths and re-colours the graph based on two selected users similar to the dialog shown in Figure 9.13. • Figure 9.16: Shows the same view mode as in Figure 9.15 but zoomed in on the found issue in the Problems Tab by comparing the actual data with the configured <access-restriction> rules in the Meta Layer XML file. The violation is clearly visible in the cluster on the right which should only be accessible by the administrator user (coloured red) but one node is also accessible by a user with fewer privileges (gradient coloured).

Chapter 9. User Manual & Development Guide

| 151

Figure 9.3: RBACVisualiser: Initial view with URL List Tab

Figure 9.4: RBACVisualiser: Initial view with Problems Tab

Chapter 9. User Manual & Development Guide

| 152

Figure 9.5: RBACVisualiser: Initial view with Access Restrictions Tab

Figure 9.6: RBACVisualiser: Mouse Modes

Chapter 9. User Manual & Development Guide

| 153

Figure 9.7: RBACVisualiser: URL List Context Menu

Figure 9.8: RBACVisualiser: URL List HTTP Request and Response Window

Chapter 9. User Manual & Development Guide

| 154

Figure 9.9: RBACVisualiser: Problems Context Menu

Figure 9.10: RBACVisualiser: Access Restrictions Context Menu

Chapter 9. User Manual & Development Guide

| 155

Figure 9.11: RBACVisualiser: Add/Edit Access Restrictions Window

Figure 9.12: RBACVisualiser: Graph Menu

Chapter 9. User Manual & Development Guide

| 156

Figure 9.13: RBACVisualiser: Cluster by User... Menu

Figure 9.14: RBACVisualiser: Cluster by User... View

Chapter 9. User Manual & Development Guide

| 157

Figure 9.15: RBACVisualiser: Re-Coloured by User... View

Figure 9.16: RBACVisualiser: Re-Coloured by User... View (Zoomed In)

Chapter 9. User Manual & Development Guide

| 158

Maven Artificat ID args4j

Version 2.0.12

URL https://args4j.dev.java.net/

License: MIT

picocontainer

2.11

http://picocontainer.org/

BSD style

neo4j https://github.com/tinkerpop/gremlin https://github.com/tinkerpop/blueprints http://jung.sourceforge.net/ BSD style BSD style BSD style

1.2

http://neo4j.org/

Apache License, v3

gremlin

0.7

blueprints

0.5-SNAPSHOT

jung-api

2.0.1

junit commons-logging slf4j-api logback-core commons-net http://commons.apache.org/ http://www.h2database.com/ http://htmlunit.sourceforge.net/ http://code.google.com/p/json-simple/

4.8.1 1.1.1 1.6.0 0.9.21 2.0

http://www.junit.org/ http://commons.apache.org/logging/ http://www.slf4j.org/ http://logback.qos.ch/ http://commons.apache.org/

CPLv1 Apache License, v2 MIT LGPL v2.1 Apache License, v2 Apache License, v2 EPL 1.0 Apache License, v2 Apache License, v2

commons-validator

1.3.1

h2 htmlunit json-simple

1.2.127 2.8 1.1

Chapter 9. User Manual & Development Guide

utils

1.07.00

http://ostermiller.org http://jakarta.apache.org/oro/

GPL Apache License, v2

oro

2.0.8

| 159

Description args4j is a small Java class library that makes it easy to parse command line options/arguments in your CUI application. PicoContainer is a tiny embeddable container for Constructor Dependency Injection (CDI) Inversion of Control (IoC) Java components. Neo4j is a graph database, a fully transactional database that stores data structured as graphs. Gremlin is a domain specific language for traversing property graphs. Blueprints is a set of interfaces and implementations of the property graph data model. JUNG is a Java-based open-source software library designed to support the modeling, analysis, and visualization of data that can be represented as graphs. JUnit testing framework Logging Framework Logging Framework Logging Framework Collection of network utilities and protocol implementations. Framework to define validators and validation rules in a XML file. Java SQL database HtmlUnit is a GUI-Less browser for Java programs. JSON.simple is a simple Java toolkit for JSON. You can use JSON.simple to encode or decode JSON text. Libraries for common programming tasks such as CSV files and Base64 encoding. The Jakarta-ORO Java classes are a set of textprocessing Java classes that provide Perl5 compatible regular expressions. Table 9.2: Used Free/Third Party Libraries

List of Figures
2.1 Web Tier and Java EE Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Java EE Server and Containers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Java EE Platform APIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 4 4

4.1 Architecture of a general combined White- and Black Box Security Testing Tool . . . . . . . 61 6.1 6.2 6.3 6.4 6.5 6.6 6.7 6.8 6.9 6.10 6.11 6.12 6.13 7.1 7.2 7.3 7.4 7.5 7.6 7.7 7.8 9.1 9.2 9.3 9.4 9.5 9.6 9.7 9.8 9.9 9.10 9.11 9.12 Framework Layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . MetaLayer API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Framework Specific Parser UML . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . JavaScript Execution Time (Logarithmic Scale) in various Browsers for HTML DOM Modifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Simplified UML 2 Activity Diagram showing the Main-Flow of the Web Spider . . . . . . . Simplified UML 2 Activity Diagram for the Non-Invasive-Smartly-Filling-Out HTML Form Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . UrlQueue Database Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Storing the crawled data in a graph optimised Database . . . . . . . . . . . . . . . . . . . . . . . Clustering by Users / Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Clustering by URL Paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Clustering by URL Paths and Re-Colouring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Clustering by URL Paths and showing differences . . . . . . . . . . . . . . . . . . . . . . . . . . . . GUI Design Prototype . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 . 91 . 92 . 101 . 103 . . . . . . . . 105 108 111 114 115 116 116 118

Struts 1.3.10 Test Web Application - URL Layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 Initial view of the collected Data from the Struts Test Application . . . . . . . . . . . . . . . . . 128 Automatically detected Access Restriction Violations in the Struts Test Application . . . . 129 Re-Colouring of the Data based on Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 Automatically detected Access Restriction Violations in the WordPress Test Application . 134 Standard User Secure Messaging Service Email Inbox . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 Analysing the collected Data from the Secure Messaging Service (Clustered by URL Paths)139 Analysing the collected Data from the Secure Messaging Service (Clustered by Users) . . . 139 Installer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . RBAC Visualiser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . RBACVisualiser: Initial view with URL List Tab . . . . . . . . . . . . . RBACVisualiser: Initial view with Problems Tab . . . . . . . . . . . . RBACVisualiser: Initial view with Access Restrictions Tab . . . . . . RBACVisualiser: Mouse Modes . . . . . . . . . . . . . . . . . . . . . . . . RBACVisualiser: URL List Context Menu . . . . . . . . . . . . . . . . . RBACVisualiser: URL List HTTP Request and Response Window RBACVisualiser: Problems Context Menu . . . . . . . . . . . . . . . . . RBACVisualiser: Access Restrictions Context Menu . . . . . . . . . . RBACVisualiser: Add/Edit Access Restrictions Window . . . . . . . RBACVisualiser: Graph Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 148 152 152 153 153 154 154 155 155 156 156

List of Figures

| 160

9.13 9.14 9.15 9.16

RBACVisualiser: Cluster by User... Menu . . . . . . . . . . . . . RBACVisualiser: Cluster by User... View . . . . . . . . . . . . . RBACVisualiser: Re-Coloured by User... View . . . . . . . . . . RBACVisualiser: Re-Coloured by User... View (Zoomed In)

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

157 157 158 158

List of Figures

| 161

List of Tables
2.2 Popularity of Open Source Java Web Application Frameworks (November 2010) . . . . . . . 29 2.4 Java Web Application Frameworks Security Features Comparison . . . . . . . . . . . . . . . . . 45 6.2 Description of the UrlQueue Entity Relationship Model . . . . . . . . . . . . . . . . . . . . . . . . 107 7.1 Access Rights of the Sample Struts 1.3.10 Web Application . . . . . . . . . . . . . . . . . . . . . . 130 9.2 Used Free/Third Party Libraries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

List of Tables

| 162

Listings
2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9 2.10 2.11 2.12 2.13 2.14 2.15 2.16 2.17 2.18 2.19 2.20 2.21 2.22 2.23 2.24 2.25 2.26 2.27 2.28 2.29 2.30 2.31 2.32 2.33 2.34 2.35 2.36 2.37 2.38 2.39 2.40 Simple Java HelloWorld Servlet (Specification 2.5) . . . . . . . . . . . . . . . . . . . . . . Deployment Descriptor for the HelloWorld Servlet . . . . . . . . . . . . . . . . . . . . . . Simple Java Servlet (Specification 3.0) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Simple JSP Page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Simple JSTL usage Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hello World Java Bean (POJO) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hello World JSF Page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Specifying Security Roles using Deployment Descriptor elements . . . . . . . . . . . . @DeclareRoles Annotation Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . @RunAs Annotation Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . @ServletSecurity Annotation Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . @RolesAllowed Annotation Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sample Spring Security XML Configuration File . . . . . . . . . . . . . . . . . . . . . . . . . @PreAuthorize Annotation Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . @PreAuthorize Annotation Example with SpEL Directive . . . . . . . . . . . . . . . . . @PostFilter Annotation Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . @PreFilter Annotation Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ESAPI Example on how to do Input Validation . . . . . . . . . . . . . . . . . . . . . . . . . . ESAPI WAF Configuration Example on how to enforce Authorisation Constraints . Apache Shiro Ad-Hoc Filter Chain Configuration Example . . . . . . . . . . . . . . . . . jGuard User Permissions Configuration Example . . . . . . . . . . . . . . . . . . . . . . . . Spring MVC Controller Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Spring HandlerInterceptor Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Struts 1.x struts-config.xml Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Struts 1.x Action example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Apache Struts 1.x Declarative Access Control Example . . . . . . . . . . . . . . . . . . . . Apache Struts Programmatic Access Control Example . . . . . . . . . . . . . . . . . . . . Apache Struts Plugin Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Apache Struts RequestProcessor Example . . . . . . . . . . . . . . . . . . . . . . . . . . . Apache Struts ActionServlet Configuration Example . . . . . . . . . . . . . . . . . . . Struts 2 Annotation Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Struts 2.x struts.xml Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Struts 2.x Action Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Struts 2.x RolesInterceptor Configuration Example . . . . . . . . . . . . . . . . . . . . Struts ExtVal usage example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . MyFaces Custom Lifecycle Factory Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . MyFaces Custom Lifecycle Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Simple Wicket Page with link Counter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Wicket Page Layout for link Counter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Wicket-Sercurity Configuration Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 5 6 7 7 8 8 11 13 13 14 15 18 19 19 20 20 21 22 23 24 30 31 32 32 33 33 34 35 36 37 37 37 38 39 40 40 41 41 42

3.1 Abstracting Application-Level Web Security with a Policy Description Language Example 47

Listings

| 163

3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 3.10 3.11 4.1 4.2 4.3 4.4 4.5 4.6 6.1 6.2 6.3 6.4 6.5 6.6 6.7 6.8 6.9 6.10 6.11 6.12 6.13

JFlow Password File Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Class encapsulating the Message-Delivery Functionality . . . . . . . . . . . . . . . . Class to exercise the Message-Delivery Functionality . . . . . . . . . . . . . . . . . . . Class encapsulating the Message-Delivery Functionality (with Authentication) Authentication Aspect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . AspectJ Tracing Aspect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . AspectJ Tracing output for a simple Struts 1.x Application . . . . . . . . . . . . . . . . PQL Query for finding SQL Injection Vulnerabilities . . . . . . . . . . . . . . . . . . . . Aspect for catching all Struts 1.x execute Methods . . . . . . . . . . . . . . . . . . . . Aclamate RBC Configuration Example execute Methods . . . . . . . . . . . . . . . . Struts 2.x XWorks Validation Example with Annotations . . . . . . Struts 2.x XWorks Validation Example with XML . . . . . . . . . . . . Spring 3.x Validation Example with Annotations . . . . . . . . . . . . Spring 3.x Validation Example in a Controller Class . . . . . . . . . Wicket Validation Example . . . . . . . . . . . . . . . . . . . . . . . . . . . ESAPI Example on how to use the AccessReferenceMap Class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

48 50 50 51 51 54 55 56 57 58 62 63 64 64 65 67 85 86 86 87 87 88 89 89 90 94 98 99 117

Meta Layer XML Structure: <info> Tag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Meta Layer XML Structure: <classes> Tag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Meta Layer XML Structure: <users> Tag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Meta Layer XML Structure: <roles> Tag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Meta Layer XML Structure: <access-restrictions> Tag . . . . . . . . . . . . . . . . . . Meta Layer XML Structure: <login> Tag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Meta Layer XML Structure: <urls> Tag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Meta Layer XML Structure: <views> Tag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Meta Layer XML Structure: <form-data> Tag . . . . . . . . . . . . . . . . . . . . . . . . . . . . Meta Layer XML File generated from a Struts 1.x Sample Application . . . . . . . . . . . Short HtmlUnit Example for submitting a HTML Login Form through a Proxy Server Dynamically created HTML Login Form with embedded JavaScript Code . . . . . . . . Sample XML Report created from found Issues . . . . . . . . . . . . . . . . . . . . . . . . . . .

7.1 Checking Access Restrictions in JSP Pages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2 Initial metalayer.xml File for the Test Application . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3 Additional Information added to the metalayer.xml File (displayed as a diff -up FILE1 FILE2 output) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4 metalayer.xml File for a Standard WordPress 3.1 Installation . . . . . . . . . . . . . . . . . . 7.5 /wordpress/wp-login.php HTML Source Code containing the HTML based Authentication Form of WordPress . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.6 Secure Messaging Service HTML based Authentication Form . . . . . . . . . . . . . . . . . . . . 7.7 metalayer.xml File for the Secure Messaging Service . . . . . . . . . . . . . . . . . . . . . . . . 7.8 Crawling Command Line Arguments for the Secure Messaging Service . . . . . . . . . . . . . 9.1 Typical output from a Framework specific Parser while generating the Meta Layer XML File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2 Command Line Options for the RBACSpider . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.3 Command Line Options for the RBACVisualiser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.4 Sample XML Report created from the found Vulnerabilities . . . . . . . . . . . . . . . . . . . .

. 123 . 123 . 125 . 130 . . . . 131 135 135 136

. . . .

144 145 146 147

Listings

| 164

Bibliography
[1] D. Alhadidi, N. Belblidia, and M. Debbabi. Aspectj assessment from a security perspective. In Workshop on Practice and Theory of IT Security, PTITS’2006, Montreal, Quebec, Canada, May 17-19 2006. [2] Dima AlHadidi, Nadia Belblidia, and Mourad Debbabi. Security crosscutting concerns and aspectj. In Proceedings of the 2006 International Conference on Privacy, Security and Trust: Bridge the Gap Between PST Technologies and Business Services, PST ’06, pages 45:1–45:1, New York, NY, USA, 2006. ACM. [3] Chris Allan, Pavel Avgustinov, Aske Simon Christensen, Laurie Hendren, Sascha Kuzins, Ondˇej r Lhoták, Oege de Moor, Damien Sereni, Ganesh Sittampalam, and Julian Tibble. Adding trace matching with free variables to aspectj. In Proceedings of the 20th annual ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications, OOPSLA ’05, pages 345–364, New York, NY, USA, 2005. ACM. [4] P Boldi and S. Vigna. The webgraph framework i: compression techniques. In Proceedings of the . 13th international conference on World Wide Web, WWW ’04, pages 595–602, New York, NY, USA, 2004. ACM. [5] A. Boukhtouta, D. Alhadidi, and M. Debbabi. Capturing returned computable values in aspectj for security hardening. Inf. Sec. J.: A Global Perspective, 18:87–98, January 2009. [6] A. Boukhtouta, D. Alhadidi, and M. Debbabi. A practical framework for the dataflow pointcut in aspectj. In Availability, Reliability and Security, 2009. ARES ’09. International Conference on, pages 835 –840, 2009. [7] Kung Chen and Chih-Mao Huang. On designing access control aspects for web applications. In AOSD 05 Workshop on Software-engineering Properties of Languages and Aspect Tehchnologies (SPLAT 05), Chicago, 2005. [8] Greg Conti. Security Data Visualization. No Starch Press, San Francisco, CA, USA, 2007. [9] Martijn Dashorst and Eelco Hillenius. Wicket in Action. Manning Publications Co., Greenwich, CT, USA, 2008. [10] Kevin Denver. Development and Integration of ASTF-Plugins in the field of Web Application Vulnerabilities. MSE Project Thesis 1, February 2009. [11] Kevin Denver. Evaluation geeigneter Plugins im Bereich Web Application Vulnerabilities für die Integration ins ASTF. MSE Seminary Thesis 1, November 2009. [12] Kevin Denver. Development of a combined white and black box security testing tool for detecting injection flaws in web applications. MSE Project Thesis 2, August 2010. [13] Kevin Denver. Preliminary Study of white box security testing for the integration into the ASTF framework. MSE Seminary Thesis 2, February 2010.

Bibliography

| 165

[14] Paul Duvall, Stephen M. Matyas, and Andrew Glover. Continuous Integration: Improving Software Quality and Reducing Risk (The Addison-Wesley Signature Series). Addison-Wesley Professional, 2007. [15] R. Fielding, J. Gettys, J. Mogul, H. Frystyk, L. Masinter, P. Leach, and T. Berners-Lee. Hypertext Transfer Protocol – HTTP/1.1. RFC 2616 (Draft Standard), June 1999. Updated by RFC 2817. [16] J. Franks, P Hallam-Baker, J. Hostetler, S. Lawrence, P. Leach, A. Luotonen, and L. Stewart. HTTP . Authentication: Basic and Digest Access Authentication. RFC 2617 (Draft Standard), June 1999. [17] Marshall Hall, Jr. Combinatorial theory (2nd ed.). John Wiley & Sons, Inc., New York, NY, USA, 1998. [18] Rod Johnson. Expert One-on-One J2EE Design & Development. Wrox Press Ltd., Birmingham, UK, UK, 2002. [19] Bart Kummel. Apache MyFaces 1.2 Web Application Development. Packt Publishing, 1st edition, 2010. [20] Ramnivas Laddad. AspectJ in Action: Enterprise AOP with Spring Applications. Manning Publications Co., Greenwich, CT, USA, 2nd edition, 2009. [21] Michael Martin, Benjamin Livshits, and Monica S. Lam. Finding application errors and security flaws using pql: a program query language, 2005. [22] Raffael Marty. Applied Security Visualization. Addison-Wesley Professional, 1 edition, 2008. [23] Hidehiko Masuhara and Kazunori Kawauchi. Dataflow pointcut in aspect-oriented programming. In In APLAS’03 - the First Asian Symposium on Programming Languages and Systems, pages 105–121, 2003. [24] Hidehiko Masuhara and Gregor Kiczales. Modeling crosscutting in aspect-oriented mechanisms. pages 2–28. Springer-Verlag, 2003. [25] H. D. K. Moonesinghe and Pang-Ning Tan. Outrank: a graph-based outlier detection framework using random walk. International Journal on Artificial Intelligence Tools, 17(1):19–36, 2008. [26] Peter Mularien. Spring Security 3. Packt Publishing, 2010. [27] Andrew C. Myers. Jflow: Practical mostly-static information flow control. In In Proc. 26th ACM Symp. on Principles of Programming Languages (POPL, pages 228–241, 1999. [28] Andrew Prunicki and Tzilla Elrad. Aclamate: An aosd security framework for access control. In Proceedings of the 2nd IEEE International Symposium on Dependable, Autonomic and Secure Computing, DASC ’06, pages 293–300, Washington, DC, USA, 2006. IEEE Computer Society. [29] Raghu Ramakrishnan and Johannes Gehrke. Database Management Systems. McGraw-Hill, Inc., New York, NY, USA, 3 edition, 2003. [30] Ian Roughley. Practical Apache Struts 2 Web 2.0 Projects (Practical Projects). Apress, Berkely, CA, USA, 2007. [31] David Scott and Cb Pz. Abstracting application-level web security. In In World Wide Web, pages 396–407, 2002.

Bibliography

| 166

[32] Robert Endre Tarjan. Depth-first search and linear graph algorithms. SIAM J. Comput., 1(2):146– 160, 1972. [33] Te Kura Tatau, Roshan Ramachandran, Supervisors David, J. Pearce, and Ian S. Welch. Aspectj for multilevel security, 2006. [34] Yuan-Hsin Tung, Shian-Shyong Tseng, Tsung-Ju Lee, and Jui-Feng Weng. A novel approach to automatic test case generation for web applications. Quality Software, International Conference on, 0:399–404, 2010. [35] Robert J. Walker and Kevin Viggers. Implementing protocols via declarative event patterns. In Proceedings of the 12th ACM SIGSOFT twelfth international symposium on Foundations of software engineering, SIGSOFT ’04/FSE-12, pages 159–169, New York, NY, USA, 2004. ACM. [36] Bart De Win, Wouter Joosen, and Frank Piessens. Developing secure applications through aspect-oriented programming. In Aspect-Oriented Software Development, pages 633–650. Addison-Wesley, 2002. [37] Bart De Win, Bart Vanhaute, and Bart De Decker. How aspect-oriented programming can help to build secure software. Informatica (Slovenia), 26(2), 2002.

Bibliography

| 167

InIT Institut für angewandte Informationstechnologie

Master Thesis

Student: Kevin Denver Partner company: PrivaSphere AG

Tutor: Advisor: Expert:

Prof. Dr. Marc Rennhard Prof. Dr. Hans-Peter Hutter Dr. Tobias Christen

Start: End: Presentation:

8. November 2010 15. April 2011 TBA

Credits:

27 ECTS (810 h)

Using Declarative Information in Web Application Frameworks to Improve Security

Introduction and Motivation
Modern Model-View-Controller (MVC) frameworks such as Struts1, Spring2, or Java Server Faces3 significantly help designing and developing Java web applications. In contrast to pure servlet- and/or JSP-based applications, they allow the developers to specify some parts of the application outside the actual code using XML configuration files or annotations. Examples are the action mappings to specify (and easily adapt) the workflow in an application or input validation configurations. In a seminary thesis [1], it was analyzed how well security testing tools could use such configurations or annotations to improve their results with respect to detecting security problems. One result of that analysis was that it may well be possible to use this information to perform completeness checks (e.g. with respect to input validation) or to supplement the web crawler component of a web application security testing tool with additional information. In addition, Ralf Hauser proposed a while ago [2] that extending these web application frameworks with additional declarative information may help a lot to both make web application more secure already during development and enable a special kind of “white-box” testing tool to make more precise statements and therefore improve testing quality in case the declarative information has not been fully reflected into the code base. An example would be to provide additional information with respect to access control (e.g. role-based), which describes what rights are necessary to access certain parts of the application. Assuming this in1 2 3

http://struts.apache.org http://www.springsource.org http://java.sun.com/javaee/javaserverfaces/overview.html

28.10.2010, Marc Rennhard (rema)

1/3

formation has been provided by the developer, it can be used in several ways (note that this list is by no means meant to be complete and it’s one essential task during the thesis to define and analyze different options):  The framework could be extended such that it directly makes use of this access control information to enforce the corresponding rules. Ideally, the developer would no longer have to deal with access control by programming the corresponding logic, but only has to specify the correct rules in a declarative way. This framework extension/plugin would then deliver all necessary functions to enforce the declarations at runtime. A testing tool that interacts with the running web application uses the access control information to check whether access control is handled correctly by the web application. To do so, the testing tool would get credentials with different privileges and attempt to access different areas of the web application. Any mismatch between declared access control rules and actual reachability is flagged as a security vulnerability. Such a testing tool could not only be limited to role-based access control, but could also take into account mechanisms such as mandatory access control or information labeling policies. Using graph theory and analyzing the access control information together with further information in the configuration files (e.g. allowed transitions between pages), a testing tool could analyze if some parts of the application cannot be reached at all or if there are ways to reach a page that requires high access privileges via any path that allows access with lower privileges.

The goal of this master thesis is to build on this basis and explore the possibilities of using available and/or new declarative information in web application frameworks to improve security.

Task Description
Due to the open nature of this master thesis, we won’t provide a detailed step-by-step task description as each project phase will significantly depend on the results of the previous phases. We therefore only describe the main phases we expect at the current time, which are the following:  In the first phase, you should study available web application frameworks (focus on Javabased ones), the Java language, and related work in the relevant areas. The outcome of this phase should provide you with the following information:     The current state of the art in the field of web application frameworks4 and Java. The capabilities of the different frameworks, especially with respect to declarative statements a developer can include. How these frameworks can be extended. What other work has recently been done in the field and published in academic papers. You should also have a look at approaches in the context of model driven security5

In the second phase, you should think about using the information already provided by web application frameworks and how this can be supplemented with additional informa-

4

This should also answer the question whether there exist potential successor concepts to MVC as we want to make sure the planned improvements in this thesis build on the current (and future) state of the art.
5

Some first pointers can be found here: http://www.torsten-lodderstedt.de

28.10.2010, Marc Rennhard (rema)

2/3

tion from both the related work analysis and novel ideas by yourself. Your previous work during the seminary thesis [1] and the ideas with respect to access control should provide starting points, but you should certainly try to be elaborate and derive further ideas. With the different proposed ideas, do the following:  Evaluate them with respect to their potential. Ideally, a good idea should be easy to be used by skilled developers, should be applicable to a wide variety of frameworks and significantly contributes to improve security. For each proposal, state precisely how you would use the corresponding information to improves security. As stated above, this could mean extending the framework such that it uses the additional information or develop (or extend) possibly multiple testing tools that can make good usage of this information.

In the third phase, the best-suited ideas derived during the second phase should be implemented. The actual implementation (e.g. framework extension, testing tool development or both) depends on the actual approaches selected during the second phase. In either case, consider the following:   The developed software should be of good quality and both usable and extensible by others. In general, it is preferable if your selected proposals can be used with many current web application frameworks. However, it is likely that only one or a few of them will actually specifically be supported as part of your thesis. In this case, you’ll decide together with the company partner and tutor about what frameworks will be supported as part of your thesis. To validate your proposals, test the developed software with different web applications. If possible, PrivaSphere’s Secure Messaging application should be among them. If possible, design the testing tools such that they can easily be integrated into ASTF.

Deliverables
The deliverables of this thesis are the following:    Developed software Written report in English, which describes the methodology and the results of your thesis Oral presentation (40 minutes plus questions)

References
[1] [2] Kevin Denver. Preliminary study of white box security testing for the integration into the ASTF framework. ZHAW MSE Seminary Thesis, February 2010. Marc Rennhard and Ralf Hauser. Automated Software Security-Testing for OnlineApplications. KTI project proposal, March 2006.

28.10.2010, Marc Rennhard (rema)

3/3

Appendix B - Meta Layer XML Structure DTD
<!ELEMENT access­restrictions ( url+ ) >  <!ELEMENT attribute EMPTY >  <!ATTLIST attribute name NMTOKEN #REQUIRED >  <!ATTLIST attribute value CDATA #REQUIRED >  <!ELEMENT class EMPTY >  <!ATTLIST class id NMTOKEN #REQUIRED >  <!ATTLIST class name NMTOKEN #REQUIRED >  <!ELEMENT class­id­ref ( #PCDATA ) >  <!ELEMENT classes ( class+ ) >  <!ELEMENT created ( #PCDATA ) >  <!ELEMENT createdBy ( #PCDATA ) >  <!ELEMENT data ( #PCDATA ) >  <!ELEMENT disable­form­submit EMPTY >  <!ATTLIST disable­form­submit pattern CDATA #REQUIRED >  <!ELEMENT form ( form­url, form­name, data, submit­element ) >  <!ELEMENT form­based­login ( form­url, form­name, data, submit­element ) >  <!ELEMENT form­data ( form ) >  <!ELEMENT form­name ( #PCDATA ) >  <!ELEMENT form­url ( #PCDATA ) >  <!ELEMENT info ( created, timestamp, createdBy, notes ) >  <!ELEMENT login ( form­based­login ) >  <!ELEMENT notes EMPTY >  <!ELEMENT role ( user­id­ref+, disable­form­submit? ) >  <!ATTLIST role id NMTOKEN #REQUIRED >  <!ATTLIST role name NMTOKEN #REQUIRED >  <!ATTLIST role priority NMTOKEN #REQUIRED >  <!ELEMENT role­id­ref ( #PCDATA ) >  <!ELEMENT roles ( role+ ) >  <!ELEMENT submit­element ( #PCDATA ) >  <!ELEMENT timestamp ( #PCDATA ) >  <!ELEMENT url ( role­id­ref*, class­id­ref? ) >  <!ATTLIST url access NMTOKEN #IMPLIED >  <!ATTLIST url id NMTOKEN #IMPLIED >  <!ATTLIST url path CDATA #IMPLIED >  <!ATTLIST url pattern CDATA #IMPLIED >  <!ELEMENT url­id­ref ( #PCDATA ) >  <!ELEMENT urls ( url+ ) >  <!ELEMENT user ( attribute?, disable­form­submit? ) >  <!ATTLIST user id NMTOKEN #REQUIRED >  <!ATTLIST user name NMTOKEN #REQUIRED >  <!ELEMENT user­id­ref ( #PCDATA ) >  <!ELEMENT users ( user+ ) >  <!ELEMENT view ( class­id­ref?, url­id­ref? ) >  <!ATTLIST view path CDATA #REQUIRED >  <!ELEMENT views ( view+ ) >  <!ELEMENT web­meta ( info, classes?, users?, roles?,  access­restrictions?, login?, urls?, views?, form­data? ) > 

Appendix C - UML class design of the web spidering component

AbstractThreadManager

<<interface>> <<realize>>

ThreadManager

setKillThreads(t : boolean) : void getThreadCount() : int cleanUp() : void getOpenUrls() : int setSavingNewUrls(flag : boolean) : void

<<create>> AbstractThreadManager(deps : ThreadManagerDependencies) initialise() : void startThreads() : void discoveredNewUrl(entry : UrlQueueEntry) : boolean threadIsFinished(thread : WebSpiderThread) : void getNextEntry() : UrlQueueEntry isSpideringConstraintViolated(entry : UrlQueueEntry) : boolean initialiseStartTime() : void isDiskFull() : boolean isMaxLevelReached(currentLevel : int) : boolean isTimeUp() : boolean deleteUrlQueueEntry(entry : UrlQueueEntry) : void updateUrlQueueEntry(entry : UrlQueueEntry) : void setEntryAsBeingFed(entry : UrlQueueEntry) : void saveUrlQueueEntry(entry : UrlQueueEntry) : int loadUrlQueueEntry(id : int) : UrlQueueEntry pathExists(entry : UrlQueueEntry,parent : UrlQueueEntry) : boolean getOpenUrls() : int saveRedirect(entry : UrlQueueEntry) : void getThreadCount() : int cleanUp() : void getUser() : User setUser(u : User) : void isKillThreads() : boolean setKillThreads(kill : boolean) : void setSavingNewUrls(flag : boolean) : void isSavingNewUrls() : boolean getAccessDeniedRegex() : String getTestData() : TestData setTestData(tData : TestData) : void getOptions() : GlobalOptions getUrlValidator() : WebSpiderUrlValidator

NormalWebSpiderThreadManager <<create>> ForcefulBrowsingWebSpiderThreadManager(deps : ThreadManagerDependencies) initialiseURLs(user : User) : void startThreads() : void discoveredNewUrl(entry : UrlQueueEntry) : boolean threadIsFinished(thread : WebSpiderThread) : void getNextEntry() : UrlQueueEntry replaceSessionIds(entry : UrlQueueEntry) : void setCurrentSessionId(sessionId : NameValuePair) : void getOpenUrls() : int

ForcefulBrowsingWebSpiderThreadManager

MissingUrlWebSpiderThreadManager <<create>> MissingUrlWebSpiderThreadManager(deps : ThreadManagerDependencies) initialiseMissingUrls(users : Users) : void startThreads() : void discoveredNewUrl(entry : UrlQueueEntry) : boolean threadIsFinished(thread : WebSpiderThread) : void getNextEntry() : UrlQueueEntry getChain(entry : UrlQueueEntry) : UrlQueueEntry replaceSessionIds(entry : UrlQueueEntry) : void setCurrentSessionId(sessionId : NameValuePair) : void

<<create>> NormalWebSpiderThreadManager(deps : ThreadManagerDependencies) initialiseFirstHtmlPage(page : HtmlPage) : void initialiseUrlSeeds(config : MetaLayerConfig) : void startThreads() : void threadIsFinished(thread : WebSpiderThread) : void discoveredNewUrl(entry : UrlQueueEntry) : boolean getNextEntry() : UrlQueueEntry

<<uses>> <<uses>> <<uses>>

NormalWebSpiderThreadImpl

ForcefulBrowingWebSpiderThreadImpl

MissingUrlWebSpiderThreadImpl

AbstractWebSpiderThread

<<interface>> <<realize>> WebSpiderThread setUrlQueueEntry(entry : UrlQueueEntry) : void getUrlQueueEntry() : UrlQueueEntry cleanUpBeforeExit() : void

<<create>> AbstractWebSpiderThread(ext : List<UrlExtractor>,wFactory : WebClientFactory) run() : void sleep() : void cleanUp(page : Page) : void cleanUp() : void cleanUpBeforeExit() : void saveWebRequestAndResponse() : void saveWebRequestAndResponse(request : WebRequest,response : WebResponse) : void saveWebRequestAndResponseOnCriticalError() : void getWebPage(parent : UrlQueueEntry) : Page isAccessDeniedPage(accessDeniedRegex : String,page : Page) : boolean printInformationForUser() : void setUrlQueueEntry(qEntry : UrlQueueEntry) : void getUrlQueueEntry() : UrlQueueEntry

Appendix D - UML class design of the URL extractors

<<interface>> UrlExtractor collectUrls(currentUrl : String,htmlPage : HtmlPage) : int getWebClient() : WebClientWrapper setWebClient(client : WebClientWrapper) : void addDiscoverUrlListeners(listener : DiscoveredUrlListener) : void getOrder() : int getTestData() : TestData setTestData(data : TestData) : void getUser() : User setUser(u : User) : void

<<realize>>

AbstractUrlExtractor <<create>> AbstractUrlExtractor(val : WebSpiderUrlValidator) collectUrls(currentUrl : String,htmlPage : HtmlPage) : int addDomChangeListener(htmlPage : HtmlPage) : void removeDomChangeListener(htmlPage : HtmlPage) : void getDomChangeListener() : DomChangeCollector addDiscoverUrlListeners(listener : DiscoveredUrlListener) : void callDiscoveredUrl(url : String,response : WebResponse) : void callDiscoveredUrl(url : String) : void getUrlValidator() : WebSpiderUrlValidator getFullyQualifiedUrlAsString(relativetUrl : String,page : HtmlPage) : String getFullyQualifiedUrlAsURL(relativetUrl : String,page : HtmlPage) : URL getWebClient() : WebClientWrapper setWebClient(client : WebClientWrapper) : void getOrder() : int compareTo(o : UrlExtractor) : int getTestData() : TestData setTestData(data : TestData) : void getUser() : User setUser(u : User) : void

HrefExtractor

RegexUrlExtractor

OnClickExtractor

FrameExtractor

OnChangeEventExtractor

OnMouseEventExtractor

FormExtractor

HtmlCommentExtractor

JavaScript DOM Modifications Performance Test
Web Browser createElement [ms] #1 #2 #3 #4 #5 269 216 242 202 244 234.6 26 25 26 25 25 25.4 3 3 2 2 2 2.4 createTextNode [ms] 92 97 177 74 106 109.2 8 11 10 11 9 9.8 4 4 4 3 3 3.6 cloneNode [ms] 584 537 607 612 564 580.8 32 29 32 30 30 30.6 14 8 10 10 12 10.8 appendChild [ms] 65 60 62 30 60 55.4 80 83 85 84 81 82.6 11 10 12 12 10 11 insertBefore [ms] 66 62 68 52 55 60.6 95 151 141 105 94 117.2 11 10 13 10 10 10.8 innerHTML [ms] 300 292 314 258 301 293 96 85 99 87 91 91.6 30 33 31 28 32 30.8 Total [ms] 1376 1264 1470 1228 1330 1333.6 337 384 393 342 330 357.2 73 68 72 65 69 69.4

HtmlUnit 2.7

Firefox 3.6.3

#1 #2 #3 #4 #5

Δ

4

Chromium 6.0.432.0

#1 #2 #3 #4 #5

Δ

19

Kevin Denver

15.06.2010

Page 1

Appendix F – HtmlUnit JavaScript Performance (Code)
HtmlUnit Java Benchmark
import java.io.IOException;  import java.net.MalformedURLException;  import java.util.ArrayList;  import java.util.List;  import com.gargoylesoftware.htmlunit.CollectingAlertHandler;  import com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException;  import com.gargoylesoftware.htmlunit.WebClient;  import com.gargoylesoftware.htmlunit.html.HtmlPage;  public class HtmlUnitBenchmark {      public static void main(final String[] args)          throws FailingHttpStatusCodeException,  MalformedURLException, IOException, InterruptedException {          final WebClient webClient = new WebClient();          final List<String> collectedAlerts = new ArrayList<String>();          webClient.setCssEnabled(false);          webClient.waitForBackgroundJavaScriptStartingBefore(9000);          webClient.setThrowExceptionOnScriptError(false);          webClient.setAlertHandler(new CollectingAlertHandler(collectedAlerts));          final HtmlPage page =  webClient.getPage("file:///home/kevin/Desktop/dom­modify.html");                   while(true) {              if (page.getTitleText().equalsIgnoreCase("Finished")) {                  break;              }                           synchronized (page) {                  page.wait(1);              }          }          System.out.println(collectedAlerts);      }  }

HTML Page with embedded JavaScript
... <script>  var timeDiff  =  {      setStartTime:function (){          d = new Date();          time  = d.getTime();      },      getDiff:function (){          d = new Date();          return (d.getTime()­time);      }  }  // Try to force real results  var ret, tmp, str;  var elems = [];  var htmlstr = document.body.innerHTML;  var div = document.createElement("div");  var num = 400;  for (var i = 0; i < 1; i++) {      str += String.fromCharCode( (25 * Math.random()) + 97 );           timeDiff.setStartTime();      for ( var i = 0; i < num; i++ ) {  ret = document.createElement("div");  ret = document.createElement("span");  ret = document.createElement("table"); 

    }      alert("createElement(): " + timeDiff.getDiff());           timeDiff.setStartTime();      for ( var i = 0; i < num; i++ ) {  ret = document.createTextNode(str);  ret = document.createTextNode(str + "2");  ret = document.createTextNode(str + "3");  ret = document.createTextNode(str + "4");  ret = document.createTextNode(str + "5");      }      alert("createTextNode(): " + timeDiff.getDiff());      timeDiff.setStartTime();      document.body.innerHTML = htmlstr;      alert("document.body.innerHTML: " +timeDiff.getDiff());           elems = [];      var telems = document.body.childNodes;      for ( var i = 0; i < telems.length; i++ ) {          elems.push( telems[i] );      }  timeDiff.setStartTime();  for ( var i = 0; i < elems.length; i++ ) {      ret = elems[i].cloneNode(false);  ret = elems[i].cloneNode(true);  ret = elems[i].cloneNode(true); 

ret = document.createElement("tr");  ret = document.createElement("select"); 

    }      alert("cloneNode(): " + timeDiff.getDiff()); 

    timeDiff.setStartTime();  for ( var i = 0; i < elems.length; i++ ) {      document.body.appendChild( elems[i] );      }      alert("appendChild(): " + timeDiff.getDiff());      timeDiff.setStartTime();  for ( var i = 0; i < elems.length; i++ ) {      document.body.insertBefore( elems[i], document.body.firstChild );  }      alert("insertBefore(): " + timeDiff.getDiff());           document.title = "Finished";  }  alert("Finished");  </script>