Building Security in Software:

Testing, Requirements and Options

Lecture 3
Dr Md Israfil Biswas
Manchester Metropolitan University
22.02.23
 Secure Software
• Confidentiality Software Security
• Disclosure of information to only intended parties • Security of Operating System
• Integrity • Security of Client Software
• Determine whether the information is correct or not
• Security of Application Software
• Data Security
• Privacy • Security of System Software
• Data Protection • Security of Database Software
• Controlled Access • Security of Software Data
• Authentication • Security of Client Data
• Access to Authorized People
• Availability • Security of System Data
• Ready for Use when expected • Security of Server Software
• Non Repudiation • Security of Network Software
• Information Exchange with proof
 Why Security Testing?
• For Finding Loopholes Approaches to Secure Testing
• For Zeroing IN on Vulnerabilities • Study of Security Architecture
• For identifying Design Insecurities • Analysis of Security Requirements
• For identifying Implementation Insecurities • Classifying Security Testing
• For identifying Dependency Insecurities and
• Developing Objectives
Failures
• For Information Security • Threat Modeling
• For Process Security • Test Planning
• For Internet Technology Security • Execution
• For Communication Security • Reports
• For Improving the System
• For confirming Security Policies
• For Organization wide Software Security
• For Physical Security
 Security Testing Techniques
• Penetration Testing
• OS Hardening • Simulating Attack from a Malicious Source
• Configure and Apply Patches • Includes Network Scanning and Vulnerability Scanning
• Updating the Operating System • Simulates Attack from someone Unfamiliar with the
System
• Disable or Restrict unwanted Services and
• Simulates Attack by having access to Source Code,
Ports Network, Passwords
• Lock Down the Ports • Port Scanning and Service Mapping
• Manage the Log Files • Identification and locating of Open Ports
• Install Root Certificate • Identification of Running Services
• Protect from Internet Misuse and be Cyber• Firewall Rule Testing
Safe • Identify Inappropriate or Conflicting Rules
• Protect from Malware • Appropriate Placement of Vulnerable Systems behind
Firewall
• Vulnerability Scanning • Discovering Administrative Backdoors or Tunnels
• Identify Known Vulnerabilities • SQL Injection
• Scan Intrusively for Unknown Vulnerabilities • Exploits Database Layer Security Vulnerability
• Unexpected Execution of User Inputs
 Security Testing Techniques …
• Cross Side Scripting • Network Scanning
• Injecting Malicious Client Side Script into Web Pages • Identifying Active Hosts on a network
• Persistent, Non-Persistent and DOM based • Collecting IP addresses that can be accessed over
Vulnerabilities the Internet
• Parameter Manipulation • Collecting OS Details, System Architecture and
Running Services
• Cookie Manipulation
• Collecting Network User and Group names
• Form Field Manipulation
• Collecting Routing Tables and SNMP data
• URL Manipulation
• HTTP Header Manipulation
• Password Cracking
• Collecting Passwords from the Stored or
• Denial of Service Testing Transmitted Data
• Flooding a target machine with enough traffic to • Using Brute Force and Dictionary Attacks
make it incapable • Identifying Weak Passwords
• Command Injection • Ethical Hacking
• Inject and execute commands specified by the • Penetration Testing, Intrusion Testing and Red
attacker Teaming
• Execute System level commands through a • File Integrity Testing
Vulnerable Application • Verifying File Integrity against corruption using
 Security Testing Techniques …
• War Dialing • Random Mutation Testing
• Using a Modem to dial a list of Telephone Numbers • Bit Flipping of known Legitimate Data
• Searching for Computers, Bulletin Board System and • Byte stream Sliding within known Legitimate
Fax Machines Data
• Wireless LAN Testing • Session Hijacking
• Searching for existing WLAN and logging Wireless • Exploitation of Valid Computer Session
Access Points • Exploitation of the Web Session control
• Buffer Overflow Testing mechanism
• Gain unauthorized access to the Web Server
• Overwriting of Memory fragments of the Process,
Buffers of Char type • Phishing
• Masquerading as a trustworthy entity in an
• Format String Testing electronic communication
• Supplying Format type specifiers in the Application
• Acquiring usernames, passwords and credit
input
card details
• Random Data Testing • URL Manipulation
• Random Data Inputs by a Program • Make a web server Deliver inaccessible web
• Encoded Random Data included as Parameters pages
• Crashing built-in code Assertions • URL Rewriting
 Security Testing Techniques …
• IP Spoofing Conclusion
• Creating Internet Protocol (IP) packets • Analyze potential Threat and its Impact
with a forged source IP address • Complete Security Testing may not be
Feasible
• Packet Sniffing
• Collect Information to Secure Business
• Capture and Analyze all of the Environment
Network traffic
• Should be done as early as possible in the
• Virtual Private Network Testing Development life Cycle
• Penetration Testing • Should be able to identify the Security
• Social Engineering Requirements
• Psychological Manipulation of People • Have Specific understanding of the Various
Processes
• Divulging confidential information
• Should provide Recommendations to
overcome Weakness
 Software Vulnerably
• Renewed interest • More than half of the vulnerabilities are due to
• “idea of engineering buffer overruns
software so that it • Others such as race conditions, design flaws are
continues to function equally prevalent
correctly under malicious
attack”
• Existing software is riddled
with design flaws and
implementation bugs
• “any program, no matter
how innocuous it seems,
can harbor security holes”
 Software Security
• It is about • Three trends
• Understanding software-induced • Connectivity
security risks and how to manage them • Inter networked
• Leveraging software engineering • Include SCADA (supervisory
practice, control and data acquisition
• thinking security early in the software systems)
lifecycle • Automated attacks, botnets
• Knowing and understanding common • Extensibility
problems • Mobile code – functionality
• Designing for security evolves incrementally
• Subjecting all software artifacts to • Web/Os Extensibility
thorough objective risk analyses and • Complexity
testing • M lines of code
• It is a knowledge intensive field • Add to that use of unsafe
languages (C/C++)
 Security problems in Software
Bug
Buffer overflow: stack smashing
Buffer overflow: one-stage attacks
Buffer overflow: string format attacks
Race conditions: TOCTOU
Unsafe environment variables
Unsafe system calls (fork(), exec(),
system())
Incorrect input validation (black list vs.
white list
Flaw
Method over-riding problems
Solution …
(subclass issues)
Compartmentalization problems in Three pillars of security
design
Privileged block protection failure
(DoPrivilege())
Error-handling problems (fails open)
Type safety confusion error
Insecure audit log design
Broken or illogical access control
Pilar I
Applied Risk Management
(role-based access control [RBAC]
over tiers)
Signing too much code

• Architectural risk analysis

• Defect • Sometimes called threat modeling or
• implementation and design
vulnerabilities security design analysis
• Can remain dormant • Is a best practice and is a touchpoint
• Bug
• An implementation level software • Risk management framework
problem
• Considers risk analysis and mitigation as
• Flaw
• A problem at a deeper level a full life cycle activity
• Bugs + Flaws
• leads to Risk
 Three pillars of security
Pillar II: Software Security Touchpoints
• “Software security is not security software”
• Software security
• is system-wide issues (security mechanisms and design security)
• Emergent property
• Touchpoints in order of effectiveness (based on experience)
• Code review (bugs)
• Architectural risk analysis (flaws)
• These two can be swapped
• Penetration testing
• Risk-based security tests
• Abuse cases
• Security requirements
• Security operations
• Many organization
• Penetration first
• Is a reactive approach
• CR and ARA can be switched however skipping one solves only half of the problem
 Three pillars of security …
Pillar III: Knowledge
• Involves
• Gathering, encapsulating, and
sharing security knowledge
• Software security knowledge
catalogs
• Principles
• Guidelines
• Rules
• Vulnerabilities
• Exploits
• Attack patterns
• Historical risks
• Can be put into three categories

• Prescriptive knowledge
• Diagnostic knowledge
• Historical knowledge
 Risk Management Framework (RMF)
• Continuous risk management
RMF Five Stages:
process
• Identify and keep track of
• RMF occurs in parallel with SDLC activities
risks overtime and a software
project unfolds
• High-level approach to
iterative risk management
that is deeply integrated
throughout the SDLC
• Identify, rank, track, and
understand software security
risks as it changes over time
 Stage 1:
Understand Business Context
• Risk management
• Occurs in a business context
• Affected by business motivation
• Key activity of an analyst
• Extract and describe business goals – clearly
• Increasing revenue; reducing dev cost; meeting SLAs; generating high return on
investment (ROI)
• Set priorities
• Understand circumstances
 Stage 2: Identify the business & technical
risks
• Business risks have impact
• Direct financial loss; loss of reputation; violation of customer or regulatory
requirements; increase in development cost
• Severity of risks
• Should be capture in financial or project management terms
• Key is –
• tie technical risks to business context
 Stage 3: Synthesize and rank the risks
• Prioritize the risks alongside the business goals
• Assign risks appropriate weights for resolution
• Risk metrics
• Risk likelihood
• Risk impact
• Number of risks mitigated over time
 Stage 4: Risk Mitigation Strategy
• Develop a coherent strategy
• For mitigating risks
• In cost effective manner; account for
• Cost Implementation time
• Completeness Impact
• Likelihood of success

• A mitigation strategy should

• Be developed within the business context
• Be based on what the organization can afford, integrate and understand
• Must directly identify validation techniques
 Stage 5: Carry out Fixes and Validate
• Execute the chosen mitigation strategy
• Rectify the artifacts
• Measure completeness
• Estimate
• Progress, residual risks
• Validate that risks have been mitigated
• Testing can be used to demonstrate
• Develop confidence that unacceptable risk does not remain
 RMF - A Multi-loop
• Risk management is a continuous process
• Five stages may need to be applied many times
• Ordering may be interleaved in different ways
• Risk can emerge at any time in SDLC
• One way – apply in each phase of SDLC
• Risk can be found between stages
• Level of application
• Primary – project level
• Each stage must capture complete project
• SDLC phase level
• Artifact level
• It is important to know that RM is
• Cumulative
• At times arbitrary and difficult to predict
 Seven Touchpoints

1. Code Review (Tools)

2. Architectural Risk
Analysis
3. Penetration Testing
4. Risk-Based Security
Testing
5. Abuse Cases
6. Security Requirements
7. Security Operations
 Touchpoints: Black and White Hat
• Mix black and white hat approaches • Both Hats
• Black hat (defense) • Risk-based security Testing
• Penetration testing • Abuse cases
• White hat (offense)
• Code review
• Architectural risk analysis
• Security requirement
• Security operations
Push Left Rule
• More economical to find software defects early in
the lifecycle
• Penetration testing at the end of lifecycles creates
reactive strategy
• Penetrate and patch approach
• “Push Left” rule gets us to the top two Cost of fixing defect at each stage
touchpoints very early in the SDLC
 Code review
Code Review Tools:
• All software project have code
• Focus is on implementation bugs
• Center software assurance activities • Essentially those that static analysis can find
around the code • Security bugs are real problems – but
architectural flaws are just as big a problem
• Large amount of security problems are
• Code review can capture only half of the
cause by simple bugs that can be spotted problems
in code • E.g.
• Buffer overflow • Buffer overflow bug in a particular line of code
• strcpy() in C • Architectural problems are very difficult to find
by looking at the code
• Code review is boring, difficult, and tedious • Specially true for today’s large software
• Analysis that practice code review often • Static analysis tools examine the text of a
are familiar with “get done, go home” program without executing it
• Favorable to manual audits (faster)
• Reviewer start out motivated and diligent • Evaluate programs more frequently
but eventually lose focus • Reach hard-to-reach states and dark corners
of code
 Code review
• Taxonomy of coding errors
• Input validation and representation
• Some source of problems
• Metacharacters, alternate encodings, numeric representations
• Forgetting input validation
• Trusting input too much
• Example: buffer overflow; integer overflow
• API abuse
• API represents contract between caller and callee
• E.g., failure to enforce principle of least privilege
• Security features
• Getting right security features is difficult
• E.g., insecure randomness, password management, authentication,
access control, cryptography, privilege management, etc.
 Code review…
Key Characteristics of a Tool Design
• Source code analysis tool must have six key • Software security requires the ability
characteristics
to think like a bad guy
1. Must be designed for security
2. Support multiple tiers • Exploiting software is not an
3. Be extensible exercise in standard-issue QA
4. Be useful for security analysts and
developers • The knowledge base built into a tool
5. Support existing development process is an essential deciding factor
6. Make sense to multiple stakeholders

Support Multiple Tiers

• Modern software applications are rarely written in a single language or platform
• Most business critical systems are highly distributed
• Automated security analysis software must support each of these languages and platforms
• A tool that can only analyze one or two is unsatisfactory
 Code review …
Be Extensible:
• Security problems evolve, grow, and mutate
• No one technique or set of rules will ever perfectly detect all security vulnerabilities
• Good tools need a modular architecture that supports multiple kinds of
techniques
• “One size fits all” approach to security is doomed to fail

Be Useful for Security Analysts and Developers:

• The best analysis tools cannot automatically fix security problems
• Best automated tools
• Assist analysis and developers to focus on their role
• Find and fix security problems efficiently
• Educate users about good programming practices
 Code review …
Support Existing Development Process
• Requires seamless integration with build processes and IDEs
• The tool must properly inter operate with existing compilers on various
platforms
• Good tools both integrate into existing build process and also coexist
with and support analysis in familiar development tools

Make Sense to Multiple Stakeholders

• Security analysis software tool also needs to support business
• Views for release managers
• Views for development managers
• Allow executive to compare using metrics and support release decisions
 Architectural Risk Analysis (ARA)
• Architectural risk analysis
is a risk management
process that identifies
flaws in a software
architecture and
determines risks to
business information
assets that result from
those flaws.
• Design flaws
• about 50% of security problem
• Can’t be found by looking at code
• A higher level of understanding
required
• Risk analysis
• Track risk over time
• Quantify impact
• Link system-level concerns to probability
and impact measures
• Fits with the RMF
 ARA within RMF
2 Measurement and reporting
Identify
the Business Technical
Risk expertise
Artifact Analysis
1 4 5
Understand Business Define the Risk
Synthesize and
the Business Context Mitigation
Rank the Risks
context Strategy
3 Identify
the Technical
Risk
Artifact Analysis
7 6
Validate the
Initiate process Fix the artifacts
artifacts
improvement Validation loop
 ARA process
• Attack resistance analysis
• Steps
• Identify general flaws using secure design literature and
checklists
• Knowledge base of historical risks useful
• Map attack patterns using either the results of abuse case or a
list of attack patterns
• Identify risk based on checklist
• Understand and demonstrate the viability of these known
attacks
• Use exploit graph or attack graph

- Note: particularly good for finding known problems

 ARA process
• Ambiguity analysis • Weakness analysis
• Discover new risks – creativity requried • Assess the impact of external software
• A group of analyst and experience helps – dependencies
use multiple points of view • Modern software
• Unify understanding after • is built on top of middleware such
independent analysis as .NET and J2EE
• Uncover ambiguity and inconsistencies • Use DLLs or common libraries
• Need to consider
• COTS
• Framework
• Network topology
• Platform
• Physical environment
• Build environment
 Software Penetration Testing
• Software penetration testing is the • Most used today
practice of testing a computer • Currently
system, network or web • Outside->in approach
application to find vulnerabilities • Better to do after code review and ARA
that an attacker could exploit • As part of final preparation acceptance
regimen
• One major limitation
• Almost always a too-little-too-late
attempt at the end of a development
cycle
• Fixing things at this stage
• May be very expensive
• Reactive and defensive
 Software Penetration Testing …
• A better approach
• Penetration testing from the beginning and throughout the life cycle
• Penetration test should be driven by perceived risk
• Best suited for finding configuration problems and other
environmental factors
• Make use of tools
• Takes care of majority of grunt work
• Tool output lends itself to metrics
• Eg.,
• fault injection tools;
• attacker’s toolkit: disassemblers and decompilers; coverage tools
monitors
 Risk based security testing
• Testing must be
• Risk-based
• grounded in both the system’s architectural reality and the
attacker’s mindset
• Better than classical black box testing
• Different from penetration testing
• Level of approach
• Timing of testing
• Penetration testing is primarily on completed software in operating
environment; outside->in
 Risk based security testing
• Security testing
• Should start at feature or component/unit level testing
• Must involve two diverse approaches
• Functional security testing
• Testing security mechanisms to ensure that their functionality is
properly implemented
• Adversarial security testing
• Performing risk-based security testing motivated by understanding
and simulating the attacker’s approach
 Abuse cases
• Abuse case defines an interaction between • Brainstorming is the most
an actor and the system that results in practical method of creating
harm to a resource associated with one of abuse case
the actors, one of the stakeholders, or the • Combine security and reliability
system itself experts with system designers
• Also called misuse cases (relies heavily on experience)
• Experts ask questions to identify
• Think like an attacker possible weakness
• What motivates an attacker? • Often, formal methods are
• Stealing money unnecessary in the real world
• Obtaining greater access level
• Create general havoc
• Spy on spouse
• How can the attacker accomplish its goal?
 Touchpoint Process: Abuse Case
Development
• Start with requirements, standards uses
cases, and list of attack patterns
• Identify your threat or actor
• Knowing the enemy that is likely to attack
is a prerequisite
• Use information gathered to create abuse
case
• Creating attack model
• Cycle through known attacks and determine if
same attack applies to your system
• Select those attack patterns and build cases
around those patterns
• Include anyone who can gain access to system
• Process is in Microsoft’s STRIDE model
• Creating anti-requirements
• Important to think about
• Things that you don’t want your
software to do
• Requires: security analysis +
requirement analysis
• Anti-requirements
• Provide insight into how a
malicious user, attacker, thrill
seeker, competitor can abuse your
system
• Considered throughout the lifecyle
• indicate what happens when a
required security function is not
included
Abuse Activity Model

37
 Software Security Best Practice
• Developers' best efforts on applying • Touchpoints are designed to
touchpoints may fail be carried out by software
• Lack of security domain knowledge security specialist in tandem
• Unfamiliar with real world attacks with development teams
• Knowledge is acquired from years of
experience in the field • Software security is a team
• Information security staff have years of effort
responding to attacks
• Have studied application vulnerabilities and
their resulting attacks profiles in minute
detail
• Few information security professionals
are developers
• Rare to find information security
professionals directly involved in
development projects
 Security Requirements and Operations
• Requirements: Abuse cases • Security requirements
• Design: Business risk analysis • Difficult tasks
• Should cover both overt functional
• Design: Architectural risk analysis security and emergent characteristics
• Test Planning: Security testing • Use requirements engineering approach

• Implementation: Code review • Security operations

• Integrate security operations
• System Testing: Penetration testing • E.g., software security should be
• Field System: Deployment and integrated with network security
operations
Security Requirements and Operations …

• Software security is everyone responsibility

• Critical touchpoint to implement first
• Code review (bugs)
• Architectural risk analysis (flaws)
• Account for 50% of security problem
• Utilize continuous risk management process