Software Security and Engineering Practices

(CT-6662)

Chapter 1: Introduction to Engineering Secure

and Safe Systems
Topics to be covered within this chapter
Introduction to Engineering Secure and Safe Systems

Introduction:
• The building and deployment of safe, secure, dependable, resilient, and
reliable software systems remains so important.
• Top class software development shops display high levels of
sophistication, structure, and control throughout their software
development life cycle processes, some accomplishing the highest levels
of capability maturity.
• Perhaps the best-known example of a process improvement approach is
the Capability Maturity Model Integration (CMMI) process introduced by
the Software Engineering Institute(SEI) of Carnegie Mellon University.

• Management effectiveness with respect to NONFUNCTIONAL

Characteristics such as : Security, Safety, Resiliency, Dependability,
Scalability, Reliability, Sustainability, Portability, and interoperability.
 …
.
 …
• Safety-critical software systems are generally more solidly built; that is, they
are more reliable and resilient than security-critical software systems.
• However, even safety-critical systems do not always achieve the highest
standards, particularly with respect to their security attributes.
• As illustrated in Figure above, we see that in general, certain aspects of the
engineering of secure and safe software systems such as interoperability and
portability diminish in the amount of attention paid to them as we enter into
these areas of more limited scope.

Terms/phrases:
- systems engineers, hardware engineers, software engineers
- systems engineering, software engineering,
- security engineering, safety engineering
- software systems engineering, systems software engineering
- software safety engineers, software security engineers
- security-critical software systems, safety-critical software systems
 …
• systems engineers pay relatively little attention to software systems
engineering;
• Systems engineers dealing with entire systems seem to pay little attention to
software considerations.
• software engineers appear to ignore both the guidance that can be gleaned
from systems engineering on the one hand, and the nonfunctional aspects of
software systems, such as security, safety, and resiliency on the other hand.

• One of the Systems Engineer’s first jobs on a project is to establish

nomenclature and terminology that support clear, unambiguous
communication and definition of the system and its functions, elements,
operations, and associated processes ...
• It is important for system engineers, software engineers, and hardware
engineers) to communicate adequately. There is a particular need to improve
communications between security and safety software systems engineers.
 …
What Are Systems?
 …
A System is -
… a collection of components organized to accomplish a specific function or set
of functions.

or
A system is a collection of related or associated entities that together accomplish
... one or more specific objectives.
Or

A system is an integrated composite of people, products, and processes that

provide a capability to satisfy a stated need or objective.

• Software is further split into its functional and nonfunctional characteristics. This
is a common way to divide software characteristics. The usual set of
nonfunctional software characteristics is as follows:
• Security;
• Safety;
• Performance;
• Reliability;
• Compliance.
 …
• The distinction between safety-critical and security-critical software systems:
• Safety-critical software: the software must not harm the world.
• Security-critical software: the world must not harm the software.
• That is to say, software systems that are judged to be safety-critical must not
present a hazard to human life or the environment. This is an outward-
looking perspective.
• On the other hand, security-critical software systems are inward-looking to
the extent that it is necessary to protect the software and any sensitive data
such as nonpublic personal information and intellectual property from
misuse, damage, or destruction from attacks by external or internal persons
or computer applications.
• it is this divergence of viewpoint between software safety engineers and
software security engineers that leads to major software systems engineering
issues, particularly when there is a requirement to combine both safety-
critical and security-critical software-intensive systems into what are
increasingly termed cyber-physical systems.
 …
• Engineering can be defined as follows:
… the creative application of scientific principles to design or develop
structures, machines, apparatus, or manufacturing processes, or works
utilizing them singly or in combination; or to construct or operate the same
with full cognizance of their design; or to forecast their behavior under
specific operating conditions; all as respects an intended function,
economics of operation and safety to life and property.

Or,
Engineering is the designing, testing and building of machines, structures
and processes using maths and science.

• At the bottom of Figure above, is included activities relating to engineering,

namely assurance and management.
- Assurance comprises testing and repairing.
- Management activities related to engineering include monitoring,
analyzing, reporting, and responding.
 …
Deconstructing Systems Engineering
• It is quite confusing to see so many combinations of the words systems, software,
and engineering. What are the precise differences between
- systems engineering,
- software engineering,
- software systems engineering, and
- systems software engineering?
Some of the differences may appear to be obvious, but others are not so obvious.

• When systems are aggregated into systems of systems, what terms should be
used to describe the characteristics of the much more complex combinations of
systems?
The combination of safety-critical systems and security-critical systems, such
as is occurring with the so-called smart grid, produces issues well beyond the
reach of the individual systems.
• The table in Figure bellow, can be used to assist with explaining the definitions
and the relationships among components. For example, we see “systems
engineering” in the first line of the table, “software engineering” in the second
line, and “software systems engineering” in the third line.
• If we were to reverse Columns 2 and 4 in the third line, we would get “systems
software engineering,” which has a completely different meaning from “software
systems engineering.”
 …
.
 …
 Systems software engineering relates to the engineering of systems software.
 Software systems engineering refers to the application of systems engineering
practices to all types of software intensive products.
 Systems software is a category of software that lies below the applications software
layer and performs many of the administrative functions that support the
applications. It includes operating systems and utilities, such as sort algorithms.

What Is Systems Engineering?

• There are many definitions of this term: e.g.
… is an interdisciplinary engineering management process that evolves and
verifies an integrated, life-cycle balanced set of systems that satisfy customer
needs.

Another definition of systems engineering is:

… is an interdisciplinary field of engineering that focuses on how complex
engineering projects should be designed and managed over the life cycle of the
project.
… is an interdisciplinary field of engineering and engineering management that
focuses on how to design, integrate, and manage complex systems over their life
cycles.
 …
And yet another definition is:
… the practical application of the scientific, engineering, and management
skills required to transform a user’s need into a description of a system
configuration that best satisfies the need in an effective and efficient way.

• Clearly these definitions suggest a life cycle process that begins with the
identification of user needs and should end up with one or more systems that
meet those needs.
 …
What Is Software?
• In simple terms, software consists of computer program code, usually written
in high-level computer languages, such as C, C++, and Java. The resulting
source code, of which much software is comprised, is then converted into
code that is understood by computers by interpretation or compilation.

• Software is a collection of programs and data that tell a computer how to

perform specific tasks. Software often includes associated software
documentation.
 …
What Are Software Systems?
• There are many flavors of software, such as end-user applications, systems
software, embedded software, and firmware.
• In Figure bellow, we see a typical traditional software structure where
applications run on systems software, which in turn, runs on hardware.
 …
• let us define software in its various forms and relate it to software
architecture, as illustrated in Figure bellow.
• The terms software and computer program are generally considered to be
synonymous, and most people do not agonize over the specific definitions
used.
• software is essential for the creation of computer hardware consisting of
transistors, other electronic components and the circuitry that are combined
to form integrated circuits.
 …
• the goal of software engineering ... is the creation of software systems ...”

• the definition of software is as follows:

Software is a generic term for organized collections of computer data and
instructions, often broken into two major categories: system software that
provides the basic non-task-specific functions of the computer, and
application software which is used by users to accomplish specific tasks.

• The above definition raises an issue as to whether data should be

incorporated into software or treated separately. It is held here that software
and data are separate entities, which have different roles within software
systems.
• Software has both functional and nonfunctional attributes, as will be
discussed when we examine security and safety engineering.
 …
• firmware (as shown in Figure , above) is defined is as follows:
... firmware is ... often used to denote the fixed, usually rather small,
programs and/or data structures that internally control various electronic
devices. Typical examples of devices containing firmware range from end-
user products such as remote controls ..., through computer parts and
devices [such as] hard disks, keyboards ..., all the way to scientific
instrumentation and industrial robots. Also more complex consumer devices,
such as mobile phones, digital cameras ..., contain firmware to enable the
device’s basic operation as well as implementing higher-level functions.

• Some restrict firmware to program code and data that cannot be changed
once burned into a chip. For example, some define firmware as unalterable
software burned into read-only memory chips.
• ... the combination of a hardware device and computer instructions and data
that reside as read-only software on that device.
• However, today much firmware can be easily updated due to the ready
availability of inexpensive nonvolatile memory devices.
 …
What is Software Systems Engineering?
 …
• Software Systems Engineering is an arm of systems engineering that
addresses the development of complex software-intensive systems.
It involves analyzing, designing, developing, testing, and maintaining a broad
range of software based on specific user needs, while taking into
consideration the quality, time, and budget required.

• We can define the purpose of software engineering as follows:

The goal of software engineering ... is the creation of software systems that
meet the needs of customers and are reliable, efficient, and maintainable.
 …
Nonfunctional Requirements
• These requirements, which include performance, security, safety,
dependability, resiliency and the like, are termed “nonfunctional” because
they are not usually included in the functional requirements of software as
specified by users.
• They are considered to be the responsibility of other areas of expertise, such
as system and network capacity planners, information security professionals,
software safety engineers, disaster recovery and business continuity
planners, and so on.
• In order to have nonfunctional requirements included, project managers
must account for these characteristics explicitly in the various stages of the
software system development life cycle.
  Classical Life Cycle Models(SDLCs)
Examples of SDLC models: (includes classical(traditional) and modern SDCLs)-
 Waterfall Model
 V-Model
 Incremental Model
 RAD Model
 Iterative Model
 Spiral Model
 Prototype model
 Agile Model
• The agile methodologies are a reaction to the failed approach of the
traditional SDLC models which concentrated more on gathering user
requirements first and creating large documentation than actually
delivering the product early and gathering feedback from actual users.
Some of the most popular models based on agile principles are
Scrum, Extreme Programming (XP), and Kanban.
Advantages and Disadvantages of Various Software Development Process
Approaches are shown in the next table.
 …
. Table 3.2
Advantages and Disadvantages of Various Software Development Process Approaches
Year of
Approach Major Advantages Major Disadvantages
Origin *
- Need to know full requirement s up front (adjusting
scope during the life cycle can kill project)
- Many problems not discovered until late in the
- Simple to use
process at which point working software is produced
- Large population of experienced project managers
- Lack of parallel activities
- Easy to manage
- Relatively inefficient use of resources
Waterfall 1970 - Phases are processed and completed one at a time
- High risk and uncertainty particularly if requirements
- Facilitates resource allocation
are likely to change
- Good for smaller projects
- Not good for complex and object-oriented projects
- Not good for larger projects that are long-lasting and
ongoing

- Planning built into process - Complicated and costly

- Risks addressed early and on continuing basis - Risk analysis requires very specific expertise
Spiral 1988 - Software produced early and evolves throughout process - Success of project depends very much on the
-Good for large and mission-critical systems quality of the risk analysis
- Overkill for small projects

- Easy to use - Not flexible

- There are specific deliverables required at the end of each phase - Changing scope is difficult and expensive
V-Model 1986 - Test plans are developed early in the life cycle thereby reducing risk - No early prototypes are produced
- Good for smaller projects with well-defined , easy-to-understand - Not clear how problems found during testing are
requirements handled

- Facilitates setting of clear and manageable objectives

- Greater level of coordination required among
- Reduces overall project complexity and risk
subprojects
- Better specification of requirements by users
- Each phase of an iteration is rigid and inflexible
- Less costly to change scope and requirements than with some other
Iterative - Phases within an iteration do not overlap
1993 approaches
Incremental - Potential problems with system architecture
- Iterations are more easily managed
because of inadequate requirements during early
- Problems are identified and handled at each iteration rather than
stages
waiting to the later stages
 …
.
Table 3.2 (Continued)

Year of
Approach Major Advantages Major Disadvantages
Origin *
- Facilitates understanding of factors, such as look
and feel of user interface
Prototyping - Helps to identify and mitigate risks
May not be cost-effective in certain cases
Proof of 1983 - May be able to reuse some of prototype code
concept - Avoids the cost and difficulty of modifying completed
software

- Comprehensive approach
- Includes aspects not included in other methods such
Overkill for small projects
Unified 1999 as business case
- Quite mature and widely used

- Demonstrable results
- Developers may be more motivated - Problematic for larger projects
Agile 2001 - Better specification of requirements by users - Documentation suffers

- Little to no control over requirements

- Utilization of potentially large number of resources
- Inconsistent or poor documentation
- Encourages personal satisfaction
Open source - Little or no assurance of availability of developers and
- Allows for tailoring and integration
support personnel
- More stable maintenance
- Lack of management control
 Steps in the Software Development Process
• To some degree, all approaches contain the fundamental steps of the
Waterfall approach, which contain the steps indicated in Table bellow.
• The phases of the SSDLC, as listed in the first column of Table , were common
in the era of mainframe computers when computer systems, along with
developers and operators, were isolated with a few (if any) remote users
connected via networks.
 …
Waterfall model
 …
.
 …
Engineering Secure and Safe Systems
Introduction:
• When it comes to building, operating, and decommissioning safe
and secure systems—particularly computer-based software-
intensive systems—practitioners often borrow methods and
practices derived from the discipline of systems engineering, as
well as from safety and security engineering.

Security Versus Safety:

• There is a considerable gap in knowledge and communications
between those who focus on security and those specializing in
safety,
- whether referring to general systems or software-intensive
systems, so that it is still appropriate to treat the disciplines
independently.
 …
• It is important to bring security and safety professionals together
to collaborate at each stage of the system development, testing,
operation, and decommissioning life cycle.

• A simple, yet meaningful, distinction between safety-critical and

security-critical software systems, which is generally applicable
to system was provided in previous slides, for software.
• Here, generalize the definitions as follows:
Safety – the [system] must not harm the world,
Security – the world must not harm the system.
 …
• This difference in viewpoint is illustrated graphically in Figure next, for
safety-critical, security-critical, and combined safety/security-critical systems,
respectively.
• Briefly, the goal of safety-critical systems is containment, whereas the
corresponding emphasis of security-critical systems is protection. Such a
difference in objectives leads to varying and sometimes conflicting criteria for
ensuring that systems are safe when compared with those criteria used to
ensure security.
 …
• There is overlap to some extent between security and safety, as,
for example, in the need for system integrity, but it is the
difference in viewpoint that leads to divergent system designs
and implementations. This divergence makes for much higher
risk relating to combined systems of systems which must be
both secure and safe.
• Another perspective derives from the differences among safety,
security and survivability, which defined as follows:
• Safety: The degree to which accidental harm is prevented, detected, and
reacted to.
• Security: The degree to which malicious harm is prevented, detected, and
reacted to.
• Survivability: The degree to which both accidental and malicious harm is
prevented, detected, and reacted to.
 …
“safety is largely about protecting valuable assets (especially people) from
harm due to accidents,”
whereas
“security is largely about protecting valuable assets (especially sensitive
data) from harm due to attacks,”
and
“survivability is largely about protecting valuable assets (essential services)
from both accidents and attacks.”

• whether the source of an incident is accidental or intentional and malicious.

In both situations, investigators need to determine the root cause of an
incident and take action to prevent the same or similar future events from
doing damage, either internally or externally.
  Software Security
• Fuzzing is a great technique for finding security-critical flaws in any software
rapidly and cost effectively.
• A security software engineers build software security systems like intrusion
detection systems and firewalls to stop leaks, taps, breaches and other cyber
crimes.

• Software security refers to the process of designing, implementing, and

maintaining software in a way that protects it from unauthorized access, use,
disclosure, disruption, modification, or destruction.
• Software security is the concept of implementing mechanisms in the
construction of security to help it remain functional (or resistant) to attacks.

• There are three software security types:

- security of the software itself,
- security of data processed by the software, and
- the security of communications with other systems over networks.
 …
. Software security refers to a set of practices that help protect software
applications and digital solutions from attackers.
• Software security testing can be introduced at various organizations,
starting from research and development (R&D), then entering the test-lab
environment, and finally in the operations (Figure ).
• In R&D, fuzzing can be used both in the early prototyping phase and in the
implementation phase, where the majority of programming takes place.

Figure : Places for using fuzzing: In development, in third-party test laboratories

and in Enterprise operations.
 …
• R&D includes testing, especially tests that take place before system
integration.
• A test lab may contain expensive, dedicated tools for load and
performance testing, as well as fuzzing.
• In operations, various post-deployment techniques are used to increase
software security.

• Vulnerability scanners or security scanners such as Nessus are most

commonly used in a live environment.
• An important criterion for test tools in an operational environment is that
they should not disrupt the operation of critical services.
• Still, penetration testing services are often conducted against live systems,
as they need to validate the real environment from real threats.
• Similarly, fuzzing should be carefully considered for operational
environments. It may be able to find more flaws than when compared to a
test lab, but it is likely to also disrupt critical services.

• Never forget that there is no silver bullet solution for security: Not even
fuzzing can guarantee that there are no flaws left in the software.
 …
 Security Incident
• The main motivation for software security is to avoid security incidents:
- events where someone can compromise the security of a system
through and active attack, and
- where data can be disclosed or destroyed through mistakes made by
people, or due to natural disasters such as floods or tornadoes.

• Accidental incidents may arise when software is misconfigured or a single

bit among massive amounts of data flips due to the infamous alpha-
particles, cosmic rays, or other mysterious reasons and result in the crash
of a critical service.
• Accidental incidents are still quite rare events. The threat related to active
attacks is much more severe.
• The means of conducting the compromise is typically done through an
attack, a script or malicious code that misuses the system, causing the
failure, or potentially even resulting in the attacker’s taking control of the
system.
 …
• An attack is used to exploit a weakness in the system. These weaknesses
when verified to be exploitable are called software vulnerabilities, but can
also be called defects, or flaws in the system.

Example threats to assets include:

• Availability of critical infrastructure components;
• Data theft using various technical means.

Example attacks are the actual exploitation tools or means:

• Viruses and worms that take advantage of zero-day flaws;
• Distributed denial of service (DDoS) attacks.

Weaknesses can be, for example,

• Openness of wireless networks;
• Processing of untrusted data received over the network;
• Mishandling of malicious content received over the network.
 …
 Disclosure Processes
Disclosure is the process of making facts or information known to the public.
• Three different models are commonly used in vulnerability disclosure
processes:
1. No disclosure:
No details of the vulnerability, nor even the existence of the vulnerability,
are disclosed publicly. This is often the case when vulnerabilities are
found internally, and they can be fixed with adequate time and
prioritization.

2. Partial disclosure:
This is the most common means of disclosure in the industry. The
problem with partial disclosure is that hackers can reverse-engineer the
corrections even when limited information is given.

3. Full disclosure:
All details of the vulnerability, including possible exploitation techniques,
are disclosed publicly. In this model, each reader with enough skill can
analyze the problem and prioritize it accordingly.
 …
 Attack Surfaces and Attack Vectors
• Attack surface has several meanings, depending on what is analyzed.
- To some, attack surface means the source code fingerprint that can be
accessed through externally accessible interfaces. This is where either
remote or local users interact with applications, like loading a document
into a word processor, or checking email from a remote mail server.
- From a system testing standpoint, the total attack surface of a system
can comprise all of the individual external interfaces it provides. It can
consist of various network components, various operating systems and
platforms running on those devices, and finally, all client applications and
services.
• Attack vectors- The various routes into a system, whether they are remote
or local, are called attack vectors.
- A local vector means that an attacker already has access to the system to
be able to launch the attack. For instance, the attacker may possess a
user-name and password, with which he or she can log into the system
locally or remotely.
 …
• As examples a local attack vector can consist of any of the following:
1. Graphical user interface (GUI);
2. Command line user interface (CLI);
3. Application Programming interfaces (API);
4. Files;
5. Local network loops (RPC, Unix sockets or pipes, etc.);
6. Physical hardware access.

• Much more interesting interfaces are those that are accessible remotely.
Those are the ones on which fuzzing has traditionally focused.
Most common categories of remote interfaces that fuzzers test are:
- Web applications - Web forms are still the most common attack vector.
- Digital media -
- Network protocols –
- Wireless infrastructure -
 …
 Reasons Behind Security Mistakes
• The core reason is that the fast evolution of communications technologies
has made software overwhelmingly complex.

• Even the developers of the software may not understand all of the
dependencies in the communication infrastructure. Integration into off-
the-shelf frameworks, third-party libraries, components and operating
systems brings along with it unnecessary network services that should be
shut down or secured before deploying the products and services.

• Past experience has shown that all complex communication software is

vulnerable to security incidents: The more complex a device is, the more
vulnerable it usually proves in practice.
 …
 Proactive Security
• For an enterprise user, there are typically two different measures available
for protecting against security incidents: reactive and proactive.
• The main difference between reactive security measures and proactive
security measures is who is in control of the process. In reactive measures,
you react to external events and keep running around putting fires out. In
proactive security measures, you take a step back and start looking at the
system through the eyes of a hacker. Again, a hacker in this sense is not
necessarily a criminal. A hacker is a person who, upon hearing about a
new technology or a product, will start having doubts on the marketing
terms and specifications and will automatically take a proactive mindset
into analyzing various technologies. Why? How? What if? These are just
some of the questions someone with this mindset will start to pose to the
system.
• In short, proactive, predeployment, or preemptive software security is
equal to catching vulnerabilities earlier in the software development life
cycle (SDLC), and catching also those flaws that have not been disclosed
publicly, which traditional reactive security measures cannot detect.
 …
• Most traditional security solutions attempt to detect and block attacks, as
opposed to discovering the underlying vulnerabilities that these attacks
target (Figure ).
• A post-deployment, reactive solution depends on other people to ask the
questions critical for security.

• An example of a reactive solution is a signature based antivirus system.

After a new virus emerges, researchers at the security vendor start poking at
it, trying to figure out what makes it tick. After they have analyzed the new
virus, they will take protective measures, trying to detect and eliminate the
virus in the network or at a host computer
 …
• Another example of a reactive solution is an intrusion detection system
(IDS) or an intrusion prevention system (IPS). These systems look for
known exploits and block them. They do not attempt to identify which
systems are vulnerable or what vulnerabilities exist in software.

• A firewall protects against known attacks by filtering communication

attempts at the perimeter.

• The common thread with most post-deployment security solutions is that

they all target attacks, and not vulnerabilities.

• One could argue that security scanners (or vulnerability scanners) are also
proactive security tools. However, security scanners are based on known
attacks and exhibit the same problems as other reactive security solutions.
A security scanner cannot find a specific vulnerability unless the
vulnerability is publicly known.

• A proactive solution will look for vulnerabilities and try to resolve them
before anyone else learns of them and before any attacks take place.
 …
 Security Requirements
• We have discussed fuzzing and its uses, but the truth is not all software is
security-critical, and not all software needs fuzzing. Introducing fuzzing
into development or deployment processes needs to be based on the
requirement set for the system.
• Typical and perhaps the most common subset of security requirements or
security goals consists of the following:
- confidentiality,
- integrity, and
- availability.

• Fuzzing directly focuses on only one of these, namely availability, although

many vulnerabilities found using fuzzing can also compromise
confidentiality and integrity by allowing an attacker to execute malicious
code on the system.
  Model most appropriate for Secure Software
Development
.
.
…
.
…
  Lifecycle Assurance of Systems and Software
• Introduction
• Everything we do these days involves system and software technology:
Cars, planes, banks, restaurants, stores, telephones, appliances, and
entertainment rely extensively on technology.
• The operational security of these software-intensive systems depends on
the practices and techniques used during their design and development.
• Many decisions made during acquisition and development have an impact
on the options for security once systems are deployed.
• Quality is important, but simply reducing software defects is not sufficient
for effective operational security.

• Lifecycle processes must consider the security-related risks inherent in the

operational environments where systems are deployed.
 …
• Today’s tools and existing products allow almost anyone to create a
software-based system that meets its functional requirements, but critical
skills and practices are needed to ensure secure deployment results.

What do We mean by Lifecycle Assurance?

• Much of the information protection in place today is based on principles
established by Saltzer and Schroeder - They defined security as
“techniques that control who may use or modify the computer or the
information contained in it” and described three main categories of
concern: confidentiality, integrity, and availability (CIA).

• As security problems expanded to include malware, viruses, Structured

Query Language (SQL) injections, cross-site scripting, and other
mechanisms, those problems changed the structure of software and how
it performs.
 …
Those working with deployed systems refer to this enhanced security need as
cyber security assurance, and those in the areas of acquisition and development
typically reference software assurance.

What do We mean by Software Assurance?

 “ Implementing software with a level of confidence that the software
functions as intended and is free of vulnerabilities, either intentionally or
unintentionally designed or inserted as part of the software, throughout the
lifecycle”

 “The level of confidence that software functions as intended and is free of

vulnerabilities, either intentionally or unintentionally designed or inserted
as part of the software throughout the lifecycle”

 “The planned and systematic set of activities that ensure that software
lifecycle processes and products conform to requirements, standards, and
procedures”