Software Engineering

Trends & Challenges ahead

S V Ramanamurthy

1
 The Context
Software has engulfed our lives today. It is
hard to imagine a world without software.
It is becoming more and more pervasive
day-by-day. Software content in virtually
every industry and service will continue to
grow dramatically.

2
 What is Software Engineering
• What is Software?
– Programs +
– Documentation +
– Data structures
• What is Engineering?
– Not Art (Subjective)
– Not Science (Theory)
– Application of Science and Art to the betterment
of Mankind

3
 Is Software Engineering? Or Art?
Before 1967 100% art Very much dependent on the programmer. No
coding standards, No formal procedures for
other stages
1967- 1995 70%-80% art Structured Programming, waterfall model,
RDBMS
1995-2000 60% - 70% art Iterative Life Cycle, OO, Internet, Web
Technologies, Distributed Software Systems
2000- ??? Software Project Management, RUP, Matured
OO methodologies

4
 History of SW Engineering
• 1968 – NATO Conference
• Masterpiece Engineering
• Report of Defense Science Board – 1994
• Chaos Report by Standish Group – 1995
• Patterns of Software Systems Failure &
Success – Jones 1996

5
 Essence of these studies
• Poor Requirements Definition
• Inadequate SW Process Management
• Lack of Integrated Product Teams
• Ineffective subcontractor management
• Lack of change management – increasing scope
• Too little attention to software architecture
• Poorly defined, inadequately controlled
interfaces
• Focus on innovation rather than cost and risk
• No flexibility in the process
6
 The Progress – Initial attempts
• Development of structured programming
languages (e.g., ALGOL, Pascal, and C).
• Development of design methods and principles
(e.g., information hiding, top-down refinement,
functional decomposition).
• Definition of software lifecycles (e.g., waterfall,
incremental development, prototype-based)
• OOAD Methodologies (RUP, Java, c++, c#, VB
etc.)
7
 The Progress - Tools
• Incremental Compilers, JIT Compilers
• Platform Independent Environments (JVM,
CLR etc.)
• Analysis and Design Tools (UML tools)
• Integrated Development Environments (IDEs)
– NetBeans etc.
• Automated Testing Tools (Rational ROBOT
etc.)
8
 The Progress - Process
• Development Process
– Personal Software Process (PSP)
– Agile Programming
– Xtreme Programming
• Management Process
– Iterative Life Cycle
– Software Project Management
– Configuration & Change Management
• CMM
– KPAs
– Process Maturity measurement

9
 Is it Enough?
What are the results?
What are the challenges still?
What is the future of Software Engineering?

10
 Is Software Engineering? Or Art?
Before 1967 100% art Very much dependent on the programmer. No
coding standards, No formal procedures for
other stages
1967- 1995 70%-80% art Structured Programming, waterfall model,
RDBMS
1995-2000 60% - 70% art Iterative Life Cycle, OO, Internet, Web
Technologies, Distributed Software Systems
2000- 40-50% art Software Project Management, RUP, Matured
OO methodologies

11
 Growth in Success Rate
• The 2009 edition of the annual CHAOS report from The
Standish Group, which researches the reasons for IT project
failure in the United States, indicates that project success
rates have increased to 32 percent of all projects. That's
more than a 100-percent improvement from the success
rate found in the first study in 1994.

• Asked for the chief reasons project success rates have

improved, Standish Chairman Jim Johnson says, "The
primary reason is the projects have gotten a lot smaller.
Doing projects with iterative processing as opposed to the
waterfall method, which called for all project requirements
to be defined up front, is a major step forward."

12
 Chaos Report 2009

Year Year Year Year Year Year Year Year

2009 2006 2004 2002 2000 1998 1996 1994
Successful 32% 35% 29% 34% 28% 26% 27% 16%
Challenged 44% 19% 53% 15% 23% 28% 40% 31%
Failed 24% 46% 18% 51% 49% 46% 33% 53%

13
The Challenges - Reliability

14
 What is reliability?
• Probability of failure-free operation for a
specified time in a specified environment for
a given purpose
• This means quite different things depending
on the system and the users of that system
• Informally, reliability is a measure of how
well system users think it provides the
services they require
Input/output mapping
Inputs causing
erroneous
Input set I outputs
e

Program

Erroneous
outputs
Output set Oe
 Failure classification

Failure class Description

Transient Occurs only with certain inputs
Permanent Occurs with all inputs
Recoverable System can recover without operator intervention
Unrecoverable Operator intervention needed to recover from failure
Non-corrupting Failure does not corrupt system state or data
Corrupting Failure corrupts system state or data
 SW Failures
• After more than three years and $170 million expected to be
spent developing the Virtual Case File, the FBI has not provided a
clear timetable or prospect for completing the VCF. (2005 – Report
of IG)
• The Mars Climate Orbitor crashed in September 1999 because of a
"silly mistake": wrong units in a program.
• Death resulted from inadequate testing of the London Ambulance
Service software.
• Several 1985-7 deaths of cancer patients were due to overdoses of
radiation resulting from a race condition between concurrent
tasks in the Therac-25 software.
• Errors in medical software have caused deaths. Details in B.W.
Boehm, "Software and its Impact: A Quantitative Assessment,
"Datamation, 19(5), 48-59(1973).
• A China Airlines Airbus Industrie A300 crashes on April 26, 1994
killing 264. Recommendations include software modifications.
18
 Costs of increasing reliability
Cost

Low Medium High Very Ultra-

high high
Reliability
The Challenges - Security

20
 Information Security
• Confidentiality
• Integrity
• Availability
• Accountability
• Assurance

21
 Confidentiality
• Spying and Theft updated
– Once spying was person against person, country
against country
– Today spies sit on our fiber-optic cables and wi-fi
networks
– They steal data and information without
breaking any glass
– Keeping data confidential is one core mission of
Information Security

22
 Data Integrity
• Incorrect Information
– Wrong information is worse than no information
– Once users lose confidence in the accuracy of
information, they will never rely on it
– Wrong information can cause havoc
– Maintaining data integrity is also a core mission
of information security

23
 Availability
• Information Security doesn’t mead locking
everything down
• If people don’t have the information they
need they cannot do their jobs
• Information security is about balancing the
access to information and risk of damage
• Third core mission of Information security is
to make information available when needed

24
 The Challenges –
Size and Complexity

25
 Forces in Software
Functionality
Cost Compatibility

Capacity Fail safe

Availability Fault tolerance

Performance Throughput

Technology churn Resilience

The challenge over the next 20 years will not be speed or cost or performance;
it will be a question of complexity. Bill Raduchel, Chief Strategy Officer, Sun Microsystems

Our enemy is complexity, and it’s our goal to kill it. Jan Baan
 Largest Software

Project SLOC Developers Notes

Lucent 5ESS Switch *100m 5,000? 20 year cycle

Windows Vista 50m 2,000
Red Hat Linux 7.1 30m** 3,700? + 200
companies
Windows XP 40m ??
Visual Studio 40m 700 30 teams, 11
labs
MS Office 30m 70??***

27
 Challenges – Size & Complexity
• In the relatively near future, systems requiring
over 1 billion LOC will begin to emerge
– Consider the interfaces for a billion LOC system
• to the outside world
• to other interoperable systems
• to the Internet (or its successor), and
• to the millions of internal components that must all
work together to make this computing monster
operate successfully.
– How do we manage the project, the teams, the communication among
software engineers?

28
 Challenges - Size & Complexity
From a testing perspective:
– Is there a viable way to validate requirements?
– Is there a reasonable way to conduct technical
reviews and communicate their results across
hundreds of teams?
– Is there a reliable way to ensure that all of the
interfaces will allow information to flow properly?
– Is there a realistic strategy for conducting tests,
tracking errors, and performing debugging?
– Is there an efficient way to perform regression tests?
– How do you conduct performance tests, stress tests,
and security tests when the system is huge?
29
 Challenges - Size & Complexity
• People and Teams
– As systems grow in size, teams grow in number,
geographical distribution, and culture
– As systems grow in complexity, team interfaces
become pivotal to success
– As systems become pervasive, teams must
manage emergent requirements
– As systems become more open, what is a team?

30
The Challenges –
New Technologies

31
 Technology “Evolution”
• Ray Kurzweil argues that
technological evolution is similar to
biological evolution, but occurs at a
rate that is orders of magnitude
faster.
– “ … the more capable methods
resulting from one stage of
evolutionary progress are used to
create the next stage.” [Kurweil, R., The Singularity is
Near, 2005]

32
 Pervasive Computing (PvC)
• Concepts such as ambient intelligence,
context-aware applications, and ubiquitous
computing—all focus on integrating software-
based systems into an environment far
broader than anything to date
• Open-world software—software that is
designed to adapt to a continually changing
environment ‘by self-organizing its structure
and self-adapting its behavior.” *
* Baresi, L., E. DiNitto, and C. Ghezzi, “Toward Open-World Software: Issues and Challenges,” IEEE
Computer, October 2006.

33
 Pervasive Computing (PvC)
• First stage (PvC-1) [today]
– Device mobility and ad hoc networking
– Simple context awareness
– Soon: smart objects implemented in devices that have the potential to
communicate with one another
• Second stage (PvC-2) [over the next decade]
– Mobile user profiles that can be recognized by other objects
– Smart objects will respond to other objects based on situational
characteristics
• Testing issues
– Considerable environmental variation
– Complex communication issues
– Adaptive processing requirements

34
 Cloud Computing
• "Cloud Computing is a paradigm in which
information is permanently stored in servers
on the Internet and cached temporarily on Cloud clients
clients that include desktops, entertainment
centers, table computers, notebooks, wall
computers, handhelds, sensors, monitors,
etc.” IEEE Internet Computing
• Provides software as a service (SaaS)
• Device and location independence enables Cloud architecture
users to access systems regardless of their
location or device
• Multi-tenancy enables sharing of resources
(and costs) among a large pool of users
• Demands reliability, scalability, security,
sustainability (Green IT) Legacy
Infrastructure

35
 Virtualization
• Types of Virtualization
–Hardware Virtualization
–Virtual Machine Virtualization
–Application Virtualization

36
The Challenges –
Trends

37
 Software Engineering Trends
Ivar Jacobson identifies four major
trends:
 Reusable components—stop reinventing the
wheel
 More up-front testing—test-driven
development
 Platform transparency— reduce today’s
programming complexity
 UML “all the way down”— model-driven
development
38
 SE Trends—Process
• Agility
• Adaptability
• Collaboration
• Communication
• ROI

39
 SE Trends—Methods
• In RE
• In design
• In languages
• In testing
• In SCM/SQA

40
 SE Trends—Tools
• Project mgmt.
• Modeling
• Programming
• Testing
• Maintenance
• SCM
• SEEs

47
 Tools Trends—Testing
• As test-driven development approaches gain
momentum, tools for selecting test cases based
on requirements and/or models must be
developed. In addition, the software engineering
community needs better tools for:
– Launching selected tests in multiple device
environments
– Assessing the test outcome (sometimes called a “test
oracle”)
– Evaluating the impact of failures and determining the
root cause of the failure
– Determining whether the testing regime is sufficient
48
 The Road Ahead

• “We're driving faster and

faster into the future, trying
to steer by using only the
rear-view mirror.” Marshall
McLuhan
• “…the best way to predict the
future is to invent it.” Alan Kay
49