SUSTAINABLE SOFTWARE ENGINEERING

Felipe Albertao: felipe@alumni.cmu.edu

Abstract
Information Technology is playing a critical role in bringing social and environmental sustainability.
However, the social and environmental impact of the IT project itself is rarely accounted or managed.
Thus, a huge amount of waste is generated: Billions of dollars are lost annually in mismanaged software
projects, while e-waste increases at astonishing rates. Despite these facts, there is little to no awareness in
the industry as to the impact of the software development process in society and the environment. Hence,
there are no standards or guidelines to effectively minimize such impacts. With this in mind, this paper is
for anyone involved in an IT project and it introduces a methodology to measure and incrementally
improve the sustainability of software projects that can be easily adopted without impacting the overall
product development cycle. This methodology is composed of a set of metrics that are used to assess the
Sustainability Performance of a given software-based project, with the goal of promoting the use of
Software Engineering, Project Management and IT Adoption practices that enable economic, social and
environmental benefits. The paper also illustrates how this methodology was used in a software project
developed for NASA.

Acknowledgements
This paper was produced in December/2004, as part of the Masters of Science in Information Technology
– Software Engineering Program at Carnegie Mellon Silicon Valley (http://sv.cmu.edu).
This paper would not have been possible without the guidance of Dr. Illah Nourbakhsh (NASA Ames
Research Center), Dr. Edward Katz and Todd Sedano (Carnegie Mellon Silicon Valley). Appreciation is
extended to the Carnegie Mellon’s faculty for the extremely valuable feedback and support, in special Dr.
Martin Griss and Dr. Jamie Dinkelacker. And very special thanks to the Tag Team who made this project
happen: Melanie Gabriel, Curtis Huang, Stephen Nelson, Thuan Ngo and Ryan Prichard.
This work was inspired by the pioneering concept of “Environmental Accounting” invented and
implemented by Eckart Wintzen at Origin in the early nineties1, and IfPeople's ResponsibleIT2 standard.

This work is licensed under the Creative Commons Attribution-Share Alike 3.0 United States License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-sa/3.0/us/ or send a letter to
Creative Commons, 171 Second Street, Suite 300, San Francisco, California, 94105, USA.

Sustainable Software Engineering, Felipe Albertao 1 / 16

Problem Statement
Human progress has been putting the environment under an extreme pressure, and many studies have
demonstrated that humanity is currently consuming more resources than the planet’s capacity to renew
them. And while the global economy has boomed, over a billion people are living in extreme poverty,
lacking basic needs and surviving under $1 a day.
Over the past two decades the concept of Sustainable Development was introduced to address those
concerns. The best accepted definition for Sustainability is “Development which meets the needs of the
present without compromising the ability of future generations to meet their own needs”3.
Typically the economy, society and environment are viewed as separated
and isolated aspects of human activities. The key to sustainable Economy
development is to consider those aspects as interdependent and
intrinsically related, by increasing efficiency while minimizing the impact Society
on the environment and at the same time enabling economic growth and
social progress. Environment

Sustainability and Engineering

For engineers worldwide, interest in sustainability is being driven by a number of factors4:
• Mounting evidence that forms of technological and economic development are physically or
socially unsustainable.
• Social changes in the past decades have lead to a more complex and demanding role for engineers
in society. It is no longer acceptable for engineers to focus exclusively on technical issues.
• Engineers must recognize and balance the social, economic and environmental impacts of their
technical decisions.
Some industry standards and guidelines have been used to address those factors in engineering activities,
most notably APEG’s SMS, ISO 14000 and LEED for Architecture and Civil Engineering. At the
Academia level, a number of institutions are researching the subject and developing innovative programs
to incorporate Sustainability into the current engineering curricula, such as Carnegie Mellon Department
of Civil and Environmental Engineering (Introduction to Sustainability Engineering), and Stanford
University.

Sustainability and Software Engineering

Information Technology is playing a critical role in bringing social development to underserved
communities by improving communication, access to information, education, health and knowledge
sharing. At the same time IT addresses environmental concerns by reducing the need for travel and
transport, and its ability to immaterialize products (ex: e-mail versus postal mail).
Although the impact of software is usually not directly perceived due to its virtual characteristic, there is
no doubt that the process used to create a software product impacts society in general, for example:
• Economic Impact: A recent study found that software bugs cost the U. S. economy an estimated
$59.5 billion annually5, and the well-known CHAOS Report6 estimated annual losses of $81
billion in cancelled software projects and additional $59 billion in completed projects that
exceeded their budget.
• Social Impact: Offshoring of software development services to countries like India are causing US
workers to lose their jobs, but at the same time it gives employment to half a million workers and
it generates more than $10 billion a year in India. In addition, a study from McKinsey Global
Institute shows that the long-term gains in the U. S. economy from offshoring are actually larger
than the benefits for India7.

Sustainable Software Engineering, Felipe Albertao 2 / 16

 • Environmental Impact: In general more than 1,000 materials are used in a computer and many of
them are highly toxic8. Software is usually developed according to the Moore’s Law, requiring
more processing power and latest hardware after each release. This cycle limits the computer’s
lifetime and increases e-waste in the ecosystem (pollution caused by disposed components and
toxic material). In 2004 over 315 million computers will become obsolete in U. S. alone9.
Despite those facts, there is no awareness in the industry on the impact of the software development
process in the society and the environment, and neither there are standards or guidelines to effectively
minimize such impacts:
• Typically the word “sustainability” is used in the software field referring to “maintenance”. A
recent paper by CMU SEI researchers actually recognizes “sustainability” as a more general term
associated with aspects of software evolution across the entire life cycle10. However it focuses in
the space of legacy systems, and broader societal aspects are not considered.
• The IEEE standard 1219-1998 (Software Maintenance Standard) is a guideline for software
evolution, but it does not address the economical, societal and environmental aspects of a
Software Engineering project in a holistic manner.
• More recently a company called IfPeople published a set of sustainability and social responsibility
guidelines for IT business processes11, however it is focused in the technology adoption rather
than software production.

Sustainable Software Engineering, Felipe Albertao 3 / 16

Proposed Solution
Sustainability should be considered as an orthogonal aspect of the entire system, similar to aspects such as
Quality and Security. Those aspects belong to the entire product life-cycle and not to a specific phase. As
such, the Sustainability aspect should be introduced (ideally) at the early stages of software development.
However, it would not be realistic to expect software development teams to change their entire
development model in order to introduce Sustainability in the process. Realistically, Sustainability
concerns can be addressed so long as it does not disrupt the model currently in place.
The proposed solution is composed of a set of techniques and metrics that will be used to assess the
Sustainability Performance of a given software-based project, with the goal of promoting the use of
Software Engineering, Project Management and IT Adoption practices that enable economic, social and
environmental benefits. The assessment is executed at the end of the product cycle, and a reflection report
is generated using the data found during the assessment phase. The reflection leads to goals for
incremental improvement and suggested actions to achieve those goals in the next software life cycle.
This approach provides a tool for Sustainability Performance improvement that can be easily adopted
without impacting the overall product development cycle, while it helps to increase the awareness of
sustainable development issues in the software industry.
Software
Life-Cycle Release

Improve Assess
Establish improvement goals the Sustainability
and actions. Performance

Reflect
Analyze assessment results

• Assessment Phase: In this phase the team compiles the data collected throughout the project in order
to measure the performance of the project against a set of properties that leverage an economical,
social and environmental benefit (modifiability, reusability, etc…).
• Reflection Phase: In this phase the team analyzes the results of the assessment phase and suggests
improvements for each one of the properties.
• Goal Improvement Phase: In this phase the team evaluates the suggestions for improvement resulted
from the Reflection Phase and selects a few that can be realistically improved during the next release
cycle based on the following criteria:
1. Customer value
2. Feasibility / likelihood for adoption
3. Level of improvement
The goal is not to implement sustainability at once, but instead to enable continuous improvement
throughout many release cycles. In fact, any effort that aims to improve sustainability must be itself
sustainable, and methodologies that embrace continuous improvement (such as Six Sigma and CMMI)
have a proven record of success exactly because of the continuous improvement aspect, which enables
organizations to increase efficiency in an ongoing and sustainable fashion.

Sustainable Software Engineering, Felipe Albertao 4 / 16

Sustainability Properties and Metrics
The Sustainability Performance of a given software project is measured and analyzed against a set of
properties, which if improved will bring a certain economical, social and environmental benefit.
Property12 Benefits
Economical Social Environmental
Development-Related Properties
Properties that impact the development process
Modifiability Minimizes Enables system to be Minimizes environmental
The ability to development and continuously adapted waste through less effort
introduce changes support costs. to meet societal in producing and
quickly and cost demands. maintaining existent
effectively.
system.
Reusability Accelerates time-to- Enables the production Minimizes environmental
Level in which system market. of new products with impact through less effort
components can be less effort. in producing system.
reused in other
systems.

Portability Increases potential Reduced cost for Minimizes e-waste by

Ability of the system to market and system’s technology adoption, extending the lifetime of
run under different lifetime. by minimizing user’s old hardware.
computing dependency on latest
environments.
technology.
Supportability Increased customer Vendor’s Indirect benefit:
System’s ability to be base due to reduced independence minimizes resources
easily configured and support costs. increases the product required to provide
maintained after usability and thus support (transportation,
deployment.
accessible to a larger physical material, etc…)
population.
Usage-Related
Properties that impact the user at run-time
Performance Improves Minimizes dependency Minimizes e-waste by
The time required by productivity. on latest technology. extending hardware
the system to respond lifetime. Minimizes
to user requests. energy consumption
through less computer
usage time.
Dependability Minimizes support Increases societal Indirect benefit:
The ability of a system and maintenance productivity. Minimizes energy waste.
to function as expected costs.
at any given time.

Usability Increases customer Contributes to the Indirect benefit: Less

Features that enable a satisfaction. digital inclusion, by waste of resources used in
system to be user- Minimizes support eliminating barriers training (books, training
friendly. costs. (learning curve) and rooms, energy, etc…).
making system more
accessible to a broader

Sustainable Software Engineering, Felipe Albertao 5 / 16

 number of users.
Accessibility Increases potential Enables technology to Indirect benefit.
The system’s ability to market and/or minorities, elderly,
serve people audience. people with
regardless of location, disabilities, non-
experience,
background, or the English speaking
type of computer communities, and
technology used.
illiterate population.
Increases multicultural
awareness and
provides equal
opportunities.
Process-Related
Properties that impact project management
Predictability Minimizes risks of Increases team’s Optimizes use of
The team’s ability to budget overrun. conditions of work environmental resources.
accurately estimate (avoid long work-
effort and cost up- hours).
front.

Efficiency Maximizes product Minimizes effort Optimizes use of

The overhead of value. waste. environmental resources.
production processes
over the bottom-line
value perceived by the
customer.

Project’s Indirect benefit. Indirect benefit. Reduces fuel consumption

Footprint and emissions, office-
Natural resources and space utilization, and
environmental impact maximizes use of shared
used during software resources.
development.

The list below outlines a set of simple metrics that will be used to quantify each one of the properties.
MODIFIABILITY
In order to measure modifiability we will use a number of OO design quality metrics13 that will result in a
single number per package, representing how much the package is prepared to accommodate change.
First we calculate the Instability of each package, which represents the ability of changing a package
without impacting the other packages:
Instability = Efferent Coupling / (Afferent Coupling + Efferent Coupling).
Where:
• Afferent Coupling: The number of classes outside a package that depend on classes inside the
package.
• Efferent Coupling: The number of classes inside a package that depend on classes outside the package.
A number close to zero means that the package is “stable” (i. e.: the package has more incoming
dependencies than outgoing dependencies), and therefore a change in the package is likely impact other

Sustainable Software Engineering, Felipe Albertao 6 / 16

packages as well. In the other hand, a number close to one means that the package is “instable”, with more
outgoing dependencies than incoming dependencies.
However, the goal is not to make an entire system stable or instable: A maximally stable system would be
unchangeable, and in fact some packages must be flexible in order to accommodate change. Therefore we
need to first identify which packages should be stable or instable by calculating the Abstractness of each
package, and then establishing a relationship between Stability and Abstractness, leading us to another
metric:
Distance From Main Sequence = | Abstractness + Instability – 1 |
Where:
• Abstractness: Number of abstract classes and interfaces / Total number of types in a package
A Distance From Main Sequence close to 0 has the “right” balance between concrete and abstract classes
in proportion to its dependencies. A number “far” from zero indicates that the package requires some
redesign or refactoring.
Note: The Distance From Main Sequence is the final metric that will be considered for analysis of this
property. All the other metrics mentioned above are used to calculate the final Distance From Main
Sequence.
REUSABILITY
Stable packages are usually be the backbone of a system, and a simple way to measure reusability is to
find how abstract the stable packages are, which is accomplished by comparing the Abstractness of each
package against its Instability. Abstract packages (i. e.: Abstractness close to one) should also be stable (i.
e.: Instability close to zero).
Note: There are other techniques to measure Design Reusability, most notably the Price and Demurijan
method14.
PORTABILITY
The metric for this property is the Estimated System Lifetime, or the estimated number of years that the
hardware required by the system reached the market.
SUPPORTABILITY
We will use two simple metrics that can be obtained by user behavior observation:
• Estimated Installation Time: User’s time to install product without assistance
• Support Rate: Number of user questions that required assistance / Number of minutes system was
used by user
PERFORMANCE
In most cases performance is relative and usually defined by the initial system requirements. For the
purpose of this methodology, performance will be measured by the Relative Response Time, calculated by
number of use cases that have an unacceptable response-time per user click / total use cases.
DEPENDABILITY
Three quality metrics are used calculate dependability: Defect Density shows how many defects the
system has according to its size. However, this metric alone does not show if the testing process was
efficient (it is possible to have a low defect density simply because the system was not tested enough), and
therefore we need to measure the efficiency and effectiveness of the quality process.
Defect Density: Number of not-fixed defects / KLOC

Sustainable Software Engineering, Felipe Albertao 7 / 16

Testing Efficiency: Total defects found / Number of days of testing
Testing Effectiveness: Total defects found and removed / Total defects
USABILITY
The metrics of this property are obtained through a Usability Testing performed using a target user as a
subject, using four of the Nielsen’s selected usability attributes15:
• Learnability: Number of minutes to accomplish first critical task without assistance / Total
minutes / Number of minutes system was used by user
• Effectiveness: Number of tasks completed / Total number of tasks
• Error Rate: Number of tasks which were completed but deviated from normal course of action /
Total number of tasks
• Satisfaction: Subjective user’s feedback
ACCESSIBILITY
A number of standards were established to introduce accessibility in technology products, most notably:
• World Wide Web Consortium’s Web Accessibility Initiative, which defines guidelines for
introducing accessibility in web pages.
• USA Government’s Section 508 (Rehabilitation Act), which requires that all software sold to
Federal Agencies must follow this standard.
In the other hand the standards mentioned above are effort demanding, and for the purpose of this
methodology we will measure accessibility by using a subjective score between 1 to 3, being 0=Non
Existent, 1=Not Adequate, 2=Acceptable, 3=Adequate16:
• Support for Motor Impaired Users (ex: keyboard commands in addition to mouse commands)
• Support for Visual Impaired Users (ex: screen enlargement for people with low vision, or screen
color changes for people with color blindness)
• Support for Blind Users (ex: text-to-speech features)
• Support for Users with Language and Cognitive Disabilities (ex: simple language)
• Support for Illiterate Users (ex: icon-based interface)
• Internationalization Support
• Localization Support
PREDICTABILITY
The project’s Estimation Quality Rate can be easily measured by calculating the budget hours / total effort
hours.
EFFICIENCY
The Project Efficiency is evaluated in terms of effort towards deliverables that add direct value to the
customer such as code, manuals, etc… and can be measured by calculating the effort hours spent on
product-related artifacts / number of total effort hours.
PROJECT’S FOOTPRINT
Although it is extremely difficult to accurately measure the actual footprint such as fuel consumed and
office resources (paper, energy, etc…), we can obtain a reasonable measure by using the total number of

Sustainable Software Engineering, Felipe Albertao 8 / 16

meetings and comparing it against the number of virtual meetings during the project. Nevertheless, it is
possible that a project poorly managed would have a low score simply due to the low number of total
meetings, and therefore we also need to measure the total number of meetings against the project size:
• Conference Calls Rate = Number of Virtual Meetings / Number of Total Meetings
• Meetings Rate = Number of Total Meetings / Number of Total Project Weeks

Future Work
By no means does this set of metrics try to encompass the entire spectrum of indicators related to
Sustainability (such as environmental and social indicators, for example). The goal of this methodology is
to simply offer a starting point for further evolution, by having a small but relevant set of metrics.
Future improvements of this methodology should use more qualitative metrics, especially in measuring
Accessibility.
This methodology does not define thresholds because the goal is to allow a continuous improvement of
the software throughout its releases, and therefore a metric does not have a “good” or “bad” meaning per
se. However it would be beneficial to include some kind of baseline for benchmarking purposes, which
can be accomplished by applying this methodology in a number of projects of different sizes.
In order to guarantee continuous sustainability improvement, the Goal Improvement Phase should define a
method of tracking the accomplishment of the goals from the previous release cycle.
A tool that automatically calculates the sustainability metrics at the end of a project’s release cycle would
increase the likelihood of the broad adoption of this methodology.

Conclusion
Given that the very first principle of the Software Engineering Code of Ethics is to act consistently with
the public interest17 and also considering the huge impact of the software industry in the society, we must
expand our understanding of how software-related activities affect human development at local and global
levels, in addition to creating practices that improve the long-term sustainability of software projects.

Sustainable Software Engineering, Felipe Albertao 9 / 16

Sustainability Report – NASA PER 2 Project
This report presents the results of a study in Software Sustainability for the NASA PER Project, using the
methodology described in the previous pages.

Background
Personal Exploration Rover (PER)18 is a project conducted by the Robotics Institute
at Carnegie Mellon University with support from NASA, which developed a robot
that resembles the Mars Exploration Robot (MER) landed on Mars. The goal is to
educate and inspire high-school students about the role of rovers in space science.
The PER 2 Project was executed by a team of 6 CMU-Silicon Valley Masters of
Software Engineering students during 3 months, with the goal of implementing new PER Robot
features to enhance the interface and improving the usability of the initial
Iconic Programming Interface software application implemented by a
previous team of CMU students (PER 1 Project).
The Iconic Programming Interface is a Java-based application that allows
the user (high-school students) to define “missions” based on the robot’s
body and head actions. The application allows the user to program
movements and transmit them to the robot via wireless networking.
Note: This study considers only the PER 2 Project and its software
components. Sustainability aspects of the robot (such as power Iconic Programming Interface
consumption, etc…) are not considered.

Sustainability Assessment and Reflection

Data Sources
The majority of the data used in this assessment was collected using the following tools, methods and
deliverables:
• Metrics, an Eclipse plug-in that automatically calculates software metrics.
• Usability Study, performed during the PER 2 Project using high-school students which used PER
in both “Play time”-mode (i. e.: no guidance) and also in guided-mode where the subjects were
asked to follow a certain script based on PER use cases. The study also allowed the subjects to
freely discuss the positive and negative experiences in using the application. For the purposes of
this report, the document “PER Product Design Document” was used, in addition to the analysis
of the video recorded during one of the sessions.
• The Statement of Work Document provided the project budget hours.
• The Test Plan was used to obtain the number of testing days.
• Issuezilla (bug tracking system) was used to obtain the number of bugs.
• The team’s Meeting Minutes (on the Wiki Server) were used to obtain the number of meetings
and conference calls.

Sustainable Software Engineering, Felipe Albertao 10 / 16

Sustainability Metrics
MODIFIABILITY
Package Distance From Main Sequence
PER 0.333
PER.infrastructure.api.component 0.233
PER.infrastructure.impl.component 0.500
PER.infrastructure.impl.loader 0.500
PER.lang.api 0.300
PER.lang.api.action 0.041
PER.lang.api.action.sensor 0.100
PER.lang.api.rover 0.125
PER.lang.api.runtime 0.082
PER.lang.api.script 0.136
PER.lang.api.sequence 0.393
PER.lang.impl 0.333
PER.lang.impl.action 0.421
PER.lang.impl.action.sensor 0.333
PER.lang.impl.rover 0.400
PER.lang.impl.runtime 0.400
PER.lang.impl.script 0.400
PER.lang.impl.sequence 0.571
PER.lang.impl.runtime.action 0.152
PER.map 0.333
PER.rover 0.000
PER.ui 0.173
Note: Unit test packages not included
From the table above we can conclude that the PER’s modifiability would be somehow benefited if the
packages under PER.lang.impl were redesigned by either minimizing dependencies to this package or by
introducing more interfaces and abstract classes. However, the greater majority of the packages are under
0.500, which indicates that the overall design is balanced.
Note that the design team decided to separate interfaces and concrete classes into different packages (api
and impl), which is not a recommended practice because it negatively impacts design metrics such as
Abstractness. As such, a refactoring to move the interfaces and concrete classes to the same packages is
highly recommended.

Sustainable Software Engineering, Felipe Albertao 11 / 16

REUSABILITY
Package Abstractnes Instability
s
PER 0.000 0.667
PER.infrastructure.api.component 0.667 0.100
PER.infrastructure.impl.component 0.000 0.500
PER.infrastructure.impl.loader 1.000 0.500
PER.lang.api 0.500 0.200
PER.lang.api.action 0.909 0.132
PER.lang.api.action.sensor 1.000 0.100
PER.lang.api.rover 1.000 0.125
PER.lang.api.runtime 0.875 0.207
PER.lang.api.script 1.000 0.136
PER.lang.api.sequence 0.500 0.107
PER.lang.impl 0.000 0.667
PER.lang.impl.action 0.000 0.579
PER.lang.impl.action.sensor 0.000 0.667
PER.lang.impl.rover 0.000 0.600
PER.lang.impl.runtime 0.000 0.600
PER.lang.impl.script 0.000 0.600
PER.lang.impl.sequence 0.000 0.429
PER.lang.impl.runtime.action 0.333 0.818
PER.map 0.000 0.667
PER.rover 0.000 1.000
PER.ui 0.027 0.800
Note: Unit test packages not included
Overall the results show that PER has a good relation between Abstractness and Instability, meaning that
highly abstract packages are also stable, and the instability is concentrated in the packages with low
Abstractness.
The few exceptions are due to the fact that most interfaces and concrete classes were separated into
different packages, a design deficiency as described in the modifiability analysis.
PORTABILITY
• Estimated System Lifetime: 10 years.
PER is based on Java 1.4.x, which requires at least Windows 98 or Red Hat 9. Both operational systems
require at least an Intel 486 processor, which was released in 1989 with broader adoption in early 1990s.
Although PER was never tested in such platforms, it would be safe to assume that it would run using at
least a Pentium processor, which was available in 1993. Therefore the application can theoretically run in
hardware as old as 10+ years, which is a reasonable lifetime from a sustainability standpoint.
SUPPORTABILITY
• Estimated Installation Time: PER does not have an automated installation process, but the latest
release includes a step-by-step manual targeted to a non-technical audience. It is estimated that a
non-technical user would take roughly 15 minutes installing PER by following the manual. It is
highly recommended that PER improves the installation process in the next releases.
• Support Rate: The subject required assistance 6 times in a 45 minutes session, resulting a rate of
0.43, reasonable from a sustainability standpoint.
PERFORMANCE
• Relative Response Time: All use cases presented less than one second response-time during the
Usability Study, which is acceptable.

Sustainable Software Engineering, Felipe Albertao 12 / 16

DEPENDABILITY
• Defect Density: 3 not-fixed defects / 5.79 KLOC = 0.518
• Testing Efficiency: 17 defects found / 6 testing days (not including development tests) = 2.833
• Testing Effectiveness: 14 defects found and removed / 17 total defects = 0.823
Although not critical, it would be beneficial if the Testing Effectiveness rate were increased in the next
release in order to minimize costs associated with product support in the field.
The Defect Density is low, and the Testing Efficiency shows that a good amount of testing was carried out,
attesting the validity of the Defect Density metric.
Note: Unit test packages were not considered.
USABILITY
• Learnability: 0.266
the subject took 12 minutes to accomplish a critical task without assistance, within a total of 45
minutes session.
• Effectiveness: 1
all tasks were completed.
• Error Rate: 0.5
3 deviated tasks / 6 tasks
• Satisfaction: Subject was overall satisfied with the user experience.
ACCESSIBILITY
• Features for Motor Impaired Users: 0 – PER does not have keyboard shortcuts
• Features for Visual Impaired Users: 2 – Although not directly implemented by PER, Java has
support for color-blindness and low-vision users.
• Features for Blind Users: 0 – PER does not have audio capabilities
• Features for Users with Language and Cognitive Disabilities: 3 – PER is iconic-based and uses
simple language.
• Features for Illiterate Users: 2 – PER is iconic-based, but it does require user to read (ex: PER
asks for the distance to move forward)
• Internationalization Features: 0 – English is the only language available for PER
• Localization Features: 2 – PER recently introduced a feature to convert centimeters to inches and
vice-versa, and it does not use other locale-sensitive information (such as date).
PREDICTABILITY
• Estimation Quality Rate: 1
The actual effort time was not measured in PER 2 Project, however it is estimated that the project was
concluded within the budget defined at the beginning of the project because the team reached all
milestones on time.

Sustainable Software Engineering, Felipe Albertao 13 / 16

EFFICIENCY
• Project Efficiency: 0.458
635 hours in deliverables that add direct value to the customer.
1386 estimated total effort hours.
Almost 50% of the time invested in the project was towards bottom-line deliverables. This is considered a
reasonable number, considering that a balanced amount of discipline increases the performance of the
other metrics (specially those related to quality).
Note 1: Due to the lack of actual effort time, the estimated time was used to calculate this metric.
Note 2: All development activities (requirements, design, development, testing) were considered as
deliverables that add direct value to the customer.
PROJECT’S FOOTPRINT
• Conference Calls Rate: 0.39
9 virtual meetings / 23 total number of meetings
• Meetings Rate: 1.91
23 total number of meetings / 12 weeks of project
The Conference Calls Rate shows that the team made a good use of virtual meetings. Although it is
important to physically hold a certain number of meetings for team building purposes, it would be possible
to increase this rate given the resources CMU Silicon Valley makes available for remote students
(conference calling system and virtual meetings capabilities – Live Meeting).
The Meetings Rate shows that the team held a reasonable amount of meetings (almost 2 per week), so the
Conference Calls Rate metric is reliable.

Sustainable Software Engineering, Felipe Albertao 14 / 16

Sustainability Improvement Goals
According to the Assessment and Reflection phases, the following suggestion for improvements were
identified:
Suggestions for Property Custom Adoption Level of
Improvement er Value Feasibilit Improveme
y nt
Refactoring Modifiability Low Low High
Estimated Installation Supportability High High High
Time
Testing Effectiveness Dependability Medium Medium Low
Learnability Usability High High High
Support for Blind Users Accessibility Medium Low Low
Conference Calls Rate Project’s Footprint Low High High

According to the criteria above, the 3 Sustainability Improvement Goals for the next release are:
1. Improve PER’s Supportability and decrease estimated installation time to 5 minutes,
by integrating an automatic installer method to PER, such as InstallShield, InstallAnywhere or
open source installers such as LiftOff, Packlet, IzPack and many others.
2. Improve PER’s Usability and increase the Learnability level to 0.13,
by introducing small but effective improvements to the UI. For example, the usability study
has shown that the user took many minutes to find out that he should start using the
application by clicking on the “add” button. Even though all buttons except the “add” are
grayed-out, still the user got confused because the “add” icon uses some gray color. Therefore
one of the ways that learnability could be improved changing the color of the “add” icon.
3. Reduce PER’s Footprint and increase Conference Calls Rate to 0.60,
by simply using more conference calling and virtual meeting systems instead of physical
meetings.

Sustainable Software Engineering, Felipe Albertao 15 / 16

1
Origin’s Original, Wired Magazine, Issue 4.11, November 1996.
2
IfPeople web site: http://www.ifpeople.net
3
Our Common Future, World Commission on Environment and Development, 1987
4
Adapted from Sustainability in Professional Engineering and Geoscience: A Primer, Professional Engineers
and Geoscientists of BC, Canada.
5
Software Errors Cost U.S. Economy $59.5 Billion Annually, National Institute of Standards and Technology,
2002.
6
The CHAOS Report (1994), The Standish Group, 1994. Note: A more recent CHAOS report has shown little
improvement over the 1994 report.
7
Exploding The Myths of Offshoring, The McKinsey Quarterly, 2004.
8
Just Say No To E-Waste, Silicon Valley Toxics Coalition.
9
Electronic Product Recovery and Recycling Baseline Report, National Safety Council’s Environmental Health
Center, 1999.
10
Measuring Software Sustainability, IEEE Computer Society.
11
ResponsibleIT: Integrating Technology with Social Responsibility, IfPeople
12
Most properties are in fact Quality Attributes .See Software Architecture In Practice - CMU SEI Series, Len
Bass, Paul Clements, and Rick Kazman.
13
OO Design Quality Metrics: An Analysis of Dependencies, Robert Martin, 1994.
14
Analyzing and Measuring Reusability in Object-Oriented Designs, Margaretha W. Price, Steven A. Demurjian,
Sr., 1997
15
Usability Engineering, Jakob Nielsen, 1994.
16
List partially adapted from the Web Accessibility Initiative, Ball State University
17
Software Engineering Code of Ethics and Professional Practice, IEEE-CS/ACM Joint Task Force, 1999.
18
The Personal Exploration Rover, The Robotics Institute at Carnegie Mellon University