You are on page 1of 6

International Journal of Advanced Computer Science, Vol. 2, No. 2, Pp. 73-78, Feb. 2012.

Evaluation of Quality Assurance Factors in Agile Methodologies


Mehreen Sirshar & Dr. Fahim Arif
Abstract__ The agile software development claims to improve the quality of the software products. This claim has increases its usage in the new industry of the software and information technology. But, due to its immature nature, very few frameworks for the evaluation of quality exist. The proposed research has introduced a tool for the evaluation of quality in agile software development. Various factors addressing the quality have been evaluated among different phases in various practiced methodologies of agile software development. The proposed tool has provided a new platform for application of various types of testing in agile methodologies. It has opened a new horizon of research in software industry Manuscript
Received: Revised: Accepted: Published:

The paper is organized into five sections. Section contains literature review. Section 3 includes quality assurance (QA) factors being evaluated in the research. Section 4 introduces tools and applications on agile methodologies. Section 5 discusses the analysis .Section 6 draws conclusion

Keywords
Agility, Software Quality assurance, Quality factors, Testing Techniques

2. Literature Review
Many definitions have been proposed for defining quality and agile development. The literature survey reveals very few definitions regarding agile and quality together. A. Defining Quality Many researchers agree on the consensus that quality is conformance to customer needs. Many researches enhance this view to define quality as characteristics bear by the software product which must reflect the needs specified by the customer. Meyer [1] defines software quality according to an adapted number of quality factors as defined by McCall [2], which are; correctness, robustness, extendibility, reusability, compatibility, efficiency, portability, integrity, verifiability, and ease of use etc. Pressman agrees with Crosby and defines quality as conformance to explicitly stated functional requirements, explicitly documented development standards, and implicit characteristics that are expected of all professionally developed software [3]. Sommerville [4] defines software quality as a management process concerned with ensuring that software has a low number of defects and that it reaches the required standards of maintainability, reliability, portability and so on. Pfleeger [5] agrees with Garvin who views quality from five different perspectives namely: The transcendental meaning that quality can be recognized but not defined, User view meaning that quality is fitness for purpose, Manufacturing meaning that quality is conformance to specification, Product view meaning that quality is tied to inherent product characteristics The value-based view meaning that quality depends on the amount the customer is willing to pay for the product. A. Defining agility In literature, agility is commonly refers to the readiness to motion. The Agile manifesto was published by group of software engineers and consultants in 2001. It is stated as

1. Introduction
The traditional methodologies for the software development focused on waterfall approach. After achieving the freezing point of requirements the system was developed in increments. However, volatile nature of market, rapid change in customer requirements ,dynamic infrastructure of business organization , exhaustive need for short time deliverables and highly variations in pace of development technologies and tools added the competitive pressure to sustain in the market. As a result many organizations adapted agile method for software development. Practical implementation of software development is highly dependent on the practices followed by different organizations. The introduction of Agile as software methodology in literature was introduced in late 90's.After that several methodologies have been proposed. With the above mention factors the conformance to quality standards was also challenging task. The agile development promises to provide quality assurance. In fact agile development is popular for quality delivery of the project in small deliverables. The element of the proposed research also targets the issues of quality in agile development.
This work was supported by the National University of Science and Technology , Rawalpindi ,Pakistan M.Sirshar, with National University of Science and Technology, Software Engineering MS Research scholar (m.sirshar@gmail.com) F.Arif, with National University of Science and Technology,Rawalpindi Pakistan(fahim_mcs@nust.edu.pk)

74

International Journal of Advanced Computer Science, Vol. 2, No. 2, Pp. 73-78, Feb. 2012. TABLE 1: QUALITY ASSURANCE FACTORS AND THEIR DEFINITIONS [20] Definitions The ability of a system to perform according to defined specification. Appropriate performance of a system under cases not covered by the specification. This is complementary to correctness. A system that is easy to adapt to new specification. Software that is composed of elements that can be used to construct different applications Software that is composed of elements that can easily combine with other elements. The ability of a system to place as few demands as possible to hardware resources, such as memory, bandwidth used in communication and processor time. The ease of installing the software product on different hardware and software platforms Releasing the software before or exactly when it is needed by the users. How well the software protects its programs and data against unauthorized access How easy it is to test the system. The ease with which people of various backgrounds can learn and use the software. The ease of changing the software to correct defects or meet new requirements Low utilization of resources, lower response time and mean time of failure and recovery define the performance of the system. The ability of a system to be completed within a given budget.

Quality Parameters Correctness Robustness Extendibility Reusability Compatibility Efficiency Portability Timeliness Integrity Verification and validation Ease of use Maintainability Performance Cost effectiveness

follows [6]-[7]: Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan

Ambler [8] considers agile quality to be a result of practices such as effective collaborative work, incremental development, and iterative development as implemented through techniques such as refactoring, test-driven development, modeling, and effective communication techniques. The definition of quality from view of agile development is still dependent on organizational practices and the agile methodology being followed. From the agile perspective quality has been defined as McBreen [9] defines agile quality assurance as the development of software that can respond to change as the customer requires it to change. This implies that the frequent delivery of tested, working, and customer-approved software at the end of each iteration is an important aspect of agile quality assurance. B. Quality assurance factors in agility Table 1, defines different quality attributes followed in agile development. Each quality factor is evaluated in different agile methodologies. Some organizations follow mixed methodologies to achieve different quality requirements. Analysis of methodologies reveals some practices which are suggested to be followed to achieve high quality. All agile methodologies do not address all of the quality assurance factors .Several methodologies address multiple factors. However combinations of methodologies are followed to ensure the required quality factors. Quality factors are non functional attributes of the system which are implicitly stated .Therefore, they are most difficult to define

and evaluate. Compatibility Platform independence can be achieved by multiple agile methodologies especially open source development methodologies. It can also be achieved by following OO (object Oriented) design in local practices of an organization. Cost Effectiveness Iterative and incremental nature helps to achieve cost effectiveness by delivering the prioritized requirements in small increments. Correctness Many agile methodologies involve active involvement of the customer therefore correctness in requirements is handled and the correctness in the behavior of the desired system is also achieved. Ease of use Active involvement of the customers increases the understanding of the system thus increasing in ease of use of the system when designing its interface. Interface is highly important when web layouts are designed. However, this may create problems when open source development is followed but it is easily achieved through scrum, XP (extreme programming), RUP (rational unified process) and JAD (joint application development) techniques. Extendibility OOP design, COTS and modular development help to cater changing requirements with greater flexibility the system can be easily extended by achieving low coupling. Almost all of the agile methodologies follow OOP designs to achieve it. Maintainability Modular design and compatibility between interfaces help to achieve maintainability. Feature driven development and crystal methodologies play important role to achieve maintainability. Performance Sprint, OO designs, code ownerships and common class activities help to increase performance in almost all of the existing agile methodologies. Portability Distributed computing and web service design has
International Journal Publishers Group (IJPG)

Mehreen Sirshar et al.: Evaluation of Quality Assurance Factors in Agile Methodologies.

75

increased portability in agile methodologies. Integrity Low coupling among design increments help to achieve integrity of the data as well as the integrity of system Reusability OO (object Oriented) designs are based on the principle of reusability. Furthermore, organizations following agile development keep track of all design increments to make them available in short time boxes. Robustness Robustness is ensured by following development standards however robustness is implicit. It cannot be explicitly stated. Timeliness Iterative development and short cycles are followed to achieve timeliness. Validation and verification Unit testing, integration testing and regression testing and interface testing are commonly followed methodologies to ensure validation and verification. Further more user involvement ensures verification with each deliverable.

3. Tool for evaluation of quality for agile methodologies


This section provides a brief overview of existing agile methodologies, quality factors in agile development and different quality assurance factors. The proposed tool provides assurance factors. The proposed tool provides different testing techniques that can be applied to different quality attributes supported by specific methodology through quality assurance factors. One such tool was proposed by Hou et al [10] based on comparative study between waterfall methodologies and existing agile methodologies. The tool had certain limitations found in analysis as mentioned below: Singles out two main aspects of quality management namely; quality assurance and verification and validation. Overlooks other vital techniques used in agile processes to achieve higher quality management. And is mainly based on Extreme Programming which on it own is not a complete representation of agile methodologies. To address these limitations another tool was proposed by Mankandla et al [11] .But the tool failed in its applicability to all agile methodologies. It focuses only on two of the widely used agile methodologies. In the subsequent development effort similar tool has been proposed in this research paper to address the limitations provided by proposed tool. The proposed tool is applicable to all agile methodologies.

4. Agile Methodologies.
Agile methodologies take the advantage of object
International Journal Publishers Group (IJPG)

Oriented programming and component based software development paradigms. Commonly followed agile methodologies have been discussed below: A. Extreme Programming Beck et al [12] defines five phases of extreme Programming: Exploration, Planning, Iteration to release, Productionizing, Maintenance and death. In exploration phase user requirements are collected on story cards, planning focuses on making plan for deliverable defined through story cards, extra testing and checking is done in productionizing phase, maintenance phase deals with keeping interest of market, customer and product in connection to changing requirements where as death phase is end or complete delivery of the product. It is notable that the focus of the maintenance phase is on execution of quality assurance activities. B. Scrum Scrum focuses on self organizing teams especially in the scenarios where the environment and resources are highly meant to variant. According to Schwaber[13] Scrum follows three phases. First phase is pre-game phase which includes planning and architecture development. Second is development phase and third is post-game phase which deals with closure of release. Major focus is on carrying the activity known as Sprint in the development phase. Sprint is iterative in nature and focus is on dealing with changing demands and requirements. C. Crystal Methodologies Crystal Methodology is among recently developed methodologies. Two well known methodologies are Crystal Clear Methodology and Crystal Orange Methodology. Crystal methodologies follows incremental delivery, direct user involvement, timeliness defined with milestones, regression testing , two user reviews per release, workshop and interviews for product and methodology tuning, quality and design standards, version control strategies and compilation tools. According to CockBurn [14] each increment of crystal include the development of requirement specification document ( not detailed ) followed by the phase of staging, includes planning and scheduling, followed by several iterations development, monitoring, stability with milestones and review before the final release. D. Feature Driven development. Feature driven development focuses on design and implementation features in five steps using object oriented methodology. Palmer et al [15] defines, develop and build overall model, build feature list, plan by feature, and design by feature and build by feature as five distinct stages. Domain or context modeling is an important aspect of this methodology which helps to increase performance by clear understanding of the system. Inspection is commonly followed defect prevention strategy. Feature defines the functionalities required from the product by the customer. E. Rational Unified process (RUP) RUP has been developed to complement UML modeling while following the agile methodologies. It is an iterative paradigm based on OO (Object Oriented) Modeling. Kructhen [16] states inception, elaboration, construction and

76

International Journal of Advanced Computer Science, Vol. 2, No. 2, Pp. 73-78, Feb. 2012.

transition as major processes followed in RUP methodology. Business Modeling and environment workflows are important part of RUP methodology. The transition phase in RUP is totally dedicated to testing to ensure quality. Inception is focused on gathering requirements, elaboration on building models and construction on development of the systems based on modeling content. F. Dynamic System Development Methodology (DSDM) DSDM has been evolved to support RAD (Rapid application Development framework).DSDM is based on learn, speculate and collaborate phases. [17] DSDM is component oriented approach. It is preferably followed for large and complex systems. In speculation phase the project is instantiated and planning is done. In collaboration phase various component based methodologies are coordinated. In last phase of learning quality reviews are carried out, quality assurance is achieved and product is release. G. Open Source Software Development(OSSD) OSS is volunteer based distributed development paradigm .It is based on versioning control systems located

on geographically distributed zones. Apache Server, Perl and Linux are common examples of it. No project planning and documentation is done. However the code is tested before launching an open source platforms. Two common open source platforms are 'Sourcefourge' and 'Freshmeat' which provide free web based hosting. Sherma et al [18] proposed problem discovery, finding volunteers, solution identification, code testing and reviewing, code change view, code commit documentation and release management as commonly followed phases. H. Agile Modeling or Lean Development It is among most recently introduced methodologies. It is based on repairing acute problems in design and documentation, and support vast cultural boundaries and geographically distributed organizations [19]. I. Pragmatic Programming Pragmatic programming has no defined processes, tools and standards .It is at the liberty of the organization following some internal methods as well upon the collaboration and communication among geographically dislocated teams. Automation of tasks is preferred in

TABLE 2: EVALUATION OF METHODOLOGIES AGAINST QUALITY FACTORS.


Quality Parameters XP (extreme programming) User stories Srcum Review meetings Agile Methodologies and quality activities Crystal methodologies Feature driven development Interviews and formal Feature lists prepared meetings, printing with customers whiteboards Tuning methods OO design Versioning systems Simple design + versioning system Configuration management OO design tools Version control Version control White boards Class code ownership Compilation tools Configuration management support Incremental 4-6 weeks Iterative development Common object model Regression testing , Interface testing GUI Designs, User manuals Versioning system Common object Model Incremental delivery on regular basis Interviews and formal meetings, printing whiteboards Tuning methods Versioning systems Configuration management tools Version control White boards Compilation tools Incremental 4-6 weeks Common object model Regression testing , Interface testing GUI Designs, User manuals Versioning system Common object Model Incremental delivery regular basis Class code ownership Inspection Feature list OO design and versioning control Class code ownership Reusable design features in iterative manner Feature lists prepared with customers OO design Simple design versioning system OO design Version control Class code ownership Configuration management support Iterative development Class code ownership Inspection Feature list OO design and versioning control Class code ownership Reusable design features in iterative manner RUP Inception Interviews stakeholders OO design UML design phase. with

Correctness

Robustness Extendibility Reusability Compatibility Efficiency Portability Timeliness Integrity Verification and validation Ease of use Maintainability Performance Cost effectiveness Correctness

Generic OO design Simple design OO design Inherit in OO deign Pair programming OO design practices Incremental and iterative development OO design Unit testing Simple design Simple design Productioniz-ing phase Iterative development User stories

OO Design Prioritization Existing design OO design Team meeting OO designs Iterative OO design Testing and reviews Simple design Compatible with existing system Sprint phase Iterative development Review meetings

UML design, COTS development Inherit in OO design UML modeling Component Based architecture Iterative development UML modeling Acceptance testing UML modeling UML compliances OO design Iterative development with OO design Inception phase. Interviews with stakeholders OO design UML design UML design, COTS development Inherit in OO design UML modeling Component Based architecture Iterative development UML modeling Acceptance testing UML modeling UML compliances OO design Iterative development with OO design

Robustness Extendibility Reusability Compatibility Efficiency Portability Timeliness Integrity Verification and validation Ease of use Maintainability Performance Cost effectiveness

Generic OO design Simple design OO design Inherit in OO deign Pair programming OO design practices Incremental and iterative development OO design Unit testing Simple design Simple design Productioniz-ing phase Iterative development

OO Design Prioritization Existing design OO design Team meeting OO designs Iterative OO design Testing and reviews Simple design Compatible with existing system Sprint phase Iterative development

on

International Journal Publishers Group (IJPG)

Mehreen Sirshar et al.: Evaluation of Quality Assurance Factors in Agile Methodologies.

77

Agile methodology Extreme Programming Scrum Crystal Methodologies

TABLE 3: TESTING TECHNIQUES AND AGILE METHODOLOGIES Testing techniques Unit Testing, acceptance testing, integration testing, functional testing by tester or user, customer Feedbacks Review meetings, unit testing , functional testing, and sprint backlogs Regression testing , interface testing and repeated GUI testing

Crystal clear and crystal Orange) Feature driven Development Rational Unified Process DSDM (dynamic system Development Methodology) Open Source software development(OSSD) Lean development or agile Modeling

Inspection, formal testing and unit testing Alpha testing, Beat testing, component integration testing Regression testing, Converge testing Beta testing Beta testing, Inspection

pragmatic programming. Table 2 provides a comparison among different discussed methodologies and quality assurance factors obtained from them in different processes. Pragmatic Programming has been intentionally left blank due to its immature nature and unavailability in literature.

5. Analysis and Suggestions


Testing techniques which are suggested to be followed in particular methodology are tabulated in Table 3. Complex documentation is avoided. Automated testing techniques are preferred due to short time boxes. Beta testing, regression testing and integration testing are usually followed. Analysis of the table reveals the fact that apart from following complex and formal testing techniques simple and commonly followed techniques are used in nature. It is due to the fact that almost all of the agile methodologies are based upon design simplicity , to promote reuse and efficiency as well as they are based on Object oriented Methodologies.

6. Conclusion
This research has introduced a innovative tool for evaluation of agile methodologies against quality assurance factors. The quality management in agile methodologies is also new era of research. The limitation of tool is that is applicability in industry has not been verified. It may be due to difference in practices followed from organization to organization. The proposed tool would spark new era of research in field of agile software development especially the scenarios of agile modeling architecture for enterprise or offshore development.

References
[1] Meyer, B., Object-Oriented Software Construction, Prentice Hall PTR, pp.4-20, 2000 [2] McCall, J.A., Richards, P. K., & Walters, G. F., Factors in Software Quality, Vols. 1, 2 and 3, National Technical Information Service, 1977 [3] Pressman, R.S., Software Engineering a Practitioners Approach, Mcgraw-Hill, 2001 International Journal Publishers Group (IJPG)

[4] Sommerville, I., Software Engineering. Addison-Wesley, 2004 [5] Pfleeger, S. L., Software Engineering: Theory and Practice, Prentice Hall, 2005 [6] CockBurn, A., Agile Software Development, Boston, Addison Wisely, 2002 (a) [7] Beck, k., Beedle, M., Van Bennekum, A., CockBurn, A., Cunningham, W., Folwer, M., Gernning, J., Highsmith, j., Hunt, A., Jeffries, R., Kern, J., Maerik, B., Martin, R., Mellor, S., Schwaber, K., Sutherland, J., & Thomas, D., Manifesto for software Agile Development (22.3.2002) ,2001, http//: agileManifesto.org [8] Ambler, S., Quality in an Agile World, Software Quality Professional, Vol. 7, No. 4, pp. 34-40, 2005 [9] McBreen, P., Quality Assurance and Testing in Agile Projects, Available from: http://www.mcbreen.ab.ca/talks/CAMUG.pdf [Accessed 26 July 2006]. [10] M. Huo, J. Verner, Y. Zhu, & M.A. Babar, "Software Quality and Agile Methods," (2004) Proceedings of the 28th Annual International Computer Software and Applications Conference (COMPSAC04), IEEE Computer [11] E. Mnkandla, & B. Dwolatzky, "Defining Agile Software Quality Assurance," (2006) Proceedings of the International Conference on Software Engineering Advances (ICSEA'06), IEEE [12] K. Beck, "Extreme Programming Explained: Embrace Change, Reading Mass, Addison Wesely," (1999) IEEE Computer vol. 32, no. 10, pp. 70-77 [13] Schwaber, K., & Beedle, M., Agile Software Development with Scrum, Upper Saddle River, NJ, Prentice-Hall, 2002 [14] CockBurn, A., Writing Effective Use cases, The crystal collection for software professionals, Addison-Wisely Professional, 2000 (b) [15] Palmer, S.R., & Fesling, J.M., A practical Guide to Feature driven Development, Upper saddle River, Prentice Hall, 2002 [16] Kructhen, P., the Rational Unified Process, an Introduction, Addison-Wesely, 2000 [17] Highsmith, J. A., Adaptive software Development, a Collaborative Approach to managing complex systems, New York, NY, Dorset House Publishing, 2000 [18] S. Sharma, V. Sugumaran, & B. RajaGopalan, "A Framework for creating hybrid open source software communities," (2002) Information system Journal, vol. 12, no. 1, pp. 7-25 [19] S. Ambler, "Lessons in agility from Internet based development," (2002) International proceedings on information systems, IEEE [20] IEEE Standard Glossary of Software Engineering Terminology, IEEE Std 610, 1990

78

International Journal of Advanced Computer Science, Vol. 2, No. 2, Pp. 73-78, Feb. 2012.

Mehreen Sirshar , was born in Rawlpindi , Pakistan ,in 1985. She Received her B.S.E degree from Fatima Jinnah University ,Rawalpindi Pakistan.She is Currently Reseach Scoholar for M.S (S,E) at National University of Science and Technology Rawalpindi ,Pakistan.She is Serving as Lecturer in Software Engineering in Fatima Jinnah Women University .She is author of 3 international publications, reviewer of various international conferences and supervisor of couple of graduate degrees. She is currently conducting her Ms-Phd courses in Millitary College of Signals.(a campus of National Univeristy of Science and Technology) Rawalpindi Pakistan.She is currently conducting her research in Digital video tracking projects. Dr Fahim Arif was commissioned in Signal Corps of Pakistan Army in 1988. He has performed duties in Army units, Research and Development setup of Corps of Signals and National University of Sciences and Technology. He has served as Telecommunication Engineer in Special Communication Organization and Frontier Works Organization in southern Sindh area. He has done Bachelors in Telecommunication from Military College of Signals (UET Lahore) in 1995 and Master in Sciences in Computer Software Engineering from National University Science and Technology in 2003. He has won NUST Endowment fund scheme scholarship for NUST in 2003 and International Research Support Initiative Program Fund from HEC in 2007. He has completed his PhD degree from National University Science and Technology in 2009. His contribution to international research in recent few years is excellent. He has presented/published eight research papers in different international conferences including USA and Canada. In addition to his research publications, he is doing as reviewer for various international conferences. He work as international researcher in System and Computer Engineering Department, Carleton University, Ottawa, Canada in 2007 and participated numerous research and academic activities. Recently, his biography has been published by South Asian Publication Whos Who in the World 2008 Edition and awarded with Star Laureate 2008 in recognition to his contributions to knowledge and research. Higher Education Commission has nominated him as their official PhD supervisor in 2010. Currently, he is teaching various academic courses at MCS and supervising one PhD student and different MS/ BE students in their final projects/thesis in NUST.

International Journal Publishers Group (IJPG)