Teaching Software Engineers in Test Automation: A Case Study

Nguyen Thanh Binh Dean of IT Faculty, Danang University of Technology Truong Anh Tuan Sr. Director, Software Division, LogiGear Vietnam

Contents
Context Needs in test automation Test automation training program Case study Thoughts for Vietnam Conclusion

Context
The application development software market is expected to grow

from an estimated $9.05 billion in 2012 to more than $10.28 billion in 2016
(Based on Gartners Report Market Trends: Application Development, Worldwide, 2012-2016)

According to a CIO survey

finding and fixing bugs account for almost 50 percent of the total cost

of software projects Offshoring of QA & Testing functions can save companies up to 75 percent compared to in-house costs
(Based on Why Outsource Testing TEAM International)

Context
Vietnam is one of the top countries in the world that is very

attractive in offshore services Vietnam is one of the top 9 in Asia and top 30 nations in the world in 2010-2011
(source: Gartner Identifies Top 30 Countries for Offshore Services in 2010-2011)

2013 top 100 outsourcing cities HCM city: 16th rank Hanoi: 23th rank
(source: 2013 Top 100 Outsourcing Destinations - Tholons 2013)

Ho Chi Minh City and Ha Noi ranked 5th and 10th respectively in

the world in the list of Top 50 Software Testing Cities

(Based on Tholons and Global Service Medias study, 2010)

Training program is essential

1. The key competitive advantages of Viet Nam currently are cost and young labor pool 2. A well-designed education program, targeted in software testing, will equip the labor pool with skills to compete globally

Test engineer hiring

High demands for test engineers by many outsourcing companies

in Vietnam
based on vietnamwork.com

But lack of resources

no testing skills Not enough experienced testers

Focus more on hiring test automation engineers

Reduce testing cost and time, improve quality and test coverage

But more difficult in hiring test automation engineers

Why hiring is in trouble?

Test engineer is not considered a definitive profession Test engineer jobs are not very attractive
Testing is often viewed as undesirable and unchallenging career

Lack of training programs about testing

Universities May or may not have testing courses Companies On-the-job training according to clients projects

Software Engineering program

Soft skills English Hard skills
Fundamental courses programming, algorithm, databases, operating system, network, hardware Specialty courses project management, open source development, SOA, e-commerce, software process, formal specification Software Testing

To train mostly software developers

Software Testing course

Covers both theoretical and practical aspects Students understand and participate in test activities
Analyze requirements of document Write a test plan Design, create and execute test cases using various testing

techniques Write bug and test reports

Software Testing training

Exists at some universities in Vietnam

HCM City University of Technology University of Information Technology, HCM City National University HCM City University of Science Danang University of Technology

More theoretical versus practical aspects Lack of test automation skills

Students may think that testing jobs are only manual Unchallenging, undesirable

What to do?
Complete software testing training with test automation
Talk later about our case study

Need to define clearly - careers for Test Engineers

Test engineer (manual)

Test automation engineer

Test engineer skills

Manual test engineer
Basic software engineering skills general knowledge of computers, software systems; software development methods; requirements analysis; operating systems, databases Soft skills Oral and written presentations of technical documents Problem solving, time management Testing skills Test planning, test requirement analysis, test case design, test execution, bug report, test report

Test automation engineer skills

Test automation engineer
The same skills as manual test engineer Test automation skills Uses and implement automation tools Create and maintain test automation Good scripting skills implementing test scripts

Career path for test engineers

Software testing sometimes is viewed as a dead-end job Skilled test engineers move up and become developers,

analysts, designers, system architects

Test engineer is viewed as a stepping stone

Each organization should define a career path for test

engineer
more defined, challenging and varied career

Career path for test engineers

For example:
(*) Engineering Manager

LogiGear test engineer career path

Sr. Engineering Manager (*)

Principal Test Consultant

Sr. Project Manager

TE C H NI C AL T R A C K

Project Manager
(*) Test Consultant

(*)
Sr. Project Lead

Sr. Test Engineer

Project Lead

M A N A G E M E N T T R A C K

(*) Test Engineer

Career path for test engineers

For example:
AT&T test engineer career path

Test Automation Training Contents

PART I: SOFTWARE TESTING FUNDAMENTALS Chapter 1: Software Development Life Cycle and Testing Chapter 2: Software Testing Overview Chapter 3: Test Requirement and Design Chapter 4: Software Error and Bug PART II: TEST AUTOMATION Chapter 5: Test Automation Overview Chapter 6: Keyword-driven & Action Based Testing Chapter 7: An Open source tool : Selenium Chapter 8: A Commercial Tool: TestArchitect Chapter 9: Automation customization

Test Automation Training Objectives

1. 2. 3. 4.

5.

Review fundamentals of practical software testing and test automation Recognize the importance of test automation within the scope of software testing Provide access to state-of-the-art in test automation: methodologies and technologies Can use automation tools to implement simple tests in practical settings Encourage and stimulate the interest of test automation as a good career path in the future

Test Automation Training Objectives

Curriculum may be slightly adjusted to align with the program at universities but needs to balance between theory and practice hours
Theory Practice

Software Development Life Cycles and Testing Software Testing Overview Test Requirement & Test Design Software Error and Bug Overview of Test Automation Keyword-Driven and Action-Based Testing Commercial and Open-Source Automation Testing Tools Using Scripting Languages to customize Test Automation

4 4 4 4 4 4 4 4 32 8 4 28 4 4 8

Practice and Evaluation

Individual 10 exercises (maximum 2 students/PC) on:

TestArchitect Selenium BugZilla Eclipse, Test NG, Python bug-injected application samples

Teamwork on a major exercise on a small real test automation

project during the course. Rate of evaluation:

Major Exercise : 30% Mid-term score: 20% Final score: 50% (theory; multiple-choice)

Example of actual exercise

Below is your assignment today: - Install Firefox 6, Firebug, FirePath, Selenium IDE for Firefox - Install Eclipse, TestNG - Create Java project MusicProject - For today exercise, please follow the instruction in Chapter 5 - Exercise - Selenium.ppt and How To Capture Interface - Selenium.ppt Expected production today: 1. Using Firebug, Firepath 2. Using Selenium IDE in Firefox to record test script Navigate to http://www.nhaccuatui.com/ Enter value for search textbox (e.g: remember when) Click search button Check search result (check first result in Top bi ht) 3. Implement below test case by using selenium 2.0 (WebDriver), java and TestNG Navigate to http://www.nhaccuatui.com/ Enter value for search textbox Click search button Check search result (check first result in Top bi ht)

Case study:
Place : Da Nang University of technology Time : 01- 23 Oct 2012 Students: 151 (last year) Number of credits: 3 Theory hours: 30 Practical hours: 28 Passed: 100%

Achievements
Only 18 absenteeism during 2 weeks of training

Most students are interested in the course

100% students completed exercises 90% project teams have done well on major exercise

(the rest did test design, test case implementation, but could not completed the full automation run)
Final test: 100% passed, 80% > 7/10

Student feedbacks
Here are actual end of course written feedbacks from students (in Vietnamese)
C th thm nhiu kha hc nh th ny vo chng trnh nh hng thm v

ngh nghip cho sinh vin khi ra trng Thi gian cho kha hc qu t, ch c 1 tun/kha Cn thm nhiu v d v bi hc.Thi gian ngn m kin thc nhiu nn kh tip thu ht Cn ci thin thm trang thit b nh my mc, microphone, projector Lp hc qu nhiu sinh vin. Nn chia nh ra Ging vin cn ni to hn Cn thm lp hc v k nng mm Cn c thm bi tp v nh sinh vin c th thc hnh thm Nn lng vo bi ging nhiu hn v kinh nghim thc t v hc tp, cng vic, cuc sng, phong cch lm vic Cn gii thiu r hn v cng vic ca test automation engineer

Student feedbacks (cont.)

C th thm nhiu kha hc nh th ny vo chng trnh nh hng thm v ngh nghip cho sinh vin khi ra trng Cn gii thiu thm cc cng c khc v test automation Nn to 1 d n mu cho sinh vin d hnh dung Cn thm ngi h tr sinh vin thc hnh Nn tng thm thi gian thc hnh Nn cung cp ti liu cho sinh vin trc khi ging dy Kha hc b ch, c tnh thc tin cao, p ng c mong i ca bn thn sinh vin Ging vin vui tnh, ging k cng, su v d hiu H tr tn tm, nhit tnh trong lc ging dy, b tr thi gian ging dy hp l Hy vng nm sau thy Tun tr li trng BK ging dy tip

Thoughts for Vietnam

Number of universities/colleges having IT training program

Source: Ministry of Information and Communication, 2010

Thoughts for Vietnam

Number of recruited students in IT per year (12% in 2010)

Source: Ministry of Information and Communication, 2010

Thoughts for Vietnam

Survey from www.pcworld.com.vn in 2010: IT is a hot

job in Vietnam
6.07

5.89
4.29

IT - Communication
7.86 40.89 Finance - Bank Management Foreign languages Law Tourism - Hotel - Restaurant

16.25

Others

18.75

Thoughts for Vietnam

Labor pool have very good potential
Low cost and young Mostly trained in software development Fundamental IT Programming skills Training supplement Soft skills Software testing Test automation

Thoughts for Vietnam

Case study at LogiGear
Halliburton project Started with a small team of three test automation engineers and a part-time project manager Defects found in Beta 2009: 222 2010: 36 REDUCTION: 84% Known issues at shipment 2009: 104 2010: 3 REDUCTION: 97% After 2 years, Halliburton have expanded to over 20 divisions at LogiGear with 3-15 test automation engineers per division

Conclusion
The time for test automation is now
Reduce cost and improve productivity

Train from test automation to software engineers Define career path for test engineers
Attract highly qualified people to a career in software testing

THANK YOU -